PaperClipAI

Jannik/PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-11-10 01:50:13 +01:00

Commit Graph

Author	SHA1	Message	Date
DottaandPaperclip	58308a3e63	perf(apps): scope sidebar review counts to one connection (#18688 ) ## Thinking Path > - Paperclip helps people manage AI agents and their governed app access. > - The Apps sidebar displays a Review badge for one connection. > - It polls the full company attention service every 30 seconds to get two counts. > - That service also loads every profile, rule and active catalog entry. > - This pull request adds an authorized connection read for only those counts. > - The same badge now avoids unrelated profile evaluation and database result decoding. ## Linked Issues or Issue Description Current behavior The contextual sidebar polls company-wide attention. It needs only one connection's signed pending action count and whether any catalog entries are quarantined. Proposed behavior Both sidebar variants poll a connection-scoped count endpoint with existing connection read authority. The full attention API stays unchanged. Reason and benefit The retained 200-profile fixture performs seven service queries and returns 80,115 rows for this badge. The new reader performs one query and returns one row. This reduces local service CPU and avoids unnecessary decoded profile/rule data. Related merged evaluator fixes are already in the baseline: Refs #18673, Refs #18681. A search found no existing PR for this scoped count reader. ROADMAP.md contains no duplicate planned work. ## What Changed - Add the narrow shared DTO, typed client, guarded REST/OpenAPI route and catalog-owned reader. - Count independent aggregates with both company boundaries and exact signed-argument semantics. Keep archived counts at zero. - Update both contextual sidebar variants. Keep the exact badge, 30 second interval and existing attention-prefix invalidation. - Add PostgreSQL differential tests, authenticated HTTP tests, sidebar tests, a repeatable benchmark and access documentation. ## Verification - Baseline: `3f8ebfff8aa9febd0733c6f65b07485dcadfd0e2`, including both merged evaluator fixes. Service treatment: `0827f7d0b3`; route/fixture refinement: `230c42ee15`; final local existence gate: `390a218254`. - Passed: focused PostgreSQL differential test; assembled HTTP test with board session and agent JWT; 14 sidebar component tests; token gates; map checks; diff check. - Fixture: 200 profiles, 76,000 rules, 1,900 active tools, 695 removed references, 12 agents, a sibling connection and 2,000 unrelated historical invocations. Complete normalized two-count results are equal on both paths. - Quiet local measurement: seven alternating warm pairs after two warmup pairs, Node 24.22.2, macOS, real disposable PostgreSQL. Legacy/new service CPU median: 213.891/1.680 ms; ranges: 209.329–228.552/0.515–6.137 ms. Wall median: 156.867/1.248 ms; ranges: 151.739–162.289/1.115–1.386 ms. Queries: 7/1. Returned rows: 80,115/1. - Run `cd server && node --expose-gc --import ./node_modules/tsx/dist/loader.mjs scripts/benchmark-connection-review-counts.ts`. See `doc/connection-review-counts.md` for scope and failure history. Byte-volume accounting runs outside timing; no HTTP payload reduction is claimed. - Initial raw scalar SQL failed with ambiguous columns. Typed subqueries fixed it and passed real PostgreSQL equality. Initial fixture agents lacked permissions and correctly received 403. The final fixture grants explicit authority and tests revocation. An initial benchmark included byte serialization in timing; the reported run excludes it. - Passed required local `pnpm -r typecheck` and `pnpm build`. The initial sandbox typecheck could not open the tsx IPC socket; the escalated rerun passed. Full monolithic `pnpm test:run` was not run locally: repeating a 15–25 minute suite serially across both performance PRs adds no distinct evidence. Broad suite acceptance uses final-head CI, which runs the same general/serialized test partitions and the native Runner lane. Final-head CI is in progress. - Passed built-static Chromium journey on an owned local-trusted server and disposable PostgreSQL: Review 3 appears, only scoped counts are requested by this sidebar, resolving requests/clearing quarantines then reloading removes the badge, and archived counts are zero. The first ad hoc launcher omitted required decision-service options and failed at startup; using the normal no-op wake callback fixed that harness. - Greptile identified an invalid resolved-request fixture label. Corrected `denied` to the declared `rejected`, then reran the real PostgreSQL differential test. The count SQL and non-pending exclusion are unchanged. - The first final-head CI failed the static cross-tenant existence-oracle guard because the new explicit resource company assertion had no nearby `hasCompanyAccess` gate. Existing management admission already rejects foreign resources with 404. The route now also follows the standard local two-step gate. The unchanged guard plus authenticated HTTP negative cases passed (3 tests). No allowlist or test was weakened. - These local service measurements do not prove hosted performance. No deployment is included. ## Risks Low risk. New read admission reuses existing current-authority checks. The old attention API is unchanged. Counts still require database work over relevant pending/catalog rows. No cache, schema migration or telemetry is added. Roll back this PR if the badge differs or scoped reads deny previously permitted users. ## Model Used OpenAI `gpt-6.1-sol` (Sol 6.1), with reasoning, tool use and code execution. The host does not expose the configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 17:12:22 -06:00
DottaandPaperclip	b82661b561	refactor(connections): retire the legacy Composio broker (#13758 ) ## Thinking Path > - Paperclip manages agents and their access to external tools. > - Connectors expose these tools through a governed MCP gateway. > - PR #13755 added a direct Composio MCP connection behind the experimental MCP aggregators flag. > - The old project API-key broker still created toolkit child connections and showed a separate Services tab. > - Keeping both paths leaves obsolete setup and session code in the product. > - This change removes the broker and preserves direct MCP setup, credentials, permissions, and execution. > - Saved legacy records fail closed and remain available for explicit removal. ## Linked Issues or Issue Description Related: #13755. This retirement supersedes the legacy-path fixes proposed in #12630, #12632, #12634, and #12906. It does not close those PRs. What existing behavior does this improve? Composio connector setup, management, and runtime dispatch. Current behavior Composio offers both direct MCP and a project API-key broker. The broker mints sessions and creates one child connection per toolkit. Proposed behavior Offer only direct MCP. Remove the toolkit Services UI, REST routes, API client, and session broker. Block saved legacy parent and child records from discovery, execution, health checks, reconnect, and OAuth. Preserve their records and credentials until the operator removes each connection. Reason and benefit The direct MCP connector becomes the single supported Composio workflow. Provider accounts remain managed in Composio. ## What Changed - Remove the API-key catalog method and its generated-source definition. - Delete Composio broker clients, session creation, account synchronization, child lifecycle, and toolkit routes. - Remove the Services tab, service rows, child provenance, and cascade-removal controls. Keep Vercel provenance intact. - Retain a shared retirement guard for stored legacy records. Show Retired status and replacement/removal guidance in the connection list and details; hide obsolete runtime controls. - Preserve the experimental MCP aggregators flag and direct MCP infrastructure. - Replace broker fixtures with retirement tests and extend direct Composio catalog/reconnect coverage. ## Verification - Focused shared, server, and UI tests passed with one worker. Server retirement tests use a name filter; no full local test suite was run, as requested. - Server and UI TypeScript checks passed. - Token gates and UI build passed. - Real browser: opened the saved Composio connection, refreshed all 11 tools, and ran the provider's read-only GitHub account-list operation through the standard Test dialog as an agent. The provider returned success using the existing OAuth credentials. - See `doc/connections/COMPOSIO-BROKER-RETIREMENT.md` for scope and live evidence. - Storybook build passed. A fresh real agent used `COMPOSIO_SEARCH_TOOLS` and `COMPOSIO_MULTI_EXECUTE_TOOL` to return the actual Paperclip DeepWiki hierarchy: one success, zero errors. Gateway audit records confirm both calls succeeded. - Browser retirement check: a credential-free legacy fixture showed the guidance, opened the direct MCP replacement flow, and was removed through the standard confirmation. - Focused regressions for the experimental settings copy and exact OpenAPI route coverage passed. All latest-head CI checks passed (54 successful, two intentionally skipped); Greptile scored 5/5 with no unresolved review threads. The PR has no merge conflicts. ## Risks This intentionally breaks the old Composio project API-key and child-connection workflow. Existing legacy records cannot run, even if their stored status is active. Operators must create a new direct MCP connection and choose access rules; credentials and grants are not migrated. Remove each old record separately to delete its credentials. No schema migration or data deletion runs automatically. Direct MCP connections keep their existing grants and secrets. ## Model Used OpenAI GPT-6 via Codex, with reasoning, code execution, and browser tools. The exact runtime variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-21 13:57:35 -05:00
scotttong	597fd63b61	feat(ui): add streamlined navigation foundation (#12746 )	2026-09-02 23:55:43 -07:00

Author

SHA1

Message

Date

DottaandPaperclip

58308a3e63

perf(apps): scope sidebar review counts to one connection (#18688 )

## Thinking Path

> - Paperclip helps people manage AI agents and their governed app
access.
> - The Apps sidebar displays a Review badge for one connection.
> - It polls the full company attention service every 30 seconds to get
two counts.
> - That service also loads every profile, rule and active catalog
entry.
> - This pull request adds an authorized connection read for only those
counts.
> - The same badge now avoids unrelated profile evaluation and database
result decoding.

## Linked Issues or Issue Description

**Current behavior**

The contextual sidebar polls company-wide attention. It needs only one
connection's signed pending action count and whether any catalog entries
are quarantined.

**Proposed behavior**

Both sidebar variants poll a connection-scoped count endpoint with
existing connection read authority. The full attention API stays
unchanged.

**Reason and benefit**

The retained 200-profile fixture performs seven service queries and
returns 80,115 rows for this badge. The new reader performs one query
and returns one row. This reduces local service CPU and avoids
unnecessary decoded profile/rule data. Related merged evaluator fixes
are already in the baseline: Refs #18673, Refs #18681. A search found no
existing PR for this scoped count reader. ROADMAP.md contains no
duplicate planned work.

## What Changed

- Add the narrow shared DTO, typed client, guarded REST/OpenAPI route
and catalog-owned reader.
- Count independent aggregates with both company boundaries and exact
signed-argument semantics. Keep archived counts at zero.
- Update both contextual sidebar variants. Keep the exact badge, 30
second interval and existing attention-prefix invalidation.
- Add PostgreSQL differential tests, authenticated HTTP tests, sidebar
tests, a repeatable benchmark and access documentation.

## Verification

- Baseline: `3f8ebfff8aa9febd0733c6f65b07485dcadfd0e2`, including both
merged evaluator fixes. Service treatment: `0827f7d0b3`; route/fixture
refinement: `230c42ee15`; final local existence gate: `390a218254`.
- Passed: focused PostgreSQL differential test; assembled HTTP test with
board session and agent JWT; 14 sidebar component tests; token gates;
map checks; diff check.
- Fixture: 200 profiles, 76,000 rules, 1,900 active tools, 695 removed
references, 12 agents, a sibling connection and 2,000 unrelated
historical invocations. Complete normalized two-count results are equal
on both paths.
- Quiet local measurement: seven alternating warm pairs after two warmup
pairs, Node 24.22.2, macOS, real disposable PostgreSQL. Legacy/new
service CPU median: 213.891/1.680 ms; ranges:
209.329–228.552/0.515–6.137 ms. Wall median: 156.867/1.248 ms; ranges:
151.739–162.289/1.115–1.386 ms. Queries: 7/1. Returned rows: 80,115/1.
- Run `cd server && node --expose-gc --import
./node_modules/tsx/dist/loader.mjs
scripts/benchmark-connection-review-counts.ts`. See
`doc/connection-review-counts.md` for scope and failure history.
Byte-volume accounting runs outside timing; no HTTP payload reduction is
claimed.
- Initial raw scalar SQL failed with ambiguous columns. Typed subqueries
fixed it and passed real PostgreSQL equality. Initial fixture agents
lacked permissions and correctly received 403. The final fixture grants
explicit authority and tests revocation. An initial benchmark included
byte serialization in timing; the reported run excludes it.
- Passed required local `pnpm -r typecheck` and `pnpm build`. The
initial sandbox typecheck could not open the tsx IPC socket; the
escalated rerun passed. Full monolithic `pnpm test:run` was not run
locally: repeating a 15–25 minute suite serially across both performance
PRs adds no distinct evidence. Broad suite acceptance uses final-head
CI, which runs the same general/serialized test partitions and the
native Runner lane. Final-head CI is in progress.
- Passed built-static Chromium journey on an owned local-trusted server
and disposable PostgreSQL: Review 3 appears, only scoped counts are
requested by this sidebar, resolving requests/clearing quarantines then
reloading removes the badge, and archived counts are zero. The first ad
hoc launcher omitted required decision-service options and failed at
startup; using the normal no-op wake callback fixed that harness.
- Greptile identified an invalid resolved-request fixture label.
Corrected `denied` to the declared `rejected`, then reran the real
PostgreSQL differential test. The count SQL and non-pending exclusion
are unchanged.
- The first final-head CI failed the static cross-tenant
existence-oracle guard because the new explicit resource company
assertion had no nearby `hasCompanyAccess` gate. Existing management
admission already rejects foreign resources with 404. The route now also
follows the standard local two-step gate. The unchanged guard plus
authenticated HTTP negative cases passed (3 tests). No allowlist or test
was weakened.
- These local service measurements do not prove hosted performance. No
deployment is included.

## Risks

Low risk. New read admission reuses existing current-authority checks.
The old attention API is unchanged. Counts still require database work
over relevant pending/catalog rows. No cache, schema migration or
telemetry is added. Roll back this PR if the badge differs or scoped
reads deny previously permitted users.

## Model Used

OpenAI `gpt-6.1-sol` (Sol 6.1), with reasoning, tool use and code
execution. The host does not expose the configured context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

2026-11-02 17:12:22 -06:00

DottaandPaperclip

b82661b561

refactor(connections): retire the legacy Composio broker (#13758 )

## Thinking Path

> - Paperclip manages agents and their access to external tools.
> - Connectors expose these tools through a governed MCP gateway.
> - PR #13755 added a direct Composio MCP connection behind the
experimental MCP aggregators flag.
> - The old project API-key broker still created toolkit child
connections and showed a separate Services tab.
> - Keeping both paths leaves obsolete setup and session code in the
product.
> - This change removes the broker and preserves direct MCP setup,
credentials, permissions, and execution.
> - Saved legacy records fail closed and remain available for explicit
removal.

## Linked Issues or Issue Description

Related: #13755. This retirement supersedes the legacy-path fixes
proposed in #12630, #12632, #12634, and #12906. It does not close those
PRs.

**What existing behavior does this improve?**

Composio connector setup, management, and runtime dispatch.

**Current behavior**

Composio offers both direct MCP and a project API-key broker. The broker
mints sessions and creates one child connection per toolkit.

**Proposed behavior**

Offer only direct MCP. Remove the toolkit Services UI, REST routes, API
client, and session broker. Block saved legacy parent and child records
from discovery, execution, health checks, reconnect, and OAuth. Preserve
their records and credentials until the operator removes each
connection.

**Reason and benefit**

The direct MCP connector becomes the single supported Composio workflow.
Provider accounts remain managed in Composio.

## What Changed

- Remove the API-key catalog method and its generated-source definition.
- Delete Composio broker clients, session creation, account
synchronization, child lifecycle, and toolkit routes.
- Remove the Services tab, service rows, child provenance, and
cascade-removal controls. Keep Vercel provenance intact.
- Retain a shared retirement guard for stored legacy records. Show
Retired status and replacement/removal guidance in the connection list
and details; hide obsolete runtime controls.
- Preserve the experimental MCP aggregators flag and direct MCP
infrastructure.
- Replace broker fixtures with retirement tests and extend direct
Composio catalog/reconnect coverage.

## Verification

- Focused shared, server, and UI tests passed with one worker. Server
retirement tests use a name filter; no full local test suite was run, as
requested.
- Server and UI TypeScript checks passed.
- Token gates and UI build passed.
- Real browser: opened the saved Composio connection, refreshed all 11
tools, and ran the provider's read-only GitHub account-list operation
through the standard Test dialog as an agent. The provider returned
success using the existing OAuth credentials.
- See `doc/connections/COMPOSIO-BROKER-RETIREMENT.md` for scope and live
evidence.
- Storybook build passed. A fresh real agent used
`COMPOSIO_SEARCH_TOOLS` and `COMPOSIO_MULTI_EXECUTE_TOOL` to return the
actual Paperclip DeepWiki hierarchy: one success, zero errors. Gateway
audit records confirm both calls succeeded.
- Browser retirement check: a credential-free legacy fixture showed the
guidance, opened the direct MCP replacement flow, and was removed
through the standard confirmation.
- Focused regressions for the experimental settings copy and exact
OpenAPI route coverage passed. All latest-head CI checks passed (54
successful, two intentionally skipped); Greptile scored 5/5 with no
unresolved review threads. The PR has no merge conflicts.

## Risks

This intentionally breaks the old Composio project API-key and
child-connection workflow. Existing legacy records cannot run, even if
their stored status is active. Operators must create a new direct MCP
connection and choose access rules; credentials and grants are not
migrated. Remove each old record separately to delete its credentials.
No schema migration or data deletion runs automatically. Direct MCP
connections keep their existing grants and secrets.

## Model Used

OpenAI GPT-6 via Codex, with reasoning, code execution, and browser
tools. The exact runtime variant and context-window size are not exposed
in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

2026-09-21 13:57:35 -05:00

scotttong

597fd63b61

feat(ui): add streamlined navigation foundation (#12746 )

2026-09-02 23:55:43 -07:00