mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-07 07:55:47 +01:00
4a9974ea03ddce3aa1e83000fe39903f7980b5b9
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
d0ffa179b4 |
perf(tools): index profile selectors per resolution (#18673)
## Thinking Path > - Paperclip manages AI agents and their work. > - Core resolves tool profiles before it prepares runtime MCP assignments. > - A selected profile can contain a rule for each individual tool. > - The resolver and profile summary scan these rules for each catalog row. > - This repeats millions of exact comparisons for a large selected profile. > - This pull request indexes each selector kind once per computation. > - The benefit is less server CPU with the same current access decisions and result contract. ## Linked Issues or Issue Description **What existing behavior does this improve?** Effective tool assignment and profile summary computation for large selected profiles. **Current behavior** Each catalog row scans the profile's includes and excludes. The work grows with catalog size times rule count. The runtime assignment path also computes a profile summary with the same scans. **Proposed behavior** Build five selector sets for each include or exclude group. Match catalog rows with exact set membership. Retain all current database reads and the full result contract. **Reason and benefit** Reduce repeated JavaScript comparisons. On the synthetic PostgreSQL fixture below, median process CPU per resolution falls from 50.068 ms to 10.129 ms. This is independent of [#18371](https://github.com/paperclipai/paperclip/pull/18371), which removes unused administrative results and narrows instruction descriptors. This PR retains those results and does not include that change. ## What Changed - Index application, connection, catalog-entry, tool-name, and risk selectors in the profile owner. - Use indexed matching in effective access resolution and profile summaries. - Add a differential test against the original single-rule matcher. Cover null applications, exact special-character names, unknown selectors, and empty rule groups. - Add a PostgreSQL regression for per-profile exclusion precedence, the union of multiple effective profiles, and current rule removal. - Add a real PostgreSQL resolver benchmark and document reproduction and limits. ## Verification Baseline: `410b0e27015495ab8951768b5b0b94727de33785`. Treatment: `895ee2eec15cf4cdedee4c257fcdddb2a7000a1b`. Node 24.22.2 on macOS arm64. Source modules run with `tsx`; these are local service measurements, not production-build or hosted acceptance. The real owning resolver ran against disposable PostgreSQL with 1,890 active catalog tools, 1,575 individual include/exclude rules, one deny-by-default effective profile, and 12 agents. The installation dependency returns an empty list because the fixture has no installed connections. There are no provider calls. Full returned assignments, profile summaries, rules, bindings, and names are identical to baseline. After two warmups per version, seven alternating samples ran ten resolutions each. Other workers paused CPU measurements and builds. Unrelated host work was not controlled. | Per resolution | Baseline median (range) | Change median (range) | | --- | --- | --- | | Process CPU | 50.068 ms (46.380–58.880) | 10.129 ms (8.089–11.072) | | Elapsed time | 50.362 ms (44.799–106.408) | 8.853 ms (7.719–18.590) | | Queries | 6 | 6 | | Returned database rows | 3,480 | 3,480 | Median process CPU falls 79.8%; median elapsed time falls 82.4%. The long baseline elapsed sample had no matching process CPU increase. Query and row counts do not decrease. Small or broadly allowed profiles can save less. This does not establish whole-run latency or hosted performance. Reproduce with `server/scripts/benchmark-tool-profile-matching.ts`; baseline module preparation is in `doc/mcp-discovery-performance.md`. Passed on the treatment source: ```sh node node_modules/vitest/vitest.mjs run server/src/__tests__/tool-profile-matching.test.ts server/src/__tests__/tool-effective-assignments.test.ts server/src/services/native-runtime/runtime-context.test.ts server/src/__tests__/heartbeat-runtime-mcp-servers.test.ts ``` All 45 tests in four suites passed. `git diff --check` passed. The first PostgreSQL test attempt failed at setup because the sandbox blocked its loopback listener (`EPERM`); the authorized unsandboxed retry passed. No behavior failure was discarded. At measured head `895ee2eec15cf4cdedee4c257fcdddb2a7000a1b`, CI typecheck, build, canary, Runner checks, and all eight browser shards passed. Server general shard 8/12 failed an unchanged background-quietness assertion in `heartbeat-run-dispatch-work.test.ts`: expected 235 select calls, observed 236 after advancing fake timers. Its cause remains unverified; no baseline reproduction was run. That head also failed the module-boundary policy because the benchmark inserted synthetic agents outside the test fixture tree. Follow-up head `e20165b6eb` moves the fixture body byte-for-byte into `server/src/__tests__/helpers/tool-profile-matching-fixture.ts`, following the existing listing benchmark pattern. The production matcher and benchmark measurement loop are unchanged. `node scripts/check-module-boundaries.mjs`, a Node 24/tsx fixture import check, and `git diff --check` pass. The benchmark was not rerun for this extraction. New-head CI and review are pending. Greptile gave the measured head 5/5 with no actionable issues. No deployment or live runtime mutation was performed. ## Risks Exact equality must remain identical to the original matcher, especially for null application selectors and exclusions. The differential and PostgreSQL tests cover these cases. Sets add linear temporary memory in rule count. They live only for the current computation. Current company-scoped state reads, profile precedence, revocation, and runtime assignment digest checks remain in place. Revert this commit to roll back. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with source inspection, code execution, tests, and local PostgreSQL benchmarking. The context window size is not available in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
482706e7dd |
perf(tools): index pending profile review scopes (#18681)
## Thinking Path > - Paperclip manages AI agents and their work. > - The Apps UI lists tool profiles and shows new tools that need review. > - Both services project new-tool review for each profile. > - The projection scans include rules and exact rules for each catalog row. > - This repeats millions of comparisons when many selected profiles exist. > - This pull request builds scope and selector indexes once per projection. > - The benefit is less server CPU with the same fresh review results. ## Linked Issues or Issue Description **What existing behavior does this improve?** New-tool review projection in profile listings, profile review, and app attention. **Current behavior** Each newer catalog row scans the profile's included scopes. Included catalog entries require a map lookup for each scope comparison. Rows in scope then scan all rules again to omit tools that were already reviewed. Profile listing and app attention repeat this work for every profile. **Proposed behavior** Index covered application and connection scopes once. Expand included catalog entries to their existing connection and non-null application. Index exact selectors across all rules once. Keep the ordered catalog filter and returned metadata. **Reason and benefit** A recent main-thread profile identified this scope-matching path as a large CPU consumer. The local reproduction below confirms multi-second CPU work in the actual owning services. This PR removes the repeated scans without caching authorization or changing database reads. Related [#18673](https://github.com/paperclipai/paperclip/pull/18673) indexes effective access and profile summaries. It leaves this pending-review path unchanged. This PR targets the separate pending-review mechanism and has no dependency on that PR. The selector helper is identical so the owners can share it when both changes land. ## What Changed - Build covered application and connection sets for included scopes. - Build exact selector sets for all rules, including exclusions, to omit already reviewed tools. - Preserve null application selectors, missing catalog references, review watermarks, status checks, output order, and names. - Add differential tests against the former scope/matching evaluator. - Add PostgreSQL checks through all three affected service methods, including current catalog and rule changes. - Add a reproducible PostgreSQL benchmark of complete profile-list and app-attention results. ## Verification Baseline: `1f8f2e195f4897cd1c237bdc068de95780ab0c3c`. Treatment: `fd59c0183ce64c44696f79604516f4d51ca5662b`. Node 24.22.2 on macOS arm64. The production projection file at this baseline is identical to the file at `410b0e27015495ab8951768b5b0b94727de33785`. The benchmark exercises the real `createToolProfileService(...).listProfiles` and `createToolAppAttention(...)` methods against disposable PostgreSQL. It uses 200 selected profiles, 1,900 active tools, 76,000 rules, and 12 agents. Profiles alternate between 65 and 695 rules. Half reference 695 removed catalog entries; half select current tools. This tests both the slow failed scope scans and the slow exact-rule scans. No provider calls or hosted traffic are generated. The complete ordered results from both services match baseline. After one equality/warmup invocation per version, five samples alternate implementation order. Forced GC runs before each measured pair of service calls. Other workers paused owned builds and benchmarks; unrelated host activity was not controlled. | One profile listing plus one attention listing | Baseline median (range) | Change median (range) | | --- | --- | --- | | Process CPU | 5,406.512 ms (5,221.533–5,648.262) | 1,770.116 ms (1,596.232–2,045.617) | | Elapsed time | 5,279.231 ms (4,932.419–5,537.105) | 1,532.414 ms (1,499.199–1,610.428) | | Queries | 14 | 14 | | Returned database rows | 156,215 | 156,215 | Median process CPU falls **67.3%** and median elapsed time falls **71.0%**. Profile summaries retain their old nested matching in this comparison, so the result isolates pending-review indexing. Query counts and row counts do not decrease. This is local source/tsx service proof. It does not establish hosted improvement, whole-run latency, or HTTP tail latency. Reproduce with `server/scripts/benchmark-profile-review-projection.ts`; baseline-module setup is documented in `doc/mcp-discovery-performance.md`. Passed: ```sh node node_modules/vitest/vitest.mjs run server/src/__tests__/tool-profile-pending-projection.test.ts server/src/__tests__/tool-profile-review-service.test.ts server/src/__tests__/tool-profile-entry-order.test.ts node scripts/check-module-boundaries.mjs ``` All five tests in three suites passed. The differential tests cover all selector kinds, include/exclude effects, null scope behavior, missing references, review watermarks, and exact special-character names. PostgreSQL checks preserve current pending counts after exclusions, catalog removal, and review; another company remains separate. `git diff --check` passes. The first new integration assertion used `summary` instead of the existing attention response's `totals` field. It failed before the mutation checks. The test assertion was corrected; no production code changed for that correction. The final run above passed. Server typecheck was attempted but did not pass local setup. The packaged command did not reach TypeScript because a global pnpm child tried to fetch and verify the project's pnpm version through an unavailable registry. A direct Node 24 TypeScript invocation then reported missing built `@paperclipai/paperclip-runner` and related package declarations, with cascading type errors in unchanged consumers. No error points to the changed projection. The prerequisite build was not expanded into a full workspace build. Full local unsharded tests and build were not run; final-head CI must supply broad acceptance. No merge or deployment was performed. ## Risks Scope expansion must preserve existing semantics. An individual catalog rule covers its connection and only a non-null application. An explicit null application selector still matches null applications. Removed references contribute no scope when absent from the supplied catalog. The oracle and PostgreSQL tests cover these boundaries. The indexes add linear temporary memory in rule count and live only for one projection. Current company-scoped database reads and review mutations remain in their owners. Exclusions remain explicitly reviewed; they do not create new review items. Revert this commit to roll back. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with source inspection, code execution, tests, and local PostgreSQL benchmarking. The context window size is not available in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f1d7808382 |
fix(tools): stabilize tied profile entry order (#18427)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Advanced YAML editor reads shared tool profiles through their owning service. > - The editor must preserve hidden settings and reject real concurrent changes. > - Batch-created entries share a creation time. Queries sorted only by that time can return the same entries in a different order. > - The editor can then report a false concurrent tool change before it sends a profile binding request. > - This pull request adds an ID tie-breaker to profile entry reads. > - The benefit is a stable saved configuration without removing the concurrent-change guard. ## Linked Issues or Issue Description Refs #18356. Related: #18168. **What happened?** The compact YAML browser case failed at its second save. Save stayed enabled and the profile binding write did not occur. The original CI artifact has no response bodies. It does not establish the exact entry order from that run. A controlled browser test reproduced this failure. It changed only the order of 30 equal-time entries in the second preflight response. The client reported `tools changed while you were editing. Reload saved before saving.` No binding request followed. A separate first-save response barrier showed that the pending-save guard worked. **Expected behavior** Reads of unchanged entries must have a stable order. Real entry changes must still trigger the existing configuration checks. Entry order does not set policy precedence; exclusions still win over matching includes. **Steps to reproduce** Create a profile with entries that share one creation time. Read the profile, change the physical row layout in a disposable PostgreSQL database, and read it again. The old service has no tie-breaker for these entries. The new regression fails against that service. ## What Changed - Sort list and detail entry reads by creation time, then ID, in the profile service. These reads supply saved configuration snapshots. Other reads retain their existing behavior. - Add one real PostgreSQL regression. It changes heap layout in both directions, checks stable list and detail values, preserves chronological ordering, checks company boundaries, and checks include/exclude results and a real entry change. - Include the exact test-only recovery prerequisite from commit `36c8d9ddd76d0b7ddb28a9a6c674d9e538f6df88`. Its file blob is unchanged. It removes a one-sweep fixture assumption while keeping bounded recovery assertions. No document UI or journey code is imported. ## Verification - Base: one frozen public master snapshot, `8c9dd1e4445e58cc9ca08076dfe9c0b27038d6ee`. The affected service, schema, projection, resolver, and save guard match the failed CI merge `94d98d5c75c8f331aad4bfab2111ee587b0c63fe`. - RED: the new real PostgreSQL test fails on the old service. Both physical layouts were verified before the ordering assertion failed. - GREEN: `cd server && pnpm exec vitest run src/__tests__/tool-profile-entry-order.test.ts src/__tests__/tool-profile-entry-validation.test.ts` — 18 tests pass with real PostgreSQL 17. - GREEN: `cd ui && pnpm exec vitest run src/lib/agent-configuration/save.test.ts src/lib/agent-configuration-yaml.test.ts --maxWorkers 1` — 51 tests pass. - GREEN: one focused Chromium treatment of the original compact YAML case passes. The treatment changes the actual disposable database heap after the first save. It does not transform HTTP responses. The 30 equal-time entries stay identical through the API, the second bind returns 201, and the original hidden-setting and disabled-Save assertions pass. - `node scripts/check-maps.mjs` and `git diff --check` pass. - Local tests use Node 24, a macOS source server, Chromium, and an isolated loopback PostgreSQL 17 container. The server and container are stopped. Private traces and control source are retained outside Git. These controls do not prove a deployed Cloud result. - A review follow-up removes two unnecessary query changes. Fresh real PostgreSQL verification still passes all 18 tests. The list/detail queries and browser path are unchanged from the passing heap-reorder treatment. - Prior head `59edcade203d6a05923955f5a0118e62c7e549c5` had a separate browser failure in run `41169427774`: `disconnect-write-recovery.spec.ts:204` kept the original title after a mocked session lookup failure. Its relevant test and UI source are unchanged from the base. The cause is unresolved. The two removed query changes do not repair it. The original log and artifact are retained; no unchanged job retry was issued. On the current head, this case passes in 14.3 seconds. The compact YAML case also passes in 11.2 seconds. The later success does not establish the prior failure cause. - Final exact head: `81c584aeac116b9d7a126b55dc99d5f6c70c195f`. [CI run 41169978784](https://github.com/paperclipai/paperclip/actions/runs/41169978784) completes with success: all 47 workflow jobs are terminal, and all 57 PR check/status entries are green or accepted neutral/skipped states. Full typecheck, build, Runner checks, security checks, and `ci / verify` pass. They were not repeated as broad local commands because the shared host is under load. - The new PostgreSQL regression also passes in Linux CI in 911 ms. Server shard 12 has 1,839 passing tests. Browser shard 3 has 45 passing tests, including the original compact YAML case. - Fresh Greptile and Paperclip reviews both give this exact head 5/5 with no findings. All review threads are resolved. The branch is mergeable and clean. ## Risks - Entries with the same creation time now use ID order. Their prior relative order was undefined. Entries with different creation times keep their existing order. - The query still sorts the same rows and adds one key to the sort. It adds no query, polling, schema change, or migration. - The configuration equality and concurrent-change checks are unchanged. This fix does not ignore real tool changes. - The original CI response order is unknown. The regression proves a permitted ordering defect and the matching failure mechanism, not the unique historical cause. ## Model Used - OpenAI Codex, based on GPT-6, with reasoning, source inspection, terminal tools, and browser test execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
13709b4c7a |
feat(connections): let agents configure connections within delegated authority (#17737)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connections provide governed tools, chat channels, model accounts,
and gateways.
> - Several management routes require a human board actor even when an
agent has the required grant.
> - Default agents must configure these resources within the responsible
user's authority.
> - This pull request adds a shared authorization path to the existing
REST and native API surface.
> - Human authentication, execution approvals, and local host boundaries
remain enforced.
> - The benefit is that a standard agent can complete authorized setup
with a bounded human handoff.
## Linked Issues or Issue Description
**Subsystem affected**
Server authorization, connection services, shared contracts, native API
discovery, database schema.
**Problem or motivation**
Connection administration is available in the web app but often
unavailable to agents. Agent grants, responsible-user authority,
credential ownership, and actor attribution need separate checks.
Opening gateway-token creation also requires preserving the issuing
agent's delegated authority during token use.
**Proposed solution**
Use the existing authorization evaluator for every connection family.
Enforce current responsible-user authority, effective trust, grant
scope, resource visibility, and target-agent access. Keep existing agent
grants unchanged. Expose authorized configuration through default
`search_api` and `call_api`. Save human authentication steps and let
agents continue after completion.
**Roadmap alignment**
This change completes the API access path for the existing Connected
Apps, Personal & Shared AI Accounts, and governed MCP Gateway
capabilities in `ROADMAP.md`. It follows the maintainer's requested
design. Related work: #11899 enforces connection execution access and
creator/manager rules; this PR extends agent management across
connection families. It does not replace that execution-policy work.
## What Changed
- Add shared connection management context and mandatory
responsible-user, trust, scope, and target checks.
- Allow authorized agents to configure tool, chat/email, GitHub/Slack,
AI, profile, and gateway resources through existing routes.
- Preserve human credential ownership and record the actual agent,
responsible user, and run or key in audit data.
- Preserve saved browser authentication handoffs and callback session
checks.
- Bind agent-issued durable gateway tokens to their responsible user and
recheck delegated authority at use.
- Add explicit REST transport and management capability metadata,
built-in guidance, and sensitive-response handling.
- Intersect caller and endpoint-agent Slack test/invitation policies,
including independent and shared rates. Require the addressed human’s
current endpoint identity link before completing identity handoffs.
- Add production-authenticated integration scenarios. Companion #17739
contains the saved setup UI (including the required human
import/rotation step), product and permission documentation,
catalog/Vercel credential and callback ownership, and operation maps to
keep each review below 100 files. Both PRs must ship together; merge
this implementation first, then its companion.
## Verification
- Current implementation head:
`2ecd6451d0d0c8d82121a4e3fa4ce94c5befbbd6`, including master
`ea727dc3c2`, 99 files. The latest follow-up fixes the two valid
findings from the preceding f9 review. Exact-head Greptile has completed
at5/5 with no findings and Superagent passed. Full exact-head Linux CI
workflow40446606056 completed SUCCESS; only the queued external
Paperclip Review remains pending; earlier Linux passes do not qualify
this successor.
- Latest preserved-stack verification passed **2,165 tests** in fresh
PostgreSQL: **70 actual-auth scenarios**, 27 OpenAPI contracts, 2,022
native API tests and 46 saved GitHub wizard cases. **141 existing
gateway/policy acceptance tests** also passed. Workspace
typecheck/build, maps and token gates passed. The first workspace
typecheck was blocked by sandbox tsx IPC permissions; its authorized
rerun passed. This is focused local evidence; the full Linux CI matrix
is still required.
- Issuer/target gateway policy regressions were red before the fix. Both
decisions now intersect, including Block/Ask first/rates, same
credential identity and current token/user/agent authority after
approval. Invocations retain their token and actual requesting agent;
deleted tokens fail closed. Thirteen actual-auth cases cover revocation,
narrowed scope, approvals, shared rates and board compatibility.
Intermediate missing-token attribution, review-context and
implicit-session actor failures remain in the qualification record; all
final cases pass.
- Activity pagination now retains a bounded cursor across private-only
windows. The actual-JWT regression reaches three older readable events
after 27 private ones without exposing private details. A native
device-login regression was red for plaintext receipts; local check and
test-call results now use existing protected-response handling. The
passing actual-auth case proves encrypted persistence, exact replay,
redacted history and no response asset.
- Previous-head `5b55195593` Linux CI initially failed e2e shard 4/8 in
the existing connection-intent continuation test: periodic recovery
queued a repair at 01:21:47, the scripted journey completed the task at
01:21:48, and the extra queued run was cancelled at 01:21:49 because the
task was terminal. The assertion selected that latest cancelled run.
GitHub refused the first attempt while the workflow was nonterminal.
After completion, the single evidence-supported failed-job rerun was
accepted as **attempt 2** on the same head (workflow 40431202466). The
failed shard and final E2E aggregate passed on attempt 2, and the
complete Linux workflow for that head is successful. The original
failure remains part of qualification. No additional unchanged-code
rerun will be used for this failure.
- Companion CI also has an unchanged login-cleanup idleness timeout,
retained in its qualification record. That same 16-case suite passed in
this parent Linux run and a fresh local database. Both preceding
workflows passed after their single failed-job reruns as attempt 2; the
original failures remain recorded. No additional unchanged-code rerun
was used.
- Migration `0380_broken_aqueduct.sql` was regenerated through the
repository workflow after master took 0379 for project Git
authorization. It adds the responsible-user column with `IF NOT EXISTS`,
preserves upstream migrations, and satisfies snapshot retention and
migration safety checks.
- Upstream published Copilot v62 and v63 identities remain byte-for-byte
unchanged. The combined guidance is now v64, digest
`sha256:22e31f95ef972bcead98a4a707c9f33433558fd111e390e027a2bbb4346e112a`.
The normal shared TypeScript/Rust identity check and **61 Copilot
installation tests passed**. The old source-identity assertion first
failed against unchanged published v63; regenerating a new v64 preserves
the published release.
- Workspace `pnpm -r typecheck` and `pnpm build`, map validation, and UI
token gates passed on the final combined stack after the upstream merge.
Node 24.22.1 / pnpm 9.15.4 were used.
- **Earlier 82 API/OpenAPI tests passed** in fresh PostgreSQL 18: 55
assembled-server scenarios and 27 contracts. They use real persistent
agent keys, run JWTs, human sessions, production defaults, and
deterministic provider fixtures. No actor injection establishes access.
- The new email regression was red before per-account scope filtering.
The Arcade regression was red before human ownership and inventory-key
fixes; it also verifies actual-agent audit identity and a human-session
inventory read. The activity regression measured **200 authorization
checks before the fix and one afterward** for the same one-item page.
- **82 GitHub UI tests passed**, including all three new
failed-replacement regressions that were red before their fix. The UI
preserves an earlier copy only after confirming its committed
generation. Changed or unreadable results block copying and reconnecting
until reconciled. Human-only secret responses now include their
nonsecret committed generation.
- Private test status now rechecks the originating principal and linked
task, including mandatory agent/responsible-user privacy. Cross-agent
tests require the full caller tool access decision and the same
credential identity, regardless of auth label; caller and target
policies/approvals and rate limits apply before execution and after
human approval. A profile-only precondition was found to reject valid
allow-policy and scoped-grant access and has been removed. Both
permitted-access cases and a no-auth-label identity regression were red
before the correction; caller-specific Block still prevents provider
execution. The private-status regression was also red before its fix.
- The tool-activity regression measured 100 capability decisions and 102
run captures on a 100-event page before the fix. It now uses two task
decisions and at most two captures while retaining the same 50 visible
events, private-event filtering, and 403 after audit-grant revocation.
The regression was red before request-context reuse and task-decision
sharing; final server typecheck/build also pass.
- Generic chat configure/reconnect now hands agent-supplied verification
credentials to the addressed human before provider work or vault
changes. Actual key/JWT regressions were red for Slack signing-key
replacement and generic channel secret-reference rebinding. They now
prove unchanged refs and vaulted signing value, human setup, permitted
agent bot-token repair, and governed test messages. All 82 API/OpenAPI
cases pass in fresh PostgreSQL. Eight native receipt protections and
final server typecheck/build also pass.
- The preceding review found that old healthy state could retire a
credential-entry card before a human changed anything. The actual-JWT
regression was red. Saved field generations now change only with
protected values and commit with refs plus a verification fence; retries
retain the baseline. Old health, bot-token repair and unchanged entry
stay pending. New signing verification and current authority permit
continuation. A separate red race proved that user revocation during
provider authentication could return 403 after refs changed; authority
is now checked before persistence and inside the credential transaction.
The final 82 API/OpenAPI cases, four neighboring real-database
Slack/GitHub lifecycle cases, and final server typecheck/build pass.
- A preceding full run crossed a rate-window boundary. Database rows
prove one consumed slot at 23:35 and another at 23:36 for the same
policy/bucket. Only Date is frozen in the rate test cases; transport and
asynchronous timers remain real. Production rate behavior is unchanged,
and the final full run passes. An intermediate fixture failure that
overwrote hidden generation metadata with an API projection is retained
in the qualification record.
- Linux CI on an earlier head found two rate-admission assertions that
omitted the newly persisted bucket key. The failures were reproduced
locally; both assertions now verify the exact actor/tool bucket and all
49 policy-service tests pass. The companion qualification record retains
the initial and intermediate failures. Catalog creator attribution is
preserved in the companion to keep this implementation review at 99
files.
- The latest review reproduced three regressions before repair: complete
human replacements failed on unavailable old signing secrets, fresh
GitHub links incorrectly selected reconnect, and different
credential-entry requests shared an idempotency key. Each old protected
ref now has a narrow known-unavailable fallback; unexpected resolver
failures and company/scope violations still abort. Fresh imports keep
their credential form, configured endpoints keep reconnect, and
canonical field sets/completion modes distinguish handoffs while
matching retries retain their baseline. All 55 actual-auth scenarios and
27 separately run OpenAPI contracts pass, as do server typecheck/build.
Companion #17739 also passes all 41 wizard tests including actual
fresh-form submission, UI typecheck, maps and token gates. Earlier red
results and a mistyped OpenAPI filename are retained in qualification.
- The configured GitHub handoff regression was red in the
browser-component tests and actual-auth link/persistence tests. Saved
links now name requested fields and all/any completion mode; the
companion form submits those supplied fields through reconnect and
retains other saved credentials. Webhook/client-secret reconnect remains
human-only; first-setup secret restrictions remain. All 46 wizard tests
pass, including combined modes and legacy links. A duplicate review
referenced the older companion bf760912; the current form is already
fixed, linked in the reply, and its thread is resolved.
- Linux CI on
|
||
|
Devin FoleyandPaperclip
|
bd77ffe78d |
perf(connections): scope profile rule and binding reads (#17589)
Read current company/agent profile bindings and rules matching the exact catalog rows being evaluated. Preserve all five selector kinds, null application matches, exclusions, and the existing access evaluator without a cross-call cache. Verified with full build and typecheck, 652 affected tests across ten suites, all 18 strengthened effective-assignment tests, exclusion mutation and byte counterfactuals, complete exact-head CI, and Greptile 5/5 Medium with no unresolved comments. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
2f637fe8c4 |
perf(tools): scope connection readiness assignment reads (#17541)
Connection readiness reads current permitted connection identities without unrelated catalog rows or complete profile-rule payloads. Reuse profile precedence and selector logic, preserve full public/runtime summaries, and keep installed connections, identity checks, and gateway authorization current. Verified with 106 focused tests, the full build, and typecheck. The direct catalog inventory regression fails at 102 rows when the scope filter is removed and passes at two rows with the filter. One unrelated CI timing assertion differed by 1 ms; the exact local test and the single failed shard rerun passed on unchanged source. CI is green and exact-head Greptile is 5/5 at Medium impact with no unresolved comments. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
233753e577 |
refactor(tools): isolate profile ownership (#17155)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tool profiles select the actions each agent can use. > - Profile reads, review, mutation and assignment resolution shared a large OAuth and connection service. > - Those paths have their own company checks, lock order and database projections. > - This pull request gives profiles explicit owners while preserving the existing transactions and public API. > - The benefit is a smaller coordinator and a reviewable access boundary. ## Linked Issues or Issue Description Depends on #17147. Stack: #17144 → #17146 → #17147 → this PR. These PRs remain open. **What existing behavior does this improve?** Maintenance of tool profile management and effective agent assignments. **Current behavior** The service entry point contains 19,293 lines. Profile behavior is spread across its closure and returned object. **Proposed behavior** Profile management accepts only a database and clock. Assignment resolution accepts three named capabilities. Projection and shared lock functions have explicit homes. Public methods and responses remain unchanged. **Reason and benefit** Reviewers can follow profile transactions, lightweight reads and precedence without navigating OAuth setup. The entry point falls to 17,752 lines, a reduction of 1,541 lines in this PR. ## What Changed - Extract profile management, projections and effective assignment resolution into three modules. Their sizes are 886, 268 and 216 lines. - Move shared actor/transaction types, identifiers and assignment locks into named modules. Preserve the public lock export. - Keep all database reads, transaction boundaries, lock order and method signatures. Add no cache or background work. - Record the next ownership cuts toward an entry point below 5,000 lines. ## Verification - Before and after: 466 cases in `tool-access-service.test.ts` and 33 cases across profile entry validation, effective assignments and catalog publication. All 499 cases pass with one database worker. - Server TypeScript check and `node scripts/check-module-boundaries.mjs` pass. `git diff --check` passes. - Normalized syntax comparison finds all 329 pre-existing function/method bodies unchanged. Wiring passes the unchanged public service tests. - The previous stack passed full typecheck and build. Its full local test run was interrupted and remains unverified; its complete CI matrix passed. This PR will record its own current-head CI result below. - Current-head CI is green. Current review: Greptile is 5/5 at `0f5c85905e`, with no inline review threads. - CI failure preserved: [general server shard 1](https://github.com/paperclipai/paperclip/actions/runs/39704932041/job/119582710174) failed the unchanged workspace-runtime test that replaces an unhealthy local process. The replacement refused connections for its original 10-second readiness budget. The fixture and runtime service are unchanged and do not import tool-access. One isolated run passed without changes or timeout increases. The failure log is preserved; the cause is not reproduced. Only that failed job was retried; it passed at the same head without code changes. The original failure remains recorded above. ## Risks - Moving profile mutations could change transaction clients or lock order. Both are unchanged and the existing database tests cover publication and assignment behavior. - Effective assignment resolution still uses the same installed-connection loader and full catalog projection. The lightweight path still omits large schema and metadata fields. - No schema, public API or access policy change is intended. Existing tests provide characterization coverage; no new test mirrors the extraction. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |