DottaandPaperclip
a26d217e94
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Cloud requests synchronize a human user's profile and company membership. > - Concurrent requests repeated the same writes before the debounce could record success. > - Upserts also updated unchanged rows and caused avoidable tuple work. > - This pull request shares identical pending writes and guards unchanged updates. > - Each request still reads current authority. Bounded profile lookup also reaches older memberships. > - The benefit is less database work during request bursts with the same actor results. ## Linked Issues or Issue Description **What happened?** A burst of 20 trusted Cloud requests issued 20 membership upserts and 40 profile plus membership tuple updates for one unchanged user. The completed-write debounce did not share in-flight work. **Expected behavior** Identical in-flight requests share persistence work. Distinct contexts retain their order. Failures remain retryable. Each request resolves its own current company and role authority. **Steps to reproduce** Run the synthetic PostgreSQL comparison below at baseline `db70c5acd92f50fa37fb4f0dea3da0e7685089a2`. The harness imports the real resolvers and counts actual writes with database triggers. Related work: #17663 covers adjacent Cloud authorization and projection behavior. This change preserves fresh imported memberships, owner-elevation settings and stale administrator removal. ## What Changed - Serialize synchronization per database client and user. Share only the last identical queued context. Preserve A/B/A transitions and retry after partial failure. - Guard user and membership updates with null-safe value comparisons. Retain default-grant repair when `RETURNING` is empty. - Keep authority reads outside the write debounce. - Resolve exact principal IDs through the existing index, then use at most one company-scoped alias query. This prevents both the old 200-row profile cutoff and repeated membership-page scans. Exact IDs take precedence over namesake aliases. - Add concurrency tests, PostgreSQL tests, a reproducible benchmark and behavior documentation. - Accept both Git 2.55 and 2.56 diagnostic messages in an existing workspace safety assertion, after full CI exposed that portability failure. No expiration behavior changes. ## Verification - Five focused suites passed, 63 tests total. These include Cloud actor concurrency, transient database retry, company provisioning, real PostgreSQL synchronization and the user profile route. - The auth-to-profile regression returned 404 before the lookup repair and 200 after it. The final fixture has 601 memberships and tests raw and normalized non-UUID IDs, missing slugs, alias ordering and exact-ID priority. A counterfactual with the old paging repair performs four membership queries for a missing slug; the final code performs two, each returning at most one row. Cross-company access returns 403 before lookup. - Local PostgreSQL 17.11 benchmark uses two warmup pairs and five alternating measured pairs. Each pair starts a fresh debounce scope. All 20 complete actor results match on every sample. - Burst median: 51.07 ms before, 25.71 ms after. Ranges: 47.21–63.25 ms and 24.17–29.32 ms. Membership statements: 20 to one. Tuple updates: 40 to zero. - Reproduce: `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-cloud-membership-sync.ts --baseline-ref=db70c5acd92f50fa37fb4f0dea3da0e7685089a2 --docker-postgres=17`. - Real PostgreSQL tests compare all four alias fields with the route's own JavaScript canonical slug output across 76 fixtures. Exhaustive Unicode lowercase inspection validates the special SQL mappings. - A real SQL_ASCII database reproduces the previous en-dash alias 404. The final whole-character replacements precede ASCII-only translation. Four alias fields and ten multibyte edge cases pass, including rejection of a false euro-to-k alias. The disposable database is migrated normally, then removed. - Server TypeScript passed with no diagnostics. - PostgreSQL tests used the normal migrations with a temporary Docker startup override because this host's native PostgreSQL semaphore allocation is exhausted. No tracked test infrastructure changed. - The first full CI run had an unrelated CLI company-import/export health-start timeout. Review also identified repeated profile scans; this follow-up repairs that finding. A later run passed 3,882 tests in server shard 4/12 and failed one existing assertion because Git 2.56 changed its error wording for a missing gitfile target. The safety behavior still refused expiration and preserved the workspace. The assertion now accepts both valid messages; the actual Git/PostgreSQL regression passes locally. Final head `3a644e600ab0ae7315f9828b1ea9dffdef407e4a` passed [full CI](https://github.com/paperclipai/paperclip/actions/runs/40537153522), including the previously failing shard. There are 55 successful checks/statuses, two skipped and one neutral. Fresh Greptile and Paperclip reviews are both 5/5 on this exact head, with zero unresolved threads. GitHub reports mergeable and CLEAN. - Full repository typecheck and build passed on the integrated final source. The local full test command was stopped after native PostgreSQL initialization failed repeatedly, producing 16 failures and skipped database suites. It is incomplete, not a pass. Focused real-database suites used disposable Docker PostgreSQL. Current-head CI runs the full gates separately. ## Risks - Coalescing is local to one process and database client. Other replicas can still take conflict locks. - `updated_at` now records actual profile and membership changes. Access lists sorted by it can show a different order. Profile resolution no longer depends on the first 200 entries. Exact principal IDs take precedence over namesake aliases. The single alias fallback still scans company candidates; no constant-CPU claim is made. - Default role grants retain their existing additive semantics. This change does not define a new demotion policy. - Local synthetic service timings exclude HTTP and network latency. No hosted deployment or production speedup is claimed. - No migration, startup job, polling change or deployment is included. ## Model Used OpenAI Codex, configured model `gpt-6.1-sol` with xhigh reasoning, tool use and code execution. Independent blast-radius review also used `gpt-6-astra` and `gpt-6.1-sol`. The runtime context window was not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
4.0 KiB
Cloud membership synchronization
Trusted Cloud requests synchronize the user's profile, company membership and default grants before they resolve authority. Synchronization writes are shared only within one database client and process. Requests with different contexts for the same user execute in order. An A, B, A sequence still applies all three contexts. Other users can proceed independently.
Successful identical contexts retain the existing five-second write debounce. Failed or partial synchronization invalidates the prior success, so a retry can repair it. Membership reads, owner-elevation settings and stale administrator removal remain fresh on each request. This cache does not store actors.
Guarded upserts preserve updated_at when profile fields, role and status do not
change. They still repair missing default grants. Access lists now sort these
rows by their last actual change. Profiles remain reachable beyond the first
200 memberships: lookup uses the existing company/principal index for an exact
principal ID, then at most one company-scoped alias query returning one row.
An exact ID takes precedence over another member's name, matching the UI's
stable-ID links. Name, email and normalized non-UUID ID aliases retain their
newest-membership-first ordering. A missing slug performs two queries, with no
membership paging. The fallback still scans company candidates; it does not
add an expression index or claim constant database CPU at every company size.
Reproduce the local comparison
From the repository root, use a disposable PostgreSQL 17 container:
pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-cloud-membership-sync.ts \
--baseline-ref=db70c5acd92f50fa37fb4f0dea3da0e7685089a2 --docker-postgres=17 \
> /tmp/cloud-membership-sync.json
The harness imports the actual resolvers, applies normal migrations, seeds
synthetic data and removes its container. It does not use DATABASE_URL.
Two warmup pairs precede five alternating measured pairs. Each pair uses a
fresh synchronization scope and 20 simultaneous requests for one unchanged
user. Row triggers count updates, including updates that preserve values.
PostgreSQL 17.11 measured a median 51.07 ms before and 25.71 ms after. The ranges were 47.21–63.25 ms and 24.17–29.32 ms. Membership statements fell from 20 to one, and profile plus membership tuple updates fell from 40 to zero. All 20 complete actor results matched on every sample. This measures local service work and authority reads. It does not establish production request latency or remove locks across replicas.
Safety checks
Run the Cloud actor, Cloud transient database retry, company provisioning,
Cloud synchronization PostgreSQL and user profile route suites. They cover
coalescing, A/B/A transitions, failed partial writes, expiry, distinct users and
databases, grant repair, real role-change invalidation, imported-membership
revocation, owner-flag changes and cross-company denial. The profile regression
first returned 404 with the former 200-row lookup. The fixture now contains
601 memberships with timestamps differing within one millisecond. It checks
raw and normalized non-UUID IDs, a missing slug's two-statement bound, tenant
isolation, exact-ID priority and fallback ordering. Real PostgreSQL requests
compare all four alias fields against the route's own JavaScript canonical
slug output, including quotes, whitespace, punctuation, Unicode and email edge
cases. An exhaustive Unicode lowercase check verifies the two non-ASCII
characters that need explicit ASCII mappings in the SQL predicate, avoiding
database-locale differences. Whole-character replacements precede an
ASCII-only translate, preserving UTF-8 byte sequences on legacy SQL_ASCII
databases. A disposable SQL_ASCII database applies normal migrations and
exercises all four alias fields with en/em dashes, euro signs, neighboring
multibyte characters, dotted I and Kelvin signs. It also verifies that a euro
sign cannot incorrectly match the profile alias k.
No schema migration, startup work or deployment is required by this change.