DottaandPaperclip
0b266bd084
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Each task prepares its workspace before it starts the provider. > - Source preparation holds a transaction-scoped advisory lock on a dedicated database connection. > - Preparation can use other connections and the filesystem for longer than PostgreSQL allows that lock connection to stay idle. > - PostgreSQL then closes the connection and releases the lock while preparation still runs. > - This pull request reuses the workspace lock keepalive during source preparation. > - Slow preparation retains exclusion and returns its original result or error. ## Linked Issues or Issue Description **What happened?** Tasks failed during workspace preparation with `CONNECTION_CLOSED` at the configured 60-second idle transaction timeout. No provider had started. A live retry showed the source-lock transaction idle while authorization reads ran on another connection. Its advisory-lock key matched the task's source-preparation key. PostgreSQL closed that transaction at the timeout. The same defect remains on current master. A real PostgreSQL regression also shows that another preparation can enter after the timeout while the original callback still runs. **Expected behavior** Source preparation must keep its advisory lock until the callback and any pending keepalive query finish. The ordinary pool must remain usable. The original preparation result or failure must survive. **Steps to reproduce** 1. Set `idle_in_transaction_session_timeout` to one second in a disposable PostgreSQL database. 2. Call `withIsolatedRepositorySourceLock` and keep its callback active for longer than one second without querying the lock connection. 3. Try to acquire the same source lock from another operation. 4. On the old code, the second operation enters and the first transaction fails with `CONNECTION_CLOSED`. **Paperclip version or commit** Reproduced on source at `cde944ed75648adfb569e86133adc8b118b55dd6`. The repair is rebased onto `d855e550ad`. **Additional context** Refs #16340. That change introduced the keepalive for workspace copyback. This PR applies the existing helper to the source-preparation transaction. ## What Changed - Read the configured idle timeout in the same query that acquires the source lock. - Keep that dedicated connection active during source preparation with the existing helper. - Add two real PostgreSQL regressions for exclusion, company independence, a one-slot ordinary pool, nested acquisition, cleanup, and preservation of callback errors. - Update the execution contract and workspace verification maps. ## Verification - Red before the production edit: both new tests failed. A competing operation entered after the lock expired. A callback failure was replaced by `CONNECTION_CLOSED`. - Green after the repair: 86 tests passed across the source lock, source admission, keepalive, and heartbeat workspace preparation suites. - `pnpm -r typecheck`, `pnpm build`, and `node scripts/check-maps.mjs` passed on final commit `2b5000ce25`. - `git diff --check` passed. The lockfile is unchanged. - After rebase, the same 86 focused tests passed. The two source-lock tests passed again after the review cleanup change. - Live acceptance on `7df94d7a06`: a fresh local instance on macOS arm64, Node v26.4.0, native Codex `gpt-6.1-sol`, and permission mode `never`. PostgreSQL had a one-second idle transaction timeout. A temporary Git wrapper delayed repository reads by four seconds. Workspace preparation after environment acquisition took 8,297 ms. The real task completed, wrote and attached the correct line-count/SHA-256 report, reached `done` with native phase `committed`, and cleared its execution lock. The attachment matched the workspace file and survived browser reload. No dedicated lock connections remained. - The live probe initially used `gpt-5.4`, which the ChatGPT account rejected. Selecting the reported task model, `gpt-6.1-sol`, resolved that readiness failure. The local Claude CLI has no login, so no live Claude run is claimed. - Full CI passed on `2b5000ce25`: all general, chat, workspace, serialized-server, Runner, and E2E test shards, plus typecheck, build, and canary packaging. Both Greptile and Paperclip Review rated this commit 5/5 with no open findings. The Paperclip check concluded neutral; two unrelated visual checks were skipped. - The duplicate local `pnpm test:run` serial invocation was stopped with SIGINT after the complete CI test shards passed. It did not finish locally; no local full-suite pass is claimed. Local proof consists of the focused real PostgreSQL tests, typecheck, build, map checks, and the real task above. - The final commit changes test teardown only; production code matches the live acceptance commit. - Initial sandbox runs could not open test database ports or build IPC sockets. The same commands ran with local process access. The frozen install also found an existing manifest/lockfile mismatch; dependency resolution was local and the lockfile was restored. ## Risks - The lock connection sends one small query per keepalive interval. The helper derives that interval from the configured timeout and caps it at 30 seconds. It does no polling when the timeout is disabled. - The helper joins an in-flight query before the transaction ends. A slow database can therefore delay cleanup. - This change does not disable database deadlines, retry preparation, or recover an unexpected connection loss. It preserves existing admission checks and company/source lock keys. - Deployment and recovery of the reported tasks remain separate from code verification. ## Model Used OpenAI GPT-6 through Codex, with repository tools, code execution, and browser verification. The exact served model variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
Paperclip feature map
Start here when reproducing a user-facing bug, verifying a change, or deciding which surfaces a fix must cover. Each recipe describes what a person can do, where they can do it, the expected result, and the evidence needed to verify it. Product behavior is still governed by the implementation spec.
The sibling agent access map records APIs, agent entry points and permission boundaries for the same capability IDs. Follow the shared map contract when maintaining either map.
Browse features
Open the published Storybook guide for the assembled feature and capability index. The Markdown recipes in this directory are the authored source. Each has a same-named agent-access companion.
Run node scripts/storybook-catalog.mjs to generate local navigation in
maps/generated/feature-map.md, maps/generated/agent-access-map.md, and the
Storybook guide. A new recipe is discovered automatically; do not register it in
this README or maintain counts here.
Before driving a journey
- Use this checkout's isolated test drive for manual product checks. It creates a temporary instance and prints its URL and data directory. Do not guess that a server on port 3100 belongs to you.
- Record the commit, URL, company, login role, relevant experimental settings, adapter, runtime mode, and live versus simulated dependencies. Use disposable tasks and provider resources. A fresh test drive creates a company and CEO, but no task or first run.
- Use the current navigation and company prefix. Paths in recipes omit that prefix. Record whether the streamlined or production shell is selected; Agent Chat, chat connectors, and the combined Inbox/Tasks view have separate gates. A hidden surface is an unmet prerequisite, not a successful test.
- Run the smallest relevant test first. Vitest commands below run from the repository root after installing dependencies. Playwright recipes use their named configuration and its isolated environment, not an unrelated running instance. Reserve expensive runner/provider tests for the behavior at issue.
- Exercise the actual user action, inspect its result, reload, and verify the persisted state or continuation. Use read-only API checks to corroborate UI evidence; creating state through the API does not prove the creation UI.
For agent-driven acceptance work, start with the Paperclip verification skill for an isolated launch, ownership/readiness checks, browser driving, evidence, and cleanup. It uses this directory as its feature map. The existing dev-workspace run/verify skill and evaluation skill describe runtime ownership and evidence handling. Reuse them; this map introduces no environment launcher, credentials, scheduled job, or second test framework.
Evidence contract
Report each entry point as passed, failed, blocked (with its missing prerequisite), or not run. Include the feature filename and entry-point ID, commit/environment, user action, expected and observed result, command/exit code, and evidence links. Screenshots or traces should show both the action and the discriminating result. Include task/run/request IDs when relevant, without secrets.
A fix is verified across its affected surfaces only when each has evidence or an explicit reason it does not apply. Shared code alone does not prove parity. Keep product regressions visible; do not change the recipe to bless a failure. For Paperclip-assigned work, attach evidence through the artifact workflow.
Inventory and remaining depth
The generated maps/generated/coverage.json lists the current non-test TSX page
modules and the recipes that directly reference each one. An unreferenced module
is a documentation candidate, not proof that its feature is absent or untested.
Remaining workflow limits live in each recipe. Inventory and example associations
describe documentation scope, not runtime health.
Maintaining the reference
The map documents user journeys. It does not execute them automatically; generated inventories record documentation scope rather than live test health.
When maintaining a recipe, compare its entry points and expected results with
the current source. Check that linked tests still exercise the stated behavior
and that local references still exist. Keep the same-named agent access companion
and stable sub-feature IDs aligned when behavior or access changes. Source-only
access inventory remains distinct from live verification. Run
node scripts/check-maps.mjs after map edits for structural/reference checks;
follow the shared contract for the source and behavioral checks it cannot perform.
Each recipe starts with a small frontmatter navigation record, then an H1:
---
navigation: {"section":"Tasks and conversations","sectionOrder":20,"order":10}
---
The H1 supplies the feature title, and the first paragraph supplies its summary. Orders are local to the section; leave gaps for additions. Equal orders use the stable filename as a tie-breaker. Keep a section's name/order consistent across its recipes. A new feature or capability needs only its recipe and companion.
Optional ### Storybook verification guidance lives inside the recipe's Gotchas
section. Each level-four heading contains a backtick-quoted capability key and
has Scope (ui, cli, or browser),
Storybook, Production, and Sources fields. These are verification instructions
and intentional scope limits, not run results or a required story for every API.
The generated gap report combines this guidance with current story associations.
After frontmatter, each recipe starts with an H1 and a user-visible description, then exactly:
- Sub-features — stable backticked IDs and observable behavior/states.
- How to get to it (user POV) — one H3 backticked entry-point ID per surface.
- Driving it with — starts with
Preconditions:; repeat each entry-point H3, withAutomated:andManual:paragraphs. Name test scope and gaps honestly. - Gotchas — misleading look-alikes, feature gates, and invalid evidence.
New or updated entries follow
Poteto's feature-map examples.
Existing entries with a Driving it heading can adopt the named harness when
their recipes are updated. The original format was inspired by
Omnigent's feature map.
Paperclip's recipes and checks follow its own task model and test infrastructure.