mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-26 10:26:26 +01:00
fix/postgres-null-socket-write-guard
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
8087661bb8 |
fix: bound workspace Git scans (#11572)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Workspaces let users and agents inspect files that belong to an issue > - Changed-file views use full-tree Git status scans > - Many issue views could start those scans at the same time and make the server unresponsive > - Route-level limits did not protect the process or coalesce work for one repository > - This pull request adds one bounded scheduler for every expensive workspace Git scan > - It also starts browser scans only when the file panel is open and visible > - The benefit is bounded child-process use and responsive health checks during request storms ## Linked Issues or Issue Description **What happened?** Many changed-file requests could start full `git status --porcelain=v1 -z --untracked-files=all` scans at the same time. One production incident produced about 270 direct Git child processes. The Node process stayed alive but stopped answering health requests in time. **Expected behavior** Paperclip must bound expensive Git work across all companies, actors, issues, repositories, and browser tabs. Duplicate requests for one worktree must share work. Excess requests must fail fast with a retryable response. Hidden or closed file panels must not start scans. **Steps to reproduce** 1. Open changed-file views for many issue and actor keys. 2. Send requests for two large workspace roots at the same time. 3. Observe that route-level limiter keys allow many full Git scans to run together. 4. Observe delayed health responses and accumulated Git children. **Paperclip version or commit** Reproduced on master before commit `43ab441f0f`. **Deployment mode** Self-hosted server with local workspace repositories. ## What Changed - Add a process-wide scheduler with configurable concurrency, queue capacity, timeout, and cache TTL. - Add fair admission, a bounded queue, canonical worktree keys, single-flight joins, and bounded result caching. - Add subprocess timeouts, TERM-to-KILL escalation, bounded output, waiter cancellation, and slot cleanup. - Route full-tree status work from file resources, workspace runtime, execution workspaces, and adapter overlay sync through the scheduler. - Return stable retryable `503` and `504` error codes for saturation and timeout. - Add structured logs with safe workspace hashes, durations, queue state, cache use, joins, and terminal outcomes. - Gate UI queries on panel and document visibility. Cancel queries on close, hide, unmount, and workspace change. - Disable focus and reconnect bursts. Keep one explicit refresh action and a retryable unavailable state. - Document the 10-second default freshness tradeoff and all configuration variables. - Add unit, route, UI, adapter, and deterministic 500-request load coverage. ## Verification - `pnpm -r typecheck` - `pnpm build` - `pnpm check:token-gates` - `pnpm --filter @paperclipai/server exec vitest run src/services/workspace-git-operation-scheduler.test.ts src/__tests__/file-resources-git-scan-load.test.ts --reporter=dot` — 16 tests passed. - `pnpm --filter @paperclipai/ui exec vitest run src/components/WorkspaceFileBrowser.test.tsx src/lib/page-visibility.test.ts --reporter=dot` — 38 tests passed. - `pnpm --filter @paperclipai/adapter-utils exec vitest run src/git-workspace-sync.test.ts --reporter=dot` — 16 tests passed. - Existing file-resource, workspace-runtime, and execution-workspace regression selections passed. - Two cleanup safety regressions prove failed scans preserve the worktree before archive and at the final deletion fence. - Before: the incident produced about 270 Git children and health requests timed out. - After: 500 concurrent requests across 500 issue keys, 73 actors, and two roots started two underlying scans. Peak scan concurrency was 2. All 500 requests succeeded. Health p99 was 4.94 ms. The harness found zero unreaped children. - The full local Vitest run passed 4,267 tests. Ten existing fixed-port HTTPS exposure tests could not run because this host already owns Tailnet listeners on ports 42000 and 52000. Clean GitHub CI is the final full-suite result. - Latest-head GitHub CI passed all required test, typecheck, build, canary, e2e, policy, and security gates. - Greptile completed at 5/5 with zero unresolved comments, recommendations, or follow-ups. ## Risks - Changed-file results can be up to 10 seconds old by default. Explicit refresh remains available. - A full queue returns a retryable `503` instead of waiting without a bound. - A scan that exceeds the default 8-second deadline returns a retryable `504` and terminates its process group. - Operators can tune all limits with documented environment variables. Safe defaults protect local and shared servers. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 family. The runtime does not expose the exact deployment ID or context-window size. High reasoning, tool use, and code execution were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7ea2068ef8 |
fix(files): only highlight accessible workspace file links (#11090)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task comments can contain references to files in project and execution workspaces. > - Paperclip detected path-shaped inline code and showed it as an actionable file chip. > - The UI did not first confirm that the current board session could open the file. > - Missing, denied, ambiguous, remote, and unsupported files therefore looked actionable and failed after a click. > - This pull request adds an issue-scoped availability check and promotes only confirmed files to chips. > - The benefit is that the task thread shows a file action only when that action can succeed. ## Linked Issues or Issue Description **What happened?** Task comments promoted path-shaped inline code to file chips before Paperclip checked the file. A chip could point to a missing, denied, ambiguous, remote, or non-previewable file. The action then failed after the user selected it. **Expected behavior** Paperclip must show a file chip only after the server confirms that the current board session can open the exact file reference. All other path-shaped text must stay ordinary inline code. **Steps to reproduce** 1. Add a task comment that contains inline code with a missing or inaccessible workspace path. 2. Open the task thread as a board user. 3. Observe that the path looks like an actionable file chip. 4. Select the chip and observe that the file cannot open. **Paperclip version or commit** `19be4cf927` and earlier. **Deployment mode** Local dev and self-hosted server. **Access context** Board user. ## What Changed - Added shared request, response, and validation contracts for batched workspace-file availability checks. - Added an issue-scoped server endpoint that resolves file references with company, issue, workspace, and preview-access checks. - Added bounded batch concurrency and tests for missing, denied, ambiguous, remote, unsupported, and available files. - Added an issue-scoped UI availability registry that deduplicates, batches, caches, and invalidates file checks. - Changed task-comment markdown rendering so only confirmed files get chip styling and file-viewer behavior. - Bound each chip to the exact workspace target that passed the availability check. ## Verification - `pnpm exec vitest run packages/shared/src/workspace-file-resource.test.ts server/src/__tests__/file-resources.test.ts ui/src/components/MarkdownBody.test.tsx ui/src/components/WorkspaceFileMarkdownBody.availability.test.tsx ui/src/lib/remark-workspace-file-refs.test.ts ui/src/lib/workspace-file-availability.test.ts` — 93 passed, 35 skipped. - `pnpm check:token-gates` — clean. - `pnpm -r typecheck` — passed. - `pnpm build` — passed. - `pnpm test:run` — all server and UI groups passed. One unchanged CLI test saw the run-injected static AWS credentials and expected only its local `AWS_PROFILE`. The same test passed, 8 of 8, after removing only `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` from its process environment. ## Risks - File chips now appear after an asynchronous availability check, so path-shaped text can briefly render as inline code. - Availability results use the existing 30-second file-resource cache window. File-resource invalidation forces a new check. - The endpoint limits each request to 100 references and the client chunks larger sets. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with GPT-5. The service did not expose a more specific model ID or context-window size. The agent used high-reasoning mode, repository tools, command execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
8f7282066e |
Add workspace file downloads
Add first-class workspace file downloads, broader attachment content-type support, and the stream-lifetime limiter fix from PR review. |
||
|
|
468edd8b22 |
Add workspace file viewer and artifact links (#7681)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent work is issue-centered, and reviewers often need to inspect files, artifacts, and path references produced during that work. > - Before this branch, workspace-relative paths and artifact file references were not first-class inspectable objects in the board UI. > - Safe file viewing needs shared resource contracts, server-side workspace boundary checks, and UI that opens files without exposing arbitrary host paths. > - The workspace file viewer branch needed to stay as one active PR and be rebased onto current `paperclipai/paperclip:master` for review. > - This pull request adds the workspace file resource API, issue-page file viewer and browser, markdown file-reference links, and artifact file chips. > - The benefit is that board users can inspect relevant files from issue context while preserving workspace boundaries and auditability. ## Linked Issues or Issue Description No public GitHub issue exists for this branch. Internal Paperclip issues: `PAP-1953`, `PAP-10539`, `PAP-10733`. Problem / motivation: - Board users need to open workspace-relative files mentioned by agents or attached as work-product metadata without switching to a terminal. - The UI needs to support both direct file-path opening and workspace browsing/searching from an issue page. - The server must enforce company access, workspace boundaries, size limits, rate limits, and safe audit logging. Related PR: - Prior closed attempt: #4442 - Single active PR for this branch: #7681 ## What Changed - Added shared workspace file resource types, validators, and workspace-file `resourceRef` metadata validation for work products. - Added server routes/services for resolving, listing, and previewing workspace-relative files with access checks, scan caps, list-specific limits, and audit logging. - Added the issue file viewer provider, sheet, workspace browser, command-palette action, markdown workspace-file autolinks, and artifact file chips. - Updated issue workspace UI and stories/tests for file browsing and workspace file opening. - Rebased the branch onto current `paperclipai/paperclip:master` and updated the existing single PR branch. - Addressed current-head Greptile follow-ups by applying `offset` consistently across search/recent/changed file listings, restoring stopped-service port ownership checks before auto-port reuse, and stabilizing the workspace browser pagination test. ## Verification Current local verification after rebase to `public/master`: - `pnpm exec vitest run packages/shared/src/work-product.test.ts server/src/__tests__/file-resources.test.ts server/src/__tests__/instance-settings-routes.test.ts server/src/__tests__/instance-settings-service.test.ts server/src/__tests__/workspace-runtime.test.ts ui/src/components/FileViewerSheet.test.tsx ui/src/components/FileViewerSheet.copy.test.tsx ui/src/components/WorkspaceFileBrowser.test.tsx ui/src/components/WorkspaceFileMarkdownBody.test.tsx ui/src/context/FileViewerContext.test.ts ui/src/lib/remark-workspace-file-refs.test.ts ui/src/lib/workspace-file-parser.test.ts ui/src/components/IssueWorkspaceCard.test.tsx` - 13 files passed, 197 tests passed. - `pnpm -r --filter @paperclipai/shared --filter @paperclipai/server --filter @paperclipai/ui typecheck` - passed. - `pnpm exec vitest run ui/src/components/WorkspaceFileBrowser.test.tsx` - 1 file passed, 25 tests passed. - `pnpm exec vitest run server/src/__tests__/file-resources.test.ts server/src/__tests__/workspace-runtime.test.ts` - 2 files passed, 90 tests passed. - `pnpm -r --filter @paperclipai/server typecheck` - passed. - Confirmed branch is `0` behind and `46` ahead of current `public/master` after rebase and follow-up commits. - Confirmed the PR diff does not include `pnpm-lock.yaml`. - Confirmed the PR diff does not include `.github/workflows` changes. - Searched GitHub for duplicate or related workspace file viewer PRs/issues; #4442 is the prior closed attempt and this PR is the single active PR for the branch. - No screenshots were committed; the task explicitly asked not to add design screenshots or images unless they were part of the work. Current remote verification on head `a698a7bc10137baf7d25bd5722e1d6e0343387c1`: - Greptile Review - success, 64 files reviewed, 0 comments added, no unresolved Greptile review threads. - PR workflow `verify` - success. - Typecheck + Release Registry, General tests, workspace test shards, serialized server suites, Build, Canary Dry Run, e2e, Socket, and Snyk - success. - `security-review` - neutral, with output saying a draft advisory was filed for maintainer review and is not a merge block. - `commitperclip PR Review / review` - cancelled after the security gate detected flags and timed out while creating/reviewing the advisory. I reran it once and it cancelled the same way; no actionable code/test failure was exposed in the job logs. ## Risks - This is a broad UI/server feature PR, so review needs to pay attention to route authorization, workspace boundary handling, and markdown autolink false positives. - Workspace browsing intentionally caps list results and scan depth; very large workspaces may require users to refine search terms. - Remote workspace preview remains unavailable until remote file-access support is implemented. - The neutral commitperclip security-review advisory needs maintainer review, but the check output says it is not a merge block. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected - check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent in a Paperclip/Codex local tool-use environment, medium reasoning, with shell/GitHub CLI tool use for branch inspection, verification, rebase, PR update, Greptile review, and CI inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |