mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-27 19:35:13 +01:00
a31eba2381802cf0d1e6f8a14798cf5bcdc7325f
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
40619e2e23 |
fix: recognize wrapped document write conflicts
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
276ae3a75d |
Harden company import: durable UI, async jobs, integrity guard, batched inserts (#10523)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Company Import/Export (#10507) moves whole companies between instances as portability bundles > - Real-world use on a large company (1,418 issues, ~10.6k comments) surfaced a cluster of related failures: the import took hours and the browser connection died while the server kept running, a retry silently produced a second partial import, the progress/error UI gave no durable signal, and a cloud-tenant user couldn't even open the companies afterward > - Root cause of the slowness: importBundle inserted every issue, comment, and document as a separate round-trip to a network Postgres — an N+1-over-network pattern > - This pull request hardens the whole import path: durable progress/error UI, an async server-side job so imports survive dropped connections (with a duplicate-submit guard), a fail-closed guard against incomplete payloads, and batched inserts that cut a large import from hours to minutes > - The benefit is that migrating a real, large company actually completes, is legible while it runs, and can't half-import twice ## Linked Issues or Issue Description - Refs #10507 (the Import/Export feature this hardens). Supersedes #10513 (the progress/error-UI piece, folded in here). No open issue; problem described above (large-company import: slow, connection-fragile, silently duplicable, opaque UI). ## What Changed - **Batched inserts (perf):** importBundle pre-generates entity ids in JS and inserts in chunked multi-row statements, so children no longer wait on parents' generated ids. A 1,418-issue import drops from ~15,600 insert statements to **82** (190×); benchmark below. Import semantics — collision handling, pause-on-import, label/blocker/monitor/attachment/embedded-asset handling, blob sha verification — are unchanged (full portability suite green). - **Async import jobs for board sessions:** the existing cloud-tenant async job path opens to board sessions with per-actor job keys; the import page submits, polls, and resumes watching after a reload or dropped connection instead of holding one fragile request. A non-terminal job blocks a duplicate submit (409 returns the running job), preventing the double-import. - **Fail-closed completeness guard:** an optional `expectedFileCount` on inline imports; the server rejects (422 `import_payload_incomplete`) a body carrying fewer files than declared, so a re-framed/short payload fails loudly instead of half-importing. - **Durable progress/error UI (was #10513):** persistent progress panels with size-aware copy, persistent error panels with retry guidance, and inline explanation when the preview button is disabled; request-lifecycle guards so stale previews/imports can't publish or detach. ## Verification - `pnpm -r` typechecks (shared, server, ui) clean. - `company-portability.test.ts` (76) + `company-portability-routes.test.ts` (30) green — the import correctness net — plus new `CompanyImport.test.tsx` async/resume/409 coverage and a new batching regression test (a 50-issue import issues <50 issue-insert statements; rows land unchanged). - **Batching benchmark (embedded Postgres):** at 1,418 issues × 7 comments × 1 doc — 82 insert statements vs ~15,598 one-per-row (190×), ~1s wall-clock; a row-verifying run at that scale imports all 1,418 issues / 9,926 comments / 1,418 documents with unique identifiers and no warnings (no rows dropped by chunking). Over a network DB the round-trip reduction is the hours→minutes lever. - What is NOT directly measured here: wall-clock against a real network Postgres (that happens on a staging deploy); the local timing is network-free. ## Risks - Batching is the load-bearing change: it rewrites the import write path. Mitigated by the unchanged 106-test correctness suite, a new scale/row-integrity test, and per-writer transactions (a failure rolls back its table group; not a single outer transaction across writers — noted, correctness preserved). - Async jobs are in-memory (lost on server restart → pollers 404 and can resubmit); matches the pre-existing cloud-tenant job semantics. - `expectedFileCount` is optional (older callers unaffected); over-count is allowed, only under-count fails closed. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic), Claude Code CLI, extended thinking + tool use; implementation across Fable 5 subagents with live diagnosis against a running instance. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
|
b13eb5b2b5 |
Skill Studio: three-pane skill IDE with sandboxed test runs (#9241)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Skills Manager gives operators a reusable skill layer, but iteration still required manual edits, ad hoc prompts, and indirect run inspection. > - Skill authors need a focused workflow for editing skill files, saving representative test inputs, and running those inputs through an agent without exposing harness tasks as normal company work. > - The backend therefore needs durable test inputs, reusable run templates, hidden harness issues, scoped run execution, retention metadata, and read-containment rules around hidden work. > - The frontend needs a three-pane Studio that keeps skill files, saved inputs/templates, and run output/history visible together while preserving the existing design system and token rules. > - This pull request ships that Skill Studio surface end to end: database migrations, shared contracts, server APIs/services, hidden harness execution behavior, UI routes/components, and focused tests. > - The benefit is faster and safer skill iteration, with inspectable outputs and fewer ways for internal harness work to leak into normal task lists, costs, or adjacent read APIs. ## Linked Issues or Issue Description No public GitHub issue exists for this feature. Feature request summary: - Problem: Skill authors need to edit and test company skills in one place instead of switching between the skill detail page, task creation, run output, and manual prompt history. - Proposed solution: Add a Skill Studio workbench with saved inputs, reusable templates, hidden sandboxed test runs, live run status, output inspection, run history, rerun/delete controls, and frontmatter-aware editing. - Expected users: Paperclip operators and agent-company maintainers who create, fork, import, and tune skills. - Related public PRs: Supersedes #9205, which was replaced so the public PR branch name follows contributor policy. - Duplicate search: searched public GitHub issues and PRs for "Skill Studio"; no other active public issue or PR directly covers this feature. ## What Changed - Added database migrations for Skill Studio test inputs, test runs, test run retention, and reusable run templates. - Added shared Skill Studio types, validators, route helpers, frontmatter utilities, and status handling. - Added server services and routes for saved inputs, test runs, templates, reruns, terminal-run deletion, hidden harness issue execution, and run-detail hydration. - Strengthened hidden-issue read containment across issue-adjacent routes and cost rollups used by skill test harness work. - Added the Skill Studio UI with skill file editing, frontmatter editing, saved inputs, templates, run creation/cancel/rerun/delete flows, output rendering, history, route support, and responsive pane behavior. - Added focused backend, shared, and UI tests for the new APIs, routing logic, editor/run behavior, hidden-issue containment, and migration safety. - Rebased onto current `master`, removed the generated lockfile diff from the PR, and verified no workflow files are changed. ## Verification - [x] `pnpm --filter @paperclipai/db check:migrations` - [x] `pnpm check:token-gates` - [x] `pnpm exec vitest run server/src/__tests__/company-skills-service.test.ts server/src/__tests__/company-skills-routes.test.ts server/src/__tests__/company-skill-test-runs-service.test.ts ui/src/lib/skill-studio.test.ts ui/src/pages/SkillStudio.test.tsx` — 5 files, 132 tests passed - [x] Greptile review on the latest PR head - [x] GitHub PR checks on the latest PR head ## Risks - Medium risk because this is a broad feature touching database schema, server orchestration, issue visibility, and a large UI surface. - Hidden harness issue containment is security-sensitive; this PR includes regression coverage for adjacent read paths and cost rollups. - The new migrations are additive and use idempotent guards where applicable, but deployed databases that previously tested draft migration numbers should still be checked carefully. - The UI depends on a new resizable panels package in `ui/package.json`; the lockfile is intentionally left to repository automation. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5 coding agent with shell, git, and GitHub CLI tool use. Earlier feature commits include assistance from other Paperclip coding agents; this PR preparation, rebase, cleanup commit, and PR body were completed by OpenAI Codex in a Paperclip worktree. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
dbebf30c89 |
Add low-trust review containment (#7530)
## Thinking Path > - Paperclip is a control plane for AI-agent companies, so execution policy and trust boundaries are part of the product's safety contract. > - Low-trust review work needs narrower authority than normal same-company agents because hostile PRs, comments, attachments, and generated output can carry prompt-injection payloads. > - The current V1 shape gives trusted workers broad company context, which is useful for normal execution but too permissive for a reviewer assigned to hostile content. > - This branch adds a `low_trust_review` preset, source-trust tagging, route-level containment, and quarantine handling so low-trust output does not automatically flow into higher-trust wake context. > - The branch has been rebased onto current `origin/master`, and the low-trust migration was renumbered to `0097_low_trust_source_trust.sql` to avoid collisions with existing `0091` through `0096` migrations. > - Greptile feedback was addressed by tightening low-trust detection, preserving project-level trust policy checks, fixing issue-kind promotion lookup, removing duplicate post-lease isolation assertion, documenting fail-closed source-trust behavior, bounding ancestry checks, enforcing runtime issue context for CEOs, awaiting accepted-plan monitor authorization, and making low-trust issue source-trust tagging atomic. > - The benefit is a first production slice of deny-by-default review containment with regression coverage for the main control-plane pivot surfaces. Fixes #7531. ## What Changed - Added shared trust-policy types and validators, plus database/source-trust fields for issues, comments, documents, and work products. - Implemented server enforcement for low-trust issue scope, agent self-view redaction, secret/plugin/runtime denial paths, promotion checks, and quarantined continuation/wake context. - Added focused low-trust regression tests for resolver behavior, source trust, route authorization, heartbeat preflight ordering, runtime containment, and quarantine redaction. - Added board UI affordances for selecting/reviewing the low-trust preset and surfacing source-trust badges in relevant issue views. - Added `doc/LOW-TRUST-PRESETS.md`, updated `doc/SPEC-implementation.md`, and committed the low-trust review contract plan under `doc/plans/`. - Rebasing note: the original `0097_low_trust_source_trust.sql` migration was renamed to `0097_low_trust_source_trust.sql`; the SQL uses `ADD COLUMN IF NOT EXISTS` so users who already applied the old-numbered migration are not broken by the renumbered migration. ## Verification - Rebased branch onto current `origin/master` and force-pushed with lease to `origin/PAP-10211-low-trust-agent` at head `2719f31e3`. - Confirmed the PR diff does not include `pnpm-lock.yaml` or `.github/workflows` changes. - Resolved upstream UI/comment conflicts by preserving deleted-comment tombstone behavior and low-trust source-trust badges/metadata. - Renumbered the low-trust source-trust migration to `0097_low_trust_source_trust.sql`; the SQL uses `ADD COLUMN IF NOT EXISTS` so users who already applied an old-numbered copy are not broken. - `pnpm exec vitest run ui/src/lib/issue-chat-messages.test.ts server/src/__tests__/heartbeat-workspace-session.test.ts` - `pnpm exec vitest run server/src/__tests__/source-trust.test.ts server/src/__tests__/workspace-runtime-service-authz.test.ts ui/src/lib/trust-policy-ui.test.ts ui/src/components/TrustPresetSection.test.tsx` - `pnpm run typecheck:build-gaps` - `git diff --check` - GitHub checks pass on head `2719f31e3`: build, typecheck/release registry, general tests, serialized server suites, e2e, canary, verify, policy/review, Socket, and Snyk. - Greptile Review passes with Confidence Score 5/5 and zero unresolved Greptile review threads. - No design screenshots/images were added because the task explicitly says not to add them unless they are specifically part of the work. ## Risks - Medium risk: this touches shared trust-policy contracts, server authorization paths, heartbeat context generation, migration metadata, and UI preset controls. - Low-trust containment is intentionally deny-by-default; legitimate future review workflows may need explicit allowlisted exceptions. - Plugin/runtime/security surfaces are broad, so regression tests cover the current known routes but future integrations must route through the same containment layer. - The PR is ready for review; GitHub checks are green and Greptile is 5/5. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent, tool-enabled shell and GitHub CLI workflow. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] UI changes are covered by focused tests; no screenshots were added per task instruction not to add design images unless specifically required - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
03ad5c5bea |
[codex] Add issue document locking (#6009)
## Thinking Path > - Paperclip orchestrates AI-agent companies through company-scoped issues, comments, and issue documents. > - Issue documents are the durable place where plans, handoffs, and other work artifacts are revised over time. > - Some documents need to be preserved as operator-approved snapshots while agents continue working on the same issue. > - Without document locking, a later board or agent write can overwrite the document key that reviewers expected to remain stable. > - This pull request adds board-managed issue document locks and makes agent writes to locked keys create a derived document instead of mutating the locked document. > - The benefit is safer document handoffs: approved or frozen issue documents stay immutable until the board explicitly unlocks them. ## What Changed - Added `locked_at`, `locked_by_agent_id`, and `locked_by_user_id` document fields plus migration `0085_tranquil_the_executioner.sql`. - Added document lock/unlock service behavior, route endpoints, activity events, and locked-document write protections. - Made agent document writes to locked keys create a new derived key such as `plan-2` rather than overwriting the locked document. - Surfaced lock state through shared issue document types, UI API methods, document header lock controls, and activity formatting. - Added server and UI tests for lock/unlock behavior, locked document immutability, and UI action visibility. - Updated `doc/SPEC-implementation.md` with the V1 document lock contract and endpoints. ## Verification - `git rebase public-gh/master` completed cleanly after committing the branch changes. - `git diff --check` passed before commit. - `pnpm run preflight:workspace-links && pnpm exec vitest run server/src/__tests__/documents-service.test.ts server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts ui/src/components/IssueDocumentsSection.test.tsx ui/src/components/IssueContinuationHandoff.test.tsx ui/src/lib/document-revisions.test.ts` passed: 5 files, 32 tests. ## Risks - Medium risk because this changes the document persistence contract and adds a migration. - The migration uses `ADD COLUMN IF NOT EXISTS` and guarded foreign-key creation so it remains safe for users who may have already applied an earlier copy of the migration. - Locked documents intentionally reject board edits/deletes/restores until unlocked; any existing workflows that expected direct overwrite need to unlock first. - Agent writes to locked keys now create derived documents, which may create extra issue documents when agents retry locked writes. ## Model Used - OpenAI Codex coding agent based on GPT-5, with tool use and local code execution in the Paperclip worktree. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
236d11d36f |
[codex] Add run liveness continuations (#4083)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - Heartbeat runs are the control-plane record of each agent execution window. > - Long-running local agents can exhaust context or stop while still holding useful next-step state. > - Operators need that stop reason, next action, and continuation path to be durable and visible. > - This pull request adds run liveness metadata, continuation summaries, and UI surfaces for issue run ledgers. > - The benefit is that interrupted or long-running work can resume with clearer context instead of losing the agent's last useful handoff. ## What Changed - Added heartbeat-run liveness fields, continuation attempt tracking, and an idempotent `0058` migration. - Added server services and tests for run liveness, continuation summaries, stop metadata, and activity backfill. - Wired local and HTTP adapters to surface continuation/liveness context through shared adapter utilities. - Added shared constants, validators, and heartbeat types for liveness continuation state. - Added issue-detail UI surfaces for continuation handoffs and the run ledger, with component tests. - Updated agent runtime docs, heartbeat protocol docs, prompt guidance, onboarding assets, and skills instructions to explain continuation behavior. - Addressed Greptile feedback by scoping document evidence by run, excluding system continuation-summary documents from liveness evidence, importing shared liveness types, surfacing hidden ledger run counts, documenting bounded retry behavior, and moving run-ledger liveness backfill off the request path. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/server-utils.test.ts server/src/__tests__/run-continuations.test.ts server/src/__tests__/run-liveness.test.ts server/src/__tests__/activity-service.test.ts server/src/__tests__/documents-service.test.ts server/src/__tests__/issue-continuation-summary.test.ts server/src/services/heartbeat-stop-metadata.test.ts ui/src/components/IssueRunLedger.test.tsx ui/src/components/IssueContinuationHandoff.test.tsx ui/src/components/IssueDocumentsSection.test.tsx` - `pnpm --filter @paperclipai/db build` - `pnpm exec vitest run server/src/__tests__/activity-service.test.ts ui/src/components/IssueRunLedger.test.tsx` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm exec vitest run server/src/__tests__/activity-service.test.ts server/src/__tests__/run-continuations.test.ts ui/src/components/IssueRunLedger.test.tsx` - `pnpm exec vitest run server/src/__tests__/heartbeat-process-recovery.test.ts -t "treats a plan document update"` - `pnpm exec vitest run server/src/__tests__/activity-service.test.ts server/src/__tests__/heartbeat-process-recovery.test.ts -t "activity service|treats a plan document update"` - Remote PR checks on head `e53b1a1d`: `verify`, `e2e`, `policy`, and Snyk all passed. - Confirmed `public-gh/master` is an ancestor of this branch after fetching `public-gh master`. - Confirmed `pnpm-lock.yaml` is not included in the branch diff. - Confirmed migration `0058_wealthy_starbolt.sql` is ordered after `0057` and uses `IF NOT EXISTS` guards for repeat application. - Greptile inline review threads are resolved. ## Risks - Medium risk: this touches heartbeat execution, liveness recovery, activity rendering, issue routes, shared contracts, docs, and UI. - Migration risk is mitigated by additive columns/indexes and idempotent guards. - Run-ledger liveness backfill is now asynchronous, so the first ledger response can briefly show historical missing liveness until the background backfill completes. - UI screenshot coverage is not included in this packaging pass; validation is currently through focused component tests. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5.4, local tool-use coding agent with terminal, git, GitHub connector, GitHub CLI, and Paperclip API access. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge Screenshot note: no before/after screenshots were captured in this PR packaging pass; the UI changes are covered by focused component tests listed above. --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
dottaandPaperclip
|
c0d0d03bce |
Add feedback voting and thumbs capture flow
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
dottaandPaperclip
|
b0b9809732 |
Add issue document revision restore flow
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bc12f08c66 |
fix(issue-documents): address greptile review
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
eb0a74384e |
fix(issues): address document review comments
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
45998aa9a0 |
feat(issues): add issue documents and inline editing
Co-Authored-By: Paperclip <noreply@paperclip.ing> |