319 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	f91a6e27c0	feat(issues): contain cross-issue agent side effects (#10837) ... ## Thinking Path > - Paperclip is the control plane that coordinates autonomous agent work. > - Agents need to collaborate on issues beyond their current assignment. > - Cross-issue comments and updates are useful, but an unbounded run can create cascading side effects. > - The control plane must preserve company-wide collaboration while containing each run's influence. > - Comment attribution must also show the responsible user and the acting agent in audits. > - This pull request adds run-bound cross-issue containment, attribution, and agent-class wake rules. > - The benefit is safer collaboration without restoring issue-assignee ownership restrictions. ## Linked Issues or Issue Description **What existing behavior does this improve?** Agent-authenticated issue comments, updates, reopen behavior, and assignee wake routing. **Subsystem affected** Cross-cutting: server routes and services, shared contracts, database schema and migration, and implementation documentation. **Current behavior** An authenticated agent can collaborate across company issues, but one heartbeat run has no per-run side-effect boundary. Comment records also do not persist the responsible user separately from the acting agent. **Proposed behavior** Require a valid heartbeat run for agent cross-issue comments and updates. Audit each attempt and cap a run at 20 cross-issue effects. Keep the cap in log-only mode until it automatically changes to enforcement at 2026-08-11 00:00 UTC. Preserve same-issue writes. Use agent-class wakes for agent comments. Keep same-run completion comments from reopening completed work. Record the responsible user on agent-authored comments and activity. **Reason and benefit** Agents can collaborate on other issues without an assignment gate, while each run has an atomic and inspectable side-effect limit. Operators can identify both the acting agent and the responsible user. **Breaking changes** After 2026-08-11 00:00 UTC, the twenty-first cross-issue comment or update from one heartbeat run returns a containment error. Agent cross-issue writes without valid run context are rejected. The migration is additive and backfills existing agent-authored comment attribution where the source data is available. ## What Changed - Added an atomic per-run counter for cross-issue agent comments and updates. - Added audit events for allowed and rejected cross-issue effects. - Added the automatic log-only to enforcement flip at 2026-08-11 00:00 UTC. - Added responsible-user attribution to agent-authored comments, activity records, shared types, and validators. - Added an additive migration and migration coverage for existing comments. - Updated reopen, resume, and wake behavior so agent comments create agent-class wakes and same-run completion comments remain inert. - Updated the implementation specification and regression coverage. ## Verification - `pnpm exec vitest run server/src/__tests__/cross-issue-influence-limit.test.ts server/src/__tests__/issue-comment-attribution-audit-routes.test.ts server/src/__tests__/issue-comment-reopen-routes.test.ts packages/db/src/issue-comment-on-behalf-migration.test.ts` — 97 tests passed. - `pnpm -r typecheck` — passed, including migration safety checks. - `pnpm test:run` — server batch: 3,364 passed and 2 skipped; UI batch: 3,504 passed. One unrelated CLI doctor test warned because this agent runtime injects static AWS credentials. - `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY pnpm exec vitest run cli/src/__tests__/secrets.test.ts` — 8 tests passed and confirmed the CLI failure was ambient-environment sensitive. - `pnpm build` — passed. ## Risks - The fixed enforcement timestamp changes production behavior automatically on 2026-08-11 00:00 UTC. Audit logs before that time provide rollout visibility. - The per-run counter serializes on the heartbeat-run row. This prevents concurrent attempts from racing past the cap but adds a small lock scope for cross-issue writes. - Existing comments can only be backfilled when their acting run or agent attribution is recoverable. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 in the Codex agent runtime. The runtime did not expose a context-window size. Reasoning, shell tools, code editing, and test execution were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-04 13:17:49 -05:00
3 people	ded813ad6f	feat(interactions): add governed agent addressees (#10252) ... ## Thinking Path > - Paperclip is the control plane that lets humans govern companies of AI agents. > - Issue-thread interactions are the structured handoff point for confirmations, questions, suggested tasks, and other governed decisions. > - Those interactions previously assumed that only board users could resolve them, preventing one agent from explicitly addressing another agent for a response. > - Agent resolution needs company-level governance, auditable resolver identity, safe terminal-state handling, and attention routing so authorization is enforced server-side rather than inferred from UI behavior. > - This pull request adds governed agent resolution, withdrawal and terminal expiry semantics, explicit agent addressees, lifecycle reconciliation, and attention-feed filtering. > - The benefit is that agents can participate in structured decisions without weakening board control, company isolation, wake behavior, or audit invariants. ## Linked Issues or Issue Description ### Subsystem affected Issue-thread interactions across database, shared contracts, server authorization/services, adapter callbacks, agent skill guidance, API docs, and UI governance surfaces. ### Problem or motivation Structured interactions were board-only, had no explicit agent addressee, and lacked durable withdrawal/terminal-expiry semantics. That made peer-agent decisions impossible to authorize and audit safely. ### Proposed solution Persist requested/effective resolver policy and addressee identity, enforce company governance and eligible agent resolution, reconcile addressee lifecycle changes, expose withdrawal and terminal expiry, and route attention to the intended active agent with board fallback. ### Alternatives considered Implicitly authorizing the issue assignee or mentioned agents was rejected as ambiguous and difficult to audit. Using comments alone was rejected because it loses structured outcomes and continuation behavior. ### Roadmap alignment Supports the ROADMAP direction for lightweight leadership-agent communication that still resolves into governed decisions and work objects. ### Additional context Public GitHub issue/PR search found no duplicate implementation; open PR search for interaction resolver governance and agent addressees only returned this PR. ## What Changed - Add company-scoped interaction resolver governance contracts and persistence. - Add requested/effective resolver policy, resolver identity, withdrawal, and terminal-expiry behavior. - Add explicit `addresseeAgentId` validation, authorization, persistence, lifecycle reconciliation, API documentation, and skill guidance. - Route pending addressed interactions to the intended invokable agent and fall back to board attention when that agent becomes ineligible or is deleted. - Preserve sandbox callback identity fields required by governed resolution paths. - Add migrations `0193` and `0194` plus route, service, attention, adapter, CLI, and UI coverage. - Add governance state and company settings UI, including responsive mobile behavior and distinct withdrawn/expired audit presentation. ## Verification - `pnpm check:token-gates` — passed. - `pnpm -r typecheck` — passed, including migration numbering and safety checks. - `pnpm test:run` — feature/server and UI workspace suites passed; one unrelated CLI AWS doctor test observed injected static AWS credentials and warned instead of passing. - `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY pnpm exec vitest run cli/src/__tests__/secrets.test.ts --project paperclipai` — 8 tests passed, confirming the failure was environment-sensitive. - `pnpm build` — passed. - Latest rebased head `e24cece6be9f1877bdbac7691bcb44fd583c0161` completed all GitHub CI jobs successfully. ## Risks - Migrations add interaction and company-governance fields; numbering is conflict-free on current `master`, additive statements are idempotent, and migration safety checks pass. - Agent authorization behavior expands beyond board-only resolution, but defaults remain board-only and coverage exercises company boundaries, resolver eligibility, lifecycle invalidation, wake behavior, withdrawal, expiry, and attention fallback. - Attention routing depends on current agent invokability; reconciliation and read-time filtering prevent stale addressees from retaining visibility or resolution authority. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using `gpt-5.6-sol` with reasoning, terminal tool use, code execution, Git/GitHub integration, and Paperclip control-plane tools. Context-window metadata was not reported by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-08-04 12:51:06 -05:00
Devin Foley	cb52f0b750	db: env-configurable client options; parallelize attention feed queries (#10795) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server stores all state in PostgreSQL through Drizzle and the postgres.js driver > - Self-hosted installs run Postgres on localhost, so per-query latency is near zero; hosted installs often attach Postgres over a network, sometimes through a transaction-mode pooler > - The DB client passes no options to the driver, so operators cannot disable prepared statements or tune the pool without a source edit, and the deploy docs told them to edit `client.ts` > - The attention feed also runs its related-data lookups one after another, so its latency grows as queries × network round trip > - This pull request adds optional environment configuration for the DB client and batches the independent attention-feed lookups with `Promise.all` > - The benefit is that network-attached deployments get correct pooler support and a much faster attention feed, while self-hosted behavior does not change ## Linked Issues or Issue Description No public issue exists for this; description follows the bug report template: **What happened?** On deployments where PostgreSQL is network-attached (managed providers, pooled endpoints), the attention feed endpoint is slow: `attentionService.list()` awaits ~15–20 queries strictly in sequence, so a 70ms round trip turns into more than one second of pure network wait per call. Separately, connecting through a transaction-mode pooler (pgbouncer, Supavisor port 6543, Neon `-pooler` hosts) requires disabling prepared statements, and the only documented way was to hand-edit `packages/db/src/client.ts` — which `doc/DATABASE.md` itself tells operators not to do. **Expected behavior** The DB client is configurable from the environment (prepared statements, pool size, timeouts) with driver defaults when unset, and hot read paths do not multiply network latency by issuing independent queries sequentially. **Steps to reproduce** 1. Run the server with `DATABASE_URL` pointing at a Postgres instance with ~70ms round-trip latency. 2. Open the attention feed (`GET /companies/:companyId/attention`) and measure response time — it exceeds one second even with little data. 3. Try to connect through a transaction-mode pooler: there is no supported configuration to disable prepared statements. ## What Changed - `packages/db/src/client.ts`: `createDb` accepts a `DatabaseClientOptions` argument and reads optional env config — `DATABASE_PREPARED_STATEMENTS`, `DATABASE_POOL_MAX`, `DATABASE_IDLE_TIMEOUT_SECONDS`, `DATABASE_CONNECT_TIMEOUT_SECONDS`. When nothing is set, no option is passed to the driver and behavior is identical to the previous bare `postgres(url)`. - `packages/db/src/client-options.test.ts` (new): env parsing and driver-option mapping tests, including malformed-value rejection. - `server/src/services/attention.ts`: the independent related-data lookups in each feed section now run under `Promise.all` (issue summary/image/plan-document maps, decision bundle titles, blocked-issue maps, the newer-runs scan). Section order, item assembly, and query shapes are unchanged. - `doc/DATABASE.md` and `docs/deploy/database.md`: the edit-source pooling instruction is replaced with the env toggle, plus a short client-tuning reference. ## Verification - `pnpm --filter @paperclipai/db exec vitest run src/client-options.test.ts` — 6 tests pass. - `pnpm --filter server exec vitest run src/__tests__/attention-service.test.ts` — 22 tests pass. - `pnpm --filter server exec vitest run src/__tests__/decisions-service.test.ts src/__tests__/decision-training.test.ts` — 45 tests pass; this covers the call path that runs `attentionService.list()` inside `db.transaction`, where postgres.js serializes queries on the reserved connection. - `tsc` reports no errors in the changed files. ## Risks - Low risk for self-hosted installs: with no env vars set, `postgres(url, {})` receives an empty options object, which postgres.js treats the same as no options — driver defaults throughout. - The `Promise.all` batches only group queries that had no data dependency on each other; on the transaction call path the driver still executes them one at a time on the reserved connection, so transactional semantics are unchanged. - Malformed env values now fail fast at startup with a clear message instead of being silently ignored; this is intentional and only affects operators who set the new variables. ## Model Used Claude Fable 5 (`claude-fable-5`), Anthropic — via Claude Code CLI, extended thinking enabled, tool use (test execution, live latency measurement against a network-attached Postgres to size the problem). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (searched "prepared statements", "pgbouncer", "pool", "attention feed", "lockfile" — closest matches are #10573/#10787 lockfile chores, unrelated to this change) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-04 06:30:36 -07:00
Devin FoleyandPaperclip	185515c97b	fix(external-objects): refresh PR status labels (#10704) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The issue properties panel can show external objects such as GitHub pull requests. > - Those objects are resolved by external-object providers and then displayed as compact status labels. > - A GitHub pull request could remain in the fallback `unknown` state and appear as `Not yet resolved`. > - That label is confusing when the object is known but has not been refreshed yet. > - This pull request refreshes due external objects from the heartbeat scheduler and improves the unknown-status copy. > - The benefit is a properties panel that moves from pending refresh to the real pull request state without a manual refresh. ## Linked Issues or Issue Description No public GitHub issue exists for this bug. I searched for related public issues and pull requests using the terms `Not yet refreshed`, `external objects refresh`, and `external PR status`, and did not find a duplicate implementation. **What happened?** The issue properties panel could show a GitHub pull request as `Not yet resolved` even when the referenced pull request was valid. The object stayed stale unless a manual refresh path ran. **Expected behavior** A known external object should show pending-refresh copy while it waits for provider data. When the scheduler refreshes it, the properties panel should show the provider status such as open, merged, or closed. **Steps to reproduce** 1. Create or view an issue that references a GitHub pull request. 2. Open the issue properties panel. 3. Observe the external object row before a manual refresh has run. **Paperclip version or commit** Current `master` before this pull request. **Deployment mode** Local dev and self-hosted server. **Installation method** Built from source. **Agent adapter(s) involved** Not adapter-specific. **Database mode** Not database-related. **Access context** Board view. **Privacy checklist** I reviewed this description and did not include logs, credentials, private URLs, internal issue IDs, or PII. ## What Changed - Added a heartbeat scheduler tick that refreshes due external objects for active companies. - Kept manual external-object refresh behavior on the same service path. - Changed display copy so known provider objects use liveness labels such as `Not yet refreshed`, while fresh unknown provider statuses show `Status unavailable`. - Added server and UI tests for scheduled refresh and label behavior. ## Verification - `corepack pnpm install --frozen-lockfile` - `pnpm check:token-gates` - `pnpm exec vitest run server/src/__tests__/external-objects-service.test.ts server/src/__tests__/server-startup-feedback-export.test.ts ui/src/components/ExternalObjectPill.test.tsx ui/src/components/IssueProperties.test.tsx ui/src/lib/external-objects.test.ts` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/server build` - `pnpm --filter @paperclipai/ui build` - `pnpm run typecheck:build-gaps` - GitHub PR checks passed on head `0e7fcd30` - Greptile reported 5/5 on head `0e7fcd30` with no unresolved review threads Notes: - I ran recursive typecheck and build first. Both hit container resource limits with exit 137 during concurrent package work, so I reran the affected server and UI targets separately. - An unrelated workspace-runtime auto-port test fails in this container with a PID ownership mismatch. It is outside the files changed here. ## Risks Low to medium risk. The scheduler does more periodic external-object work, so the main risk is extra provider refresh load. The implementation bounds the work to active companies, due non-terminal objects, and 50 objects per company per tick. The path also stays behind the external-objects experimental setting. ## Model Used OpenAI GPT-5 Codex in the Codex execution environment, with shell and GitHub CLI tool use. The runtime did not expose a more specific internal model ID or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-02 20:24:52 -07:00
3 people	0a09e4d975	feat(decisions): add desk workflow and retention (#10672) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Decisions desk shows work that needs a human decision > - The queue foundation can group and rank decision work > - Operators also need a focused daily view and a safe way to handle old work > - This pull request adds the desk controls, the aging shelf, and reversible retention > - It also binds bulk archive decisions to the exact reviewed item set > - The benefit is a smaller daily queue without lost or orphaned work ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting: server, UI, database, and shared contracts. **Problem or motivation** Decision work can grow into one large company list. Operators need quick queue and date controls. Old items also need a safe retention path that does not delete work. **Proposed solution** Add queue and date controls to the Decisions desk. Compute the aging shelf on the server. Archive idle items after 90 days unless an operator keeps them. Keep archived items searchable and revivable. Notify origin agents in one batch per sweep. Bind bulk archive proposals to a signed, exact item manifest. **Alternatives considered** Client-only aging can drift across browsers and source kinds. Deleting old rows removes audit and recovery paths. An unsigned dynamic bulk query can archive items that the reviewer did not inspect. **Roadmap alignment** This work supports the Work Queues and decision-memory directions in `ROADMAP.md`. It extends the Decisions and attention-feed foundation from #10651. Related earlier work includes #9380, #10010, and #10474. ## What Changed - Added the queue rail, date chips, decide split, triage strip, queue page, and aging shelf UI. - Added server-owned shelf state with per-queue retention overrides. - Added reversible retention state, archive history, and an idempotent notification outbox. - Added the 90-day archive sweeper, Keep exemption, archived feed query, and revive actions. - Added one origin-agent notification per agent and sweep. - Added signed bulk archive proposals with exact-set and version checks. - Persisted queue-exclusion reasons atomically and kept cross-domain source resolution per-item until it has an exact-set transaction contract. - Added API contracts, OpenAPI entries, migration coverage, focused tests, and Storybook screens. ## Verification - `pnpm -r typecheck` - `pnpm test:run` (server: 329 files and 3,453 tests passed; UI: 408 files and 3,362 tests passed) - `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY pnpm exec vitest run cli/src/__tests__/secrets.test.ts` - `pnpm build` - `pnpm check:token-gates` - Focused retention, attention, decisions, migration replay, startup, and UI API tests. - Complete queue snapshot regression with 51 items across the normal 50-item page boundary. The unmodified CLI test reports one warning assertion in this runtime because the harness injects static AWS credential variables. The isolated test passes when those two variables are removed. ## Risks - The migration adds retention and notification outbox tables. It uses idempotent table, index, and foreign-key creation. - Retention runs on the heartbeat scheduler interval. Compare-and-set version checks prevent stale archive writes. - Bulk archive acceptance fails closed when authority, activity, version, or the reviewed set changes. - Archive is reversible and does not delete source records. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex with `gpt-5.6-sol`. The model used tool calls, code execution, database migration generation, and test execution. The context-window size is not exposed in this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-02 10:47:03 -05:00
DottaandPaperclip	30c49c8327	feat(decisions): add queues and prioritized attention feed (#10651) ... <!-- Write all pull request text in Simplified Technical English (ASD-STE100): short sentences, one instruction per sentence, simple approved vocabulary, and the active voice. --> ## Thinking Path > - Paperclip is the open source control plane for companies of AI agents. > - Operators use the attention feed to find decisions that need action. > - The feed has eleven source kinds, but it has no durable queue or triage state. > - The feed also returns every item and lacks decision deadlines, snooze state, and decision-focused ordering. > - This pull request adds secure queue sidecars and enriches the attention feed with triage data, filters, cursor pagination, and decide-now ranking. > - The benefit is a bounded feed that can show the most urgent decisions first without weakening source visibility rules. ## Linked Issues or Issue Description This pull request replaces the closed [#10634](https://github.com/paperclipai/paperclip/pull/10634). It combines that queue foundation with the dependent attention-feed change as one review unit. **Subsystem affected** Database schema, shared contracts, server authorization and REST APIs, and the UI attention client library. **Problem or motivation** The attention feed can contain hundreds of mixed decision items. Operators cannot group them into durable queues, set a decision deadline, snooze an item, or request a bounded page ordered by urgency. The current client must download the full feed on each refresh. **Proposed solution** Store queue membership and triage state by stable attention identity. Re-authorize each source during queue reads and writes. Enrich attention items with queue, deadline, snooze, expiry, rule, and origin data. Add activity and queue filters, opaque cursor pagination, decide-focused ordering, and a decide-now count. **Alternatives considered** Adding queue fields to every source would duplicate schema and authorization logic across eleven source kinds. Client-only filtering and sorting would still transfer the full feed and would make pagination unstable. **Roadmap alignment** This change improves the core decision-attention surface and operator oversight. It does not implement the separate general-purpose work queue milestone in `ROADMAP.md`. ## What Changed - Added company-scoped queue, membership, triage, and append-only event tables with actor and run provenance. - Added queue CRUD, item membership, starter-rule discovery, and decide-by and snooze endpoints. - Kept source authorization on each queue mutation, read, and count. - Added attention fields for expiry, rule, origin agent, queues, decide-by attribution, and snooze state. - Added activity date filters, queue filters, opaque cursor pagination, and configurable page limits. - Added decide-now ordering by deadline, expiry, severity, and activity. - Added `decideNowCount` and excluded actively snoozed items from the default feed. - Updated the shared and UI client contracts. - Added focused server, route, OpenAPI, and UI client tests. ## Verification - `pnpm exec vitest run server/src/__tests__/attention-service.test.ts server/src/__tests__/decision-queues-routes.test.ts server/src/__tests__/openapi-routes.test.ts ui/src/api/attention.test.ts ui/src/lib/attention.test.ts` (72 tests passed) - `pnpm --filter @paperclipai/db check:migrations` - `pnpm -r --filter @paperclipai/db --filter @paperclipai/shared --filter @paperclipai/server --filter @paperclipai/ui typecheck` - `git diff --check origin/master...HEAD` ## Risks - The migration adds four company-scoped tables and provenance foreign keys. Migration numbering and safety checks pass. - Attention reads can lazily create starter queues and memberships. Inserts are idempotent, audited, and transactional. - Cursor validity depends on the filtered feed. The API returns a clear validation error when the cursor item no longer exists in that feed. - Queue reads re-check source visibility. This favors correct authorization over fewer queries. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, model `gpt-5`. The runtime used agentic reasoning, repository tools, code execution, and test execution. The runtime did not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-01 20:37:39 -05:00
3 people	9c1f8e7887	feat(decisions): add first-class propose mode (#10010) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents can currently perform many mutations directly, while humans often need a durable review point before cross-issue or destructive actions occur > - Existing approvals and issue-thread interactions do not provide a standalone, reusable object for presenting options, collecting typed inputs, detecting stale targets, and auditing effect execution > - The control plane therefore needs a first-class propose mode that separates an agent's recommendation from the governed mutation it may cause > - This pull request adds Decisions v1 across the database, shared contracts, server execution and telemetry, agent skill guidance, and operator UI > - The benefit is that agents can propose multi-option actions safely while operators get explicit provenance, fail-closed execution, per-effect results, and a focused attention workflow ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting: `packages/db`, `packages/shared`, `server`, and `ui`. ### Problem or motivation Agents need a governed way to propose consequential work without immediately mutating issues, especially when one choice can affect several issue trees. Existing approvals and issue-thread interactions do not provide a standalone object with typed options, target snapshots, effect-level authorization, expiration, execution outcomes, and reusable attention-feed presentation. ### Proposed solution Add first-class Decisions that store options and typed inputs, surface open proposals in the operator attention feed, validate target freshness and the origin-agent/operator authorization intersection at decision time, execute a bounded set of auditable effects, and retain terminal outcomes. Decisions v1 supports comments, status and assignee changes, follow-up issue creation, blocker resolution, and issue-tree cancellation, plus bundle grouping, expiration/dismissal, rule-key telemetry, and agent-facing API guidance. ### Alternatives considered - Extend approvals with arbitrary effects: rejected because approvals represent governed yes/no actions and would become an unsafe generic mutation envelope. - Model every proposal as an issue-thread interaction: rejected because decisions can span several targets and need independent lifecycle, telemetry, idempotency, and effect results. - Let agents perform the mutation and ask for retrospective review: rejected because it removes the pre-execution governance boundary this feature is meant to provide. ### Roadmap alignment Aligns with `ROADMAP.md` sections **Agent Reviews and Approvals**, **Enforced Outcomes**, **MCP Tool Gateway & Apps (governed tool access)**, and **Activity History** by making explicit decisions, authorization gates, auditable execution, and terminal outcomes first-class control-plane objects. ### Additional context This does not replace existing approvals or issue-thread interactions, and it does not add an unrestricted generic mutation effect. ## What Changed - Added company-scoped decision, option, target, and effect-execution schema plus migration and shared TypeScript/Zod contracts. - Added decision routes and services for propose, list/get, decide, dismiss, cancel, target freshness checks, authorization intersection, idempotency, activity logging, and execution auditing. - Added rule-key decision telemetry and attention-feed metadata so open decisions are visible and measurable. - Added agent skill documentation for proposing and resolving decisions through the Paperclip API. - Added the Decisions UI: API client, query keys, inline attention resolver, bundle grouping, target-issue strip, terminal history, destructive confirmation, and per-effect result rendering. - Added server service coverage, DecisionCard state tests, and Storybook stories for the supported visual states. ## Verification - `pnpm -r typecheck` — passed. - `pnpm test:run` — 2,876 passed, 1 skipped, with one unrelated cross-suite cleanup-order failure in `heartbeat-responsible-user-invariant.test.ts`; the failing file passes in isolation (`6/6`). - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-responsible-user-invariant.test.ts` — passed. - `pnpm --filter @paperclipai/ui exec vitest run src/components/DecisionCard.test.tsx` — passed (`9/9`). - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/authz-existence-oracle-guard.test.ts src/__tests__/openapi-routes.test.ts` — passed (`5/5`). - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/decisions-service.test.ts` — passed (`16/16`). - `pnpm --filter paperclipai exec vitest run src/__tests__/company-import-export-e2e.test.ts` — passed (`1/1`). - `pnpm --filter @paperclipai/server typecheck` and `pnpm --filter paperclipai typecheck` — passed. - `pnpm build` — passed. - Rebased-head focused suite — passed (`6` files, `88` tests): shared decision contracts, Decisions service, OpenAPI routes, startup feedback export, DecisionCard states, and attention helpers. The follow-up stale-secondary-target regression passes in the DecisionCard suite (`10/10`). - Rebased-head scoped typechecks — passed for `@paperclipai/shared`, `@paperclipai/db`, `@paperclipai/server`, and `@paperclipai/ui`. - Rebased-head migration numbering and safety checks — passed after renumbering the additive migration to `0193` and making it replay-safe for environments that applied the earlier feature-branch number. - `pnpm check:token-gates` — passed with all gates clean. - GitHub PR workflow and Greptile review for `1f9f7645882d05dfdd9c99377c03a1f53f20e8be` — running after the stale-secondary-target fix and PR metadata refresh on July 27, 2026. - `pnpm --filter @paperclipai/ui build-storybook` exposes an existing Storybook version mismatch (`storybook` 10.4.6 vs `@storybook/addon-docs` 10.5.0); Decisions stories were validated with the docs addon temporarily disabled and the tracked config remains unchanged. ## Risks - **Migration:** Adds replay-safe migration `0193`; migration numbering and safety checks pass. The new tables and indexes are additive. - **Authorization:** Effect execution intersects the proposing agent's permissions with the responsible user context and fails closed; mistakes could reject a valid proposal rather than silently over-authorize it. - **Concurrency:** Target snapshots and idempotency keys protect against stale or duplicate execution, but reviewers should focus on mixed-effect partial outcomes and retry behavior. - **UI:** Decisions are integrated into the existing attention feed rather than a separate navigation surface, reducing routing risk but increasing the importance of attention-item metadata compatibility. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex CLI using `gpt-5.6-sol` for final PR preparation, review fixes, and verification; repository tools and code execution were enabled, and context-window size is not exposed in this runtime. - Anthropic Claude Opus 4.8 with 1M context assisted with the Decisions UI implementation, as recorded in the relevant commits. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-31 19:17:02 -07:00
3 people	fc5a30805e	feat(cli): add managed install, update, and service lifecycle (#10045) ... ## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies > - Operators need a predictable installation path that survives beyond an ephemeral `npx` process > - A durable installation needs an owned per-user payload store, stable command shim, safe shell integration, and supported service lifecycle > - Updates must preserve recoverability by backing up data, installing side-by-side, verifying the new payload, and retaining rollback state > - Bootstrap scripts and privileged service operations must fail closed across download, filesystem, ownership, and consent boundaries > - This pull request integrates managed install, update, rollback, service, uninstall, doctor, bootstrap-installer, and runtime-serving support into one workflow > - The benefit is a recoverable, inspectable, and documented installation lifecycle with explicit safety boundaries across Linux, macOS, containers, WSL, npm, npx, and source checkouts ## Linked Issues or Issue Description ### Problem Paperclip lacks a first-class durable installation and lifecycle workflow. Operators currently have to assemble npm/npx installation, PATH setup, background-service management, updates, rollback, diagnostics, and uninstall behavior themselves. That makes upgrades harder to recover, creates inconsistent behavior across platforms, and leaves shell/download/service trust boundaries without one documented implementation. ### Proposed Solution Add a managed per-user install store and stable shim, a verified shell bootstrap installer, service lifecycle commands, install-mode-aware update/rollback behavior, doctor checks, and documentation. Managed updates back up the database, install and smoke-test a side-by-side payload, atomically switch `current`, and retain prior payloads. The shell installer pins registry/download trust boundaries and requires explicit consent for non-interactive privileged actions. ### Alternatives Considered - Keep recommending `npx`: simple for evaluation, but ephemeral and unsuitable for stable services, atomic updates, or rollback. - Require global npm installation only: familiar, but cannot provide the owned side-by-side payload store and retained rollback semantics. - Split the capability across multiple PRs: rejected because install, update, service, uninstall, bootstrap, and serving behavior share contracts and security boundaries that need review together. ### Related Pull Requests - Supersedes #10042 and #10044 with one integrated final diff. - Incorporates and replaces the closed preparatory work in #10032 and #10034. ## What Changed - Added `paperclipai install`, `update`/`upgrade`, rollback, uninstall, service lifecycle, onboarding integration, and managed-install doctor checks. - Added a private managed payload store, verified manifest/marker ownership, exclusive mutation locks, atomic manifest/current/shim writes, retained previous payloads, and provenance validation. - Added npm and GitHub-ref install sources with exact target resolution, registry isolation, database backup, side-by-side verification, atomic activation, service restart coordination, and failure rollback. - Made managed-update backups report actionable service-start and `--no-backup` recovery guidance for unreachable databases, while clean never-onboarded instances skip an empty backup. - Added systemd user and launchd service definitions, status/health/log commands, single-instance coordination, stale-port recovery, and explicit sudo/lingering consent handling. - Added the `scripts/install.sh` bootstrap path with checked two-stage downloads, pinned public npm registry usage, platform checks, dry-run/non-interactive controls, and Docker fixtures. - Added embedded Postgres/native bootstrap integration, hot-restart/systemd-notify serving support, passive update notices, configuration contracts, README/CLI/install documentation, and focused regression tests. - Security re-review should explicitly re-verify: (1) `addManagedPathBlock`/`removeManagedPathBlock` reject symlinked or non-regular rc files, assert current-user ownership, preserve restrictive modes, and replace atomically; (2) managed shim replacement rejects unsafe parents, foreign-owned or multiply linked files, and uses checked atomic replacement; (3) the shell installer and sudo path preserve explicit consent and checked downloads; and (4) installed service/runtime serving remains bound to the validated managed shim and instance configuration. ## Verification - `bash -n scripts/install.sh scripts/clean-install-git.sh scripts/clean-install-npm.sh scripts/test-install-sh-docker.sh` - `pnpm exec vitest run cli/src/__tests__/install-store.test.ts cli/src/__tests__/install-command.test.ts cli/src/__tests__/managed-install-check.test.ts cli/src/__tests__/onboard-service.test.ts cli/src/__tests__/service-health-check.test.ts cli/src/__tests__/service-manager.test.ts cli/src/__tests__/update-command.test.ts cli/src/__tests__/update-notice.test.ts packages/db/src/embedded-postgres-native.test.ts` — 9 files, 66 tests passed - `pnpm --dir cli typecheck` - `pnpm --dir cli build` - Follow-up verification: `pnpm exec vitest run cli/src/__tests__/update-command.test.ts` (14/14), `pnpm --dir cli typecheck`, `pnpm --dir cli build`, and `pnpm --filter @paperclipai/server typecheck`. - `pnpm -r typecheck` - `pnpm build` - Full `pnpm test:run` exercised all suites; an injected static AWS credential changed one unrelated doctor expectation, which passed when those credentials were removed. A second run cleared that case and exposed stale pre-existing adapter-utils `dist` output; rebuilding `@paperclipai/adapter-utils` made the isolated test pass. The updated PR CI is the authoritative clean-workspace full-suite run. ## Risks - Installer/update code writes executable shims, symlinks, shell rc blocks, service definitions, and managed payloads; ownership, regular-file, symlink, hard-link, marker, and path-containment checks fail closed before destructive changes. - The bootstrap installer executes downloaded tooling; downloads are staged and checked before execution, npm traffic is pinned to the public registry, and non-interactive privileged behavior requires explicit consent. - Linux lingering may invoke `sudo`; the command is surfaced and confirmed before execution, and unsupported service managers fall back to foreground-run guidance. - Database migrations remain forward-only; payload rollback does not reverse migrations, so managed updates create a backup before activation unless explicitly disabled. - Service restart and runtime serving touch process/port ownership; lifecycle locks, health/version checks, and stable-shim service definitions reduce split-brain and stale-process risk. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agents using GPT-5.5 and GPT-5.6-sol, with reasoning, repository/API access, shell execution, and test tooling. The runtime did not expose a reliable context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-31 18:52:23 -07:00
Nicky LeachandPaperclip	4813ed3f0c	fix(db): harden embedded Postgres test start with bounded retry (#10540) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The database layer uses embedded Postgres for isolated test runs > - A port probe can fail when another process takes the same port before Postgres binds it > - That race can make a test fail even when the code under test is fine > - This pull request adds bounded retry and clearer error text to the embedded Postgres start path > - The benefit is more stable tests and faster diagnosis when startup still fails ## Linked Issues or Issue Description No public GitHub issue exists for this change. This PR fixes a flaky embedded Postgres test start path. The helper can lose a port between probe and bind. This PR retries the start with a fresh port and a fresh data directory. Related public context: - Refs #7259 - Refs #9769 ## What Changed - Add bounded retry around embedded Postgres initialization and start. - Stop each failed attempt and remove its data directory before the next attempt. - Capture Postgres output in the thrown error so the failure is easier to read. - Add unit coverage for retry success, retry exhaustion, and the improved error text. ## Verification - `pnpm --filter @paperclipai/db exec vitest run src/test-embedded-postgres.test.ts src/embedded-postgres-error.test.ts` - `pnpm --filter @paperclipai/db exec vitest run` - `pnpm --filter @paperclipai/db exec vitest run` passed in the worktree after the change. - `worktree.test.ts > quarantines copied live execution state in seeded worktree databases` passed. - A real cluster loop of 100 starts passed with 0 failures. ## Risks - The retry can hide a real startup fault until the fifth try. - The bound keeps the wait short, and the final error still shows the captured Postgres log. - This change only affects the embedded Postgres test start helper. ## Model Used OpenAI Codex, GPT-5, tool-use enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-30 22:22:31 -07:00
Devin Foley	916c13501f	Replace host-to-host Cloud Sync with full-fidelity company Import/Export (#10507) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A company accumulates real state — issues, labels, blockers, documents, work products, monitors, attachments, agents, routines — and people need to move that state between instances: self-hosted to cloud, cloud back to self-hosted, or plain backups > - The experimental, flag-gated Cloud Sync transport (#6548) tried to solve this host-to-host: the source pushed into a receiver over HTTPS with a cross-instance consent/token handshake, which required the destination to be publicly reachable and broke for common self-hosted topologies (plain-HTTP LAN/VPN origins); the receiver half never landed upstream at all > - Meanwhile the portability bundle and the existing export/import pages already move companies offline with none of those networking constraints — but silently dropped labels, blockers, issue documents, work products, monitors, and every attachment > - This pull request removes the host-to-host transport and makes Import/Export the single data-movement path: the pages become first-class company-settings destinations, exports declare exactly what they do not carry, and bundle schemaVersion 6 now carries all of the above, with attachments as content-addressed sha256 blobs verified before a single row is written > - The benefit is a migration and backup flow that works between any two instances with no reachability requirements, no cross-instance auth, and no silent data loss ## Linked Issues or Issue Description - Refs #6548 — the original Cloud Sync sender this PR supersedes and removes. - Related, not duplicates: #1697 (goals in the portability manifest — orthogonal field addition), #954 (an earlier import/export + skill-visibility proposal predating the current portability bundle). - No open issue describes this directly, so in brief (feature-request shape): **Problem** — moving a company between instances silently lost labels (imports with label references actually hard-failed), blocker relations, issue documents, work products, monitor state, and all attachments, and the alternative Cloud Sync transport required the destination to be publicly reachable over HTTPS plus a consent handshake, which failed for typical self-hosted setups. **Desired behavior** — one Import/Export flow in company settings that produces a portable bundle carrying all of that data, tells the operator up front what it cannot carry, imports with automations paused, and offers real one-click activation afterwards. ## What Changed - New export fidelity report (`GET /api/companies/:companyId/export/fidelity`) + an "Export fidelity" panel on the Export page listing anything a bundle will not include (now only: approvals, cost history, activity history) - Imports accept `pauseAutomations`; imported agents and routines land paused, the import result reports created routines, and the Import page ends in an activation panel that actually resumes selected agents/activates routines - Export and Import pages promoted into the company-settings nav; the Cloud Upstream wizard, ux-lab page, and API client removed; the old settings route redirects to Export - Host-to-host transport removed: upstream-sync/receiver-client routes and services, CLI `cloud connect`/`cloud push` + keypair store, the shared upstream transfer contract, and the `enableCloudSync` flag; migration `0196` drops the two experimental `cloud_upstream_*` sender tables - Bundle schemaVersion 6: labels (definitions + per-task names, remapped by name on import), blocker relations (`blockedBy` slugs, cycle-tolerant), issue documents (`tasks/<slug>/documents/<key>.md`), work products (system refs nulled), monitors (notes/scheduledBy restored, imported un-armed) - Attachments travel as content-addressed `blobs/<sha256>` entries (deduped; comment-scoped attachments re-link via comment index); every blob is hash-verified **before any write**, so a corrupted bundle cannot leave a partially imported company; both zip codecs now round-trip extensionless/binary entries byte-exactly; the Import page preflights the inline body limit and offers continue-without-attachments - v5 (and older) bundles still import, with an informational warning; bundles newer than v6 are rejected cleanly - Docs: board-operator import/export guide, CLI README, README/ROADMAP updated ## Verification - `pnpm -r` typechecks (shared, db incl. migration numbering/safety checks, server, ui, cli) and `pnpm check:token-gates` — clean - Vitest: full server + shared sweep 4,888 passed / 1 skipped, with the only 3 failures being pre-existing on `master` (2× heartbeat-workspace-branch-containment, 1× workspace-runtime auto-port; reproduced identically with this change stashed); ui + cli suites green; the embedded-Postgres export-fidelity suite applies the full migration chain including the new `0196` against a fresh database - Live end-to-end on a scratch instance: seeded a company with labels, a blocker pair, an issue document, a work product, a monitor, an agent, a routine, and two binary attachments (one comment-scoped) → export → import into a fresh company → labels remapped to new ids, blocker edge and document restored, monitor un-armed with notes intact, attachments byte-identical (sha256-compared through the API), agents/routines paused → activation panel resumed them; a v5-shaped bundle imported with only the info warning; flipping one byte in a blob made the import 422 with **zero** rows created - Reviewer repro: create a company with a labeled issue + attachment → Settings → Export → download → Settings → Import on another company/instance → watch the preview, apply with "start paused", then activate ## Risks - Migration `0196` drops `cloud_upstream_connections`/`cloud_upstream_runs` — experimental tables behind a default-off flag; their connection/run history is intentionally discarded - Breaking removals are all of experimental, flag-gated surface: `/api/upstream-sync/*` + `/api/cloud-upstreams/*` routes, `paperclipai cloud connect|push`, and the `enableCloudSync` flag (stale keys in stored instance settings parse harmlessly) - Import remains non-atomic on mid-apply errors generally (pre-existing behavior); the new blob verification specifically moved ahead of all writes so tampered bundles cannot create partial state - GitHub-sourced imports do not fetch `blobs/*` and skip attachments with a warning ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic), via Claude Code CLI with extended thinking, tool use, and subagent orchestration; implementation and review split across Fable 5 subagents, with live end-to-end verification against a running instance ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-30 11:37:00 -07:00
Devin Foley	dc12197cce	fix: prevent duplicate built-in agents and self-heal reconciliation (#10223) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Every company is auto-provisioned a set of built-in agents (e.g. the Summarizer), and a startup reconciler keeps that set correct across every company on boot. > - Provisioning marks these agents with `metadata.paperclipBuiltInAgent.key`, but nothing in the database enforced one active agent per `(company, key)` — `provision()`/`ensure()` did a check-then-insert with no guard. > - Two concurrent server processes (e.g. a `tsx watch` double-boot) could both read "no summarizer exists" and both insert, leaving a company with duplicate built-in agents plus paired orphan pending `hire_agent` approvals. > - That data blemish then became a recurring outage: `findSingleAgent` throws on >1 marked row, and because the throw escaped `reconcileBuiltInAgentsOnStartup`'s sequential loop, **every company after the affected one was silently skipped** on each boot — no auto-provisioning, no default grants — until manual DB surgery. > - This pull request closes the race at the database level and makes reconciliation self-healing and fault-isolated. > - The benefit is that concurrent provisioning can no longer create duplicates, and even pre-existing duplicates are resolved automatically instead of bricking startup reconciliation for unrelated companies. ## Linked Issues or Issue Description - [x] I searched the GitHub PR list (open and recently closed) for similar PRs and confirmed this is not a duplicate. No public GitHub issue exists; describing the bug in-PR (bug-report shape): **What happened** A dev instance booted with two concurrent server processes. Both ran built-in agent provisioning for the same company at the same time, and the check-then-insert in `provision()`/`ensure()` (`server/src/services/built-in-agents.ts`) let both writers see "no summarizer exists" and each create one — the company ended up with two identical Summarizer agents (identical `paperclipBuiltInAgent` markers) plus two paired pending `hire_agent` approvals. From then on, **every** server boot logged: ``` ERROR: startup reconciliation of built-in agents failed Multiple built-in agents found for summarizer (built_in_agent_duplicate_instance) ``` because `findSingleAgent` throws on >1 marked row rather than resolving the duplicate. Worse, `reconcileBuiltInAgentsOnStartup` loops companies sequentially and the throw escaped the loop, so every company *after* the affected one was silently skipped on every boot. **Expected behavior** 1. Concurrent provisioning must not create duplicate built-in agents (there was no DB uniqueness constraint on the marker key per company). 2. Reconciliation should be resilient: if duplicates exist anyway, self-heal (keep the oldest row, terminate the newer dupe, cancel its orphan pending `hire_agent` approval), and never let one bad company abort reconciliation for the rest. **Steps to reproduce** - Race two `provision(companyId, "summarizer")` calls for a company with board approval for new agents enabled (or simulate a double-boot); both insert. - Restart the server → startup reconciliation error fires, companies later in the loop are never reconciled. ## What Changed **Part 1 — stop creating duplicates** - Migration `0192_built_in_agent_unique_marker` adds a **partial unique index** on `(company_id, metadata->'paperclipBuiltInAgent'->>'key')` where the marker exists and `status != 'terminated'`. It first resolves any pre-existing duplicates (keep oldest by `created_at`, terminate newer dupes, cancel their orphan pending `hire_agent` approvals, revoke their API keys) so the index can be created on already-affected instances. - `provision()`/`ensure()` now catch the losing race's `23505` unique violation (walking the driver's wrapped cause chain) and re-resolve to the winning row instead of surfacing the error. **Part 2 — resilient reconciliation** - `findSingleAgent` self-heals: keeps the oldest marked row, terminates the newer duplicates, and cancels each one's orphan pending `hire_agent` approval (idempotent) instead of throwing. - `reconcileBuiltInAgentsOnStartup` isolates per-company failures in both loops so one bad company can't abort reconciliation for the rest; it surfaces a `companyFailures` count in the startup log. - Adds `approvalService.cancel()` for system-initiated cancellation of an orphan approval. ## Verification - `pnpm --filter @paperclipai/db run check:migrations` → numbering + safety checks pass. - `packages/db` migration test (real embedded Postgres) — seeds pre-index duplicate state, runs the migration, asserts dupes resolved + index enforced: **1 passed**. - `server` `built-in-agents.test.ts` — self-heal, concurrent races (plain and board-gated), and startup self-heal-without-aborting-later-companies: **34 passed**. ``` pnpm --filter @paperclipai/db exec vitest run src/built-in-agent-unique-marker-migration.test.ts pnpm --filter @paperclipai/server exec vitest run src/__tests__/built-in-agents.test.ts ``` ## Risks - **Migration safety**: the migration mutates data (terminates duplicate rows, cancels their orphan pending approvals, revokes their API keys) before creating the index. It keeps the oldest row per `(company, key)` and only touches non-terminated marked rows; the destructive step is covered by the migration test and the safety-check baseline. On a clean instance it is a no-op cleanup followed by `CREATE UNIQUE INDEX IF NOT EXISTS`. - Otherwise low risk: the unique index is partial (excludes terminated rows, so re-provisioning after a termination stays possible), and the conflict handling degrades gracefully to re-resolving the existing winner. ## Model Used Claude Opus 4.8 (`claude-opus-4-8`), 1M context window, extended thinking, with tool use.	2026-07-28 11:12:58 -07:00
DottaandPaperclip	c3bd0c5d50	feat(skills): add beta releases for the core Paperclip skill (#10228) ... ## Thinking Path > - Paperclip is the open source control plane people use to organize and operate AI-agent companies. > - Agent behavior depends partly on the bundled Paperclip core skill synchronized into each runtime. > - The existing database and runtime plumbing already supports immutable skill-version snapshots and per-agent version selections, but no product workflow exposed that capability. > - Replacing the live bundled skill globally would make champion adoption risky and difficult to compare across agents. > - This pull request adds an experimental, instance-level beta-skills gate plus a repository release registry, immutable seeded releases, enforcement, and a per-agent release picker. > - The benefit is controlled per-agent evaluation of frozen core-skill releases while the default-off path remains behaviorally unchanged. ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting: `server/`, `ui/`, `packages/db`, and `packages/shared`. ### Problem or motivation Paperclip needs a safe way to evaluate improved versions of its core operating skill without globally replacing the live default. Today the version-snapshot and per-agent pin plumbing exists, but operators cannot use it. A global replacement would make regressions difficult to contain and would prevent controlled comparisons across agents. ### Proposed solution Add a default-off instance experiment that exposes immutable, named core-skill releases. When enabled, operators can pin each agent to a seeded release; when disabled, every agent resolves the live default while saved pins remain intact. Validate pinned writes at the API boundary, gate reads at runtime, and expose the selection in the agent Skills tab. ### Alternatives considered - **Replace the bundled core skill globally:** rejected because it changes every agent at once and provides no rollback/isolation boundary. - **Ship releases as separate skills:** rejected because releases are versions of one core capability, not independently enabled skills. - **Store release snapshots only outside the repository:** rejected because repository provenance and hashes make builds reproducible and reviewable. ### Roadmap alignment This extends the Skills Manager / Skill Studio direction in `ROADMAP.md` by making core-skill versions operable per agent. It does not duplicate another open implementation PR; GitHub searches found no related `enableBetaSkills` change. ### Additional context The feature remains experimental and default off. The V7 champion was selected through a multi-model evaluation process, and the frozen release contents are verified by SHA-256 below. ## What Changed - Added the default-off instance-level `enableBetaSkills` experimental flag. - Added `skills-releases/paperclip/` with the ordered release registry and frozen `v0` plus `v7-roster` snapshots. - Added release metadata to `company_skill_versions` and idempotent release seeding. The migration was planned as `0191`, then renumbered to `0192` because current `master` claimed `0191` before final rebase. - Added read-time gating and write-time validation so disabled instances always resolve the live default and reject pinned-version writes. - Added the per-agent Release picker in the agent Skills tab, including responsive layout and beta-pin state. - Kept `EDITS.md` out of the release registry and PR diff. ### V7 Adoption Evidence - Paid roster: 6 models, 94-case suite. - Result: 553/564 pass-within-2, mean 92.17/94, versus the P2 baseline of 544/564. - Reference model improved 84→91; maximin improved 84→90. - Final report: https://pages.paperclip.ing/skills/optimization/paperclip/pap-14624-p3-final-20260721/ ### Provenance - `v7-roster` is the Phase 1 champion plus additions-only edits E107–E112. Per-edit rationale remains in the evals repository at `source/v7-roster/EDITS.md` and is deliberately excluded from this PR. - `v0` is the `skills/paperclip` tree from commit `ea66ea81`. - Champion selection was accepted on July 21, 2026 via board card `9c304fc2` (PAP-14624 G3). - This delivery mechanism was accepted on July 24, 2026 via plan revision `2367abd2` (PAP-14858). ### QA Evidence - P4 QA matrix comment `b7f40522-4e9b-4a3a-9821-28e86fe1a987`: all 6 acceptance criteria passed. - Automated QA matrix: 166 tests passed with 0 failures, including real filesystem materialization and full SHA-256 assertions. - UI QA exercised the real agent Skills tab at desktop and mobile widths with the experimental flag both on and off. ## Verification - `pnpm check:token-gates` - Focused beta-release matrix: 169 tests passed across shared validators, server services/routes/heartbeat behavior, instance settings UI, and release picker UI. - `pnpm -r typecheck` - `pnpm build` - `pnpm test:run`: server and UI partitions passed; one CLI doctor test inherited temporary AWS credentials from the agent heartbeat and expected no static credentials. The isolated rerun with `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_SESSION_TOKEN` unset passed 8/8. - V7 SHA-256: - `SKILL.md`: `53ab290489684cbf116fdd1406a95f6b6f53c9c36358b1bf8bfeae481e253575` - `references/cases.md`: `3b821f59064a7761091020a14819a8d787131f24029748563d6c0e1be7e6eaec` - `references/workflows.md`: `69747bd6e05f7e3673d1e67b07ff295df1869c05e1fd029804d5fa9177db92cd` - Confirmed 49 changed files, no `pnpm-lock.yaml`, no workflow changes, and no `EDITS.md`. ## Risks - **Migration:** low-to-moderate risk. Three nullable columns and one partial unique index are added idempotently; existing rows remain valid. - **Behavior:** low risk while the flag is off because read-time resolution forces the live default and saved pins are preserved but inactive. - **Frozen content:** release snapshots intentionally diverge from future live skill edits; provenance and hashes make that divergence explicit and reproducible. - **UI:** low risk. The picker only renders for the bundled core skill when the experimental flag is enabled and seeded releases exist. > This extends the existing Skills Manager / Skill Studio direction described in `ROADMAP.md`; it does not duplicate another open implementation PR. The GitHub PR search found no related `enableBetaSkills` change. ## Model Used - OpenAI Codex using `gpt-5.5` with reasoning and terminal/code-execution tools; context-window size is not exposed by this runtime. Earlier implementation commits also record Claude Opus 4.8 assistance where applicable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [ ] I have not referenced internal/instance-local Paperclip issues or links (required governance identifiers are included above; no internal URL is included) - [ ] My branch name describes the change and contains no internal Paperclip ticket id (the approved delivery plan mandated this shared branch name) - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-27 19:45:59 -05:00
DottaandPaperclip	c111ee4cb3	feat(server): add per-user document stars (#9952) ... ## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their work. > - Artifacts and documents are first-class outputs, but users need a personal way to keep important documents easy to find. > - Existing resource memberships already model per-user starred projects and agents with company scoping and activity logging. > - Documents lacked the equivalent membership model, route, and artifact filtering behavior. > - The shared membership contract also needs to remain safe for existing UI project/agent mutation helpers when documents become a recognized resource type. > - This pull request extends the existing resource-membership system with per-user document stars and a starred artifacts view. > - The benefit is a company-scoped, idempotent server foundation for a dedicated starred-documents experience without weakening authorization or artifact filtering semantics. ## Linked Issues or Issue Description ### Problem / Motivation Board users cannot star individual documents, and the company artifacts API cannot return only the current user's starred documents. ### Proposed Solution Add company/user-scoped document memberships, a board-only document star route, document membership data in the shared contract, and a `starred=true` artifacts filter. ### Alternatives Considered A document column was rejected because stars are per-user; a separate star API was rejected because projects and agents already use resource memberships. ### Roadmap Alignment This extends the existing Artifacts & Work Products roadmap area and does not duplicate another open pull request found in the repository search. ## What Changed - Added the `document_memberships` schema and migration with company/user/document uniqueness and starred ordering. - Extended shared resource-membership and artifact-query contracts for documents and `starred=true`. - Added company-scoped document star/unstar service and board-only route behavior with activity logging. - Added starred document artifact filtering, including user-authored documents, document kinds, cursor ordering, and incompatible-kind handling. - Preserved idempotency under concurrent star requests and synchronized UI membership defaults/helpers with the expanded contract. - Added focused shared, route, service, and UI regression coverage. ## Verification - `pnpm exec vitest run packages/shared/src/resource-memberships.test.ts server/src/__tests__/company-artifacts-service.test.ts server/src/__tests__/resource-memberships-routes.test.ts` - `pnpm exec vitest run ui/src/components/SidebarAgents.test.tsx ui/src/components/SidebarProjects.test.tsx ui/src/components/SidebarStarredProjects.test.tsx` - `pnpm --filter @paperclipai/db check:migrations` - `pnpm -r typecheck` - `pnpm test:run` - `pnpm build` ## Risks - The migration adds a new membership table and non-concurrent indexes; migration safety gates pass with the repository's established policy. - The starred artifacts query intentionally returns only documents and relaxes the normal agent-authored/system-kind predicates for documents the current user explicitly starred. - Document membership mutations remain board-user-only; agent callers receive no document-star capability. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex CLI; runtime model ID and context-window size were not exposed to this session. Reasoning, repository tool use, code execution, and test execution were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-27 19:13:35 -05:00
DottaandPaperclip	c481be44e3	fix(task-watchdogs): deduplicate unchanged stopped-state wakes (#10207) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Task watchdogs review issue subtrees when no run or queued wake keeps work live > - Pending human interactions and approvals are valid stopped states that still need one watchdog review > - The existing fingerprint included volatile activity timestamps, so unchanged stopped trees could wake repeatedly after comments, documents, work products, or sibling completions > - This pull request fingerprints only review-material leaf and wait state, persists the reviewed snapshot, and suppresses shrink-only repeats > - The benefit is one review per materially new stopped state without weakening liveness classification or hiding human waits ## Linked Issues or Issue Description ### What happened? Task-watchdog stop fingerprints changed for metadata-only activity and completed siblings, producing duplicate wakes after an unchanged stop had already been reviewed. ### Expected behavior Pending interactions and approvals remain classified as stopped, but a reviewed stopped state only wakes again when waits, non-terminal leaves, status, assignment, or blockers gain material changes. ### Steps to reproduce 1. Review a stopped watched subtree with a pending human wait or multiple non-terminal stopped leaves. 2. Add only comment/document/work-product activity, or complete one stopped sibling without changing the wait set. 3. Observe a duplicate wake from the timestamp-heavy fingerprint. Related public work: Refs #9452 for overlapping task-watchdog service edits and #10043 for related no-op fingerprint suppression. ## What Changed - Added fingerprint v2 over non-terminal material leaves plus subtree-wide pending wait ids, excluding volatile timestamps while retaining them in wake context. - Added nullable observed/reviewed JSONB stop snapshots and shrink-only reviewed-state suppression with legacy exact-fingerprint fallback. - Added pending interaction kinds and approval ids to watchdog wake context, review comments, and comment metadata. - Added classifier and scheduler coverage for waiting-leaf liveness, metadata stability, sibling shrink suppression, material changes, snapshot promotion, legacy rows, and unchanged idempotency keys. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/task-watchdogs-classifier.test.ts src/__tests__/task-watchdogs-scheduler.test.ts` — 2 files, 36 tests passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `git diff --check origin/master...HEAD` — passed. ## Risks - Fingerprint version 2 intentionally re-fingerprints every currently stopped watched tree once after deployment, causing a one-time wake burst before the new reviewed snapshots are established. - Migration `0191_task_watchdog_stop_snapshots.sql` only adds two nullable JSONB columns with no backfill; legacy rows continue exact-fingerprint behavior until a post-deploy review promotes a snapshot. - PR #9452 edits the same service file; whichever lands second may need a trivial rebase. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, exact model ID `gpt-5.6-sol`, high reasoning mode, with repository tool use and code execution. The runtime did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-25 08:34:41 -05:00
DottaandPaperclip	8f08ec5ce6	feat(status-cards): join summary-mentioned issues to watched set (#10205) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Status cards summarize changing company work and watch issues so later changes can produce useful deltas > - A summary can explicitly reference issues that are important to the update even when those issues do not match the card's configured queries > - Previously, those referenced issues were not retained in the watched set, so their later status, assignee, or comment changes could be missed > - The watched snapshot must avoid artificial additions or removals caused only by a summary changing which issues it references > - This pull request resolves issue references when a summary is written, persists them, and joins them to the watched snapshot with stable delta semantics > - The benefit is that status cards continue tracking the exact issues their latest update called out while keeping follow-up updates relevant and non-duplicative ## Linked Issues or Issue Description ### Pre-submission checklist - [x] I have searched existing open and closed issues and this is not a duplicate. - [x] I am on the latest released version of Paperclip (or can reproduce on `master`). - [x] I have confirmed the error originates in Paperclip itself — not in my agent adapter, API provider, or local configuration. ### What happened? When a status-card summary explicitly referenced an issue by identifier or `/issues/<uuid>` URL, that issue was not automatically retained in the card's watched set unless it independently matched a configured query. Later status, assignee, or comment changes to an issue highlighted by the latest update could therefore be omitted. ### Expected behavior References in the latest summary should resolve only within the card's company, appear in dry runs and the watched-issues UI, count and fingerprint like query matches, and enter or leave the watched set without artificial added/removed deltas already represented by the summary change. ### Steps to reproduce 1. Create a status card whose query does not match a second issue in the same company. 2. Write a summary that references the second issue by identifier or issue URL. 3. Inspect the card's watched count or Watched issues tab. 4. Change the referenced issue's status, assignee, or comments and run the next update. 5. Before this change, the referenced issue is absent from the watched snapshot and its later change does not produce the expected delta. ### Paperclip version or commit - Reproduced on `master` before this PR (base commit `762ce5b4ef`). ### Deployment mode - Local dev (`pnpm dev`), built from source. ### Agent adapter(s) involved - Not adapter-specific (core bug). ### Database mode - Embedded Postgres test environment; the schema change uses standard PostgreSQL JSONB. ### Access context - Board (human operator). ## What Changed - Added migration `0191` and schema support for persisted `status_cards.mentioned_issue_ids`. - Resolved summary references by issue identifier or `/issues/<uuid>` URL within the status card's company when summaries are written. - Joined mentioned issues into watched counts and fingerprints so later status, assignee, and comment changes generate normal update deltas. - Suppressed artificial added/removed deltas when the latest summary starts or stops mentioning an issue. - Added `mentionedIssues` to dry-run responses and a “Mentioned in the latest update” group in the Watched issues tab. - Updated the summarizer prompt to explain that referenced issues automatically join the watched set. - Added focused server and UI coverage for reference resolution, snapshot behavior, deltas, API responses, and rendering. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/status-cards.test.ts src/__tests__/status-card-update-engine.test.ts` — 31 tests passed. - `pnpm --filter @paperclipai/ui exec vitest run src/pages/StatusCards/StatusCardTile.test.tsx` — 11 tests passed. - Earlier implementation verification also passed database/shared/server typechecks, UI `tsc -b`, the broader StatusCards UI test set, and embedded-Postgres migration application. ### Visual Verification - Greptile T-Rex ran Playwright browser checks successfully and captured the Status Card drawer Watched tab showing the new “Mentioned in the latest update” grouping: https://app.greptile.com/trex/runs/15796101/artifacts ## Risks - The migration adds a nullable JSONB column and is backward-compatible; existing cards have no mentioned issues until their next summary write. - Reference extraction is company-scoped to prevent cross-company issue association. - Watched counts and future fingerprints change for cards whose latest summaries reference issues; tests cover additions, removals, and suppression of spurious deltas. - This targeted status-card fix does not introduce a new roadmap subsystem or external integration. ## Model Used - Anthropic Claude Fable 5 (Paperclip model alias; exact underlying provider model ID and context window were not recorded in the implementation task metadata), with extended reasoning, tool use, and code execution. - OpenAI Codex coding agent (runtime model identifier and context window not exposed to this task) prepared the PR, rebased the branch, and ran focused verification with terminal tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-24 16:44:56 -05:00
DottaandClaude Fable 5	3a16b91217	feat(status-cards): single-message setup drives query and update prompt (#10202) ... ## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their ongoing work. > - Status cards turn a standing question into recurring, agent-generated summaries on the board. > - The existing setup split intent across a watch prompt and separate update instructions, which made creation and later behavior harder to understand. > - A status card should have one durable source of truth for both deciding what to watch and telling the summarizer what each update must contain. > - This pull request makes the card prompt that source of truth, simplifies creation to one step, and lets operators choose the running agent immediately. > - The benefit is a smaller mental model, fewer configuration modes, and consistent update instructions throughout the card lifecycle. ## Linked Issues or Issue Description Status cards currently require operators to express the same intent in two places: the watch prompt and optional update instructions with append/replace/none modes. This feature simplifies the experimental status-card workflow so a single prompt defines both the watch query and every generated update. The create flow must also support selecting the responsible agent without a second setup step. Related prior status-card work: #10101. ## What Changed - Use the status card's single prompt to compile the watch query and directly instruct every summary update. - Add migration `0190_status_card_single_prompt` to remove `status_cards.instructions_mode` and `status_cards.instructions`. - Add `agentId` to `createStatusCardSchema`, validate company membership, and default new cards to the built-in Summarizer. - Replace the two-step create flow with one prompt-and-agent dialog and extract a shared `SummarizerAgentSelect` for create/settings surfaces. - Remove the extra-instructions settings section, reset incremental history when the prompt changes, and rename the board page to "Status". - Update the bundled `status-card-query` skill and board-operator documentation, then regenerate the skills catalog manifest. ## Verification - Server status-card suites: 29/29 passing. - UI `StatusCards` suites: 22/22 passing. - Skills catalog suite: 20/20 passing. - `tsc -b` passes for server, UI, shared, and database packages. - `pnpm check:migrations` passes. - Light and dark mode screenshots cover the new create dialog and settings tab. ## Risks - Migration `0190` intentionally drops existing separate instruction text. Existing card prompts remain and become the update instructions under the new model; status cards are experimental and feature-flagged. - Prompt edits now reset the incremental summary chain and trigger a full rebuild, which is intentional because the prompt is also the update contract. - Agent selection is company-scoped; invalid agent ids return a validation error rather than creating a misrouted card. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Implementation: Anthropic Claude via the `claude_local` adapter, agent label "Claude Fable 5"; extended reasoning, tool use, and code execution. The exact provider model id and context-window value were not retained in the task metadata. - PR preparation: OpenAI GPT-5.4 through Codex CLI, with reasoning, repository inspection, GitHub CLI, and Paperclip API tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-24 14:06:14 -05:00
LeonSGPandAndrew Aymeloglu	b083b173ea	fix(db): use calendar month retention for backups (#3718) ... ## Thinking Path > - Paperclip includes built-in database backup and retention behavior as part of its operational reliability surface. > - The retention implementation lives in `packages/db/src/backup-lib.ts`. > - Monthly pruning is intended to keep the newest backup per retained calendar month. > - The current cutoff uses a fixed 30-day approximation, which deletes January backups too early when the current month has 31 days. > - This pull request switches the monthly cutoff to calendar-month boundaries instead of a fixed day multiplier. > - The benefit is that monthly retention now matches the documented calendar-month behavior and does not prune valid backups prematurely. ## Linked Issues or Issue Description Fixes #3713 Two other pull requests implemented the same fix and have already been closed as duplicates of this one: - #3798 — same author's later take. Anchors the cutoff to the 1st correctly, but mutates the date in local time and leaves `monthKey` on local time, and unit-tests the helper in isolation rather than end to end. - #4031 — decrements the month without anchoring to the 1st, so partial-month drift and a `setMonth` day-overflow edge case remain. No tests. ## What Changed - Replaced the fixed `30 * 24h` monthly retention cutoff with a calendar-month cutoff anchored to the first day of the earliest retained month. - Added a regression test that freezes `Date.now()` at March 31 and proves the newest January backup is retained when `monthlyMonths=2`. - Kept the rest of the pruning behavior unchanged: daily and weekly tiers still use their existing windows and bucket selection rules. ## Verification - `pnpm --filter @paperclipai/db exec vitest run src/backup-lib.test.ts` - `pnpm --filter @paperclipai/db exec tsc --noEmit` - Note: `pnpm --filter @paperclipai/db typecheck` hits an environment-specific `check:migrations` runtime failure on this host (`Cannot find module ./cjs/index.cjs from ` via Bun), so I used plain `tsc --noEmit` to validate the code changes themselves. ## Risks - Low risk. This only changes the monthly retention cutoff calculation. - The pruning buckets are still selected the same way; the fix only widens the retained month window to align with calendar-month semantics. ## Model Used - OpenAI Codex GPT-5 coding agent with terminal tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --- <sub>Edited by commitperclip during triage: added the **Linked Issues** section (`Fixes #3713`) and the duplicate-PR search line the PR template requires. The duplicate search was performed by the triage pipeline, which grouped this PR with #3798 and #4031 and selected this one as the canonical fix. Everything else is the author's original description.</sub> --------- Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com>	2026-07-24 13:12:43 -05:00
3 people	7e40ed8c43	feat(status-cards): add experimental status card update view (#10101) ... ## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies. > - Operators need a board-level way to monitor a changing slice of company work without repeatedly rebuilding filters or reading raw task threads. > - Existing summaries are useful snapshots, but they do not provide a dedicated query-backed card with refresh policy, change tracking, update history, and per-update cost visibility. > - The capability needs to be safe to evaluate before it becomes part of the default product surface. > - This pull request adds end-to-end experimental Status Cards, from schema and query compilation through update orchestration and operator UI. > - The entire feature is gated behind the `enableStatusCards` experimental toggle, including its route and sidebar entry. > - The benefit is a governed, inspectable way to keep focused operational rollups current while preserving explicit controls over refresh frequency and spend. ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting (`packages/db`, `packages/shared`, `server/`, `ui/`, and bundled skills/docs). ### Problem or motivation Operators cannot currently define a reusable natural-language view of company work, compile it into an inspectable query, and keep its summary current as matching issues change. Rebuilding filters and rereading task threads makes board-level monitoring repetitive and hides the relationship between source changes, refresh cost, and the resulting summary. ### Proposed solution Add experimental Status Cards that compile operator intent into a query, summarize matched work, record each update, expose manual/interval/reactive refresh policies and costs, and preserve the last good result across stale, updating, paused, and error states. The capability is off by default and fully gated behind `enableStatusCards`, including its route and navigation entry. ### Alternatives considered - Extend existing one-off summaries: rejected because status cards require persistent query provenance, refresh policy, update history, and card-specific cost controls. - Add a dashboard-only filter widget: rejected because it would not provide governed background refresh, an update ledger, or an inspectable compile pipeline. - Ship the surface by default: rejected in favor of an experimental toggle while behavior and operator value are evaluated. ### Roadmap alignment This advances Paperclip’s board-level execution visibility and output-first product goals. `ROADMAP.md` was checked and no duplicate status-card initiative was found. ### Additional context No related open PR was found in the public GitHub search for status cards. The PR-only design wireframes were removed from the repository after review; the published prototype remains external to the production source tree. ## What Changed - Added company-scoped status-card schema, CRUD APIs, compile provenance, update ledger, shared contracts, validators, and OpenAPI coverage. - Added the text-to-query compile pipeline, bundled `status-card-query` agent skill, query versioning, and authorized write-back flow. - Added the experimental board, create flow, lifecycle tiles, detail/settings/debug drawers, archived view, routing, navigation, and instance setting. - Added a change-gated update engine with manual, interval, and reactive refresh policies, trigger selection, active hours, and daily token caps. - Added per-update token/cost recording, today and lifetime rollups, and policy-derived cost previews. - Added operator documentation and agent-authoring hardening for compile and update behavior. - Added PR-prep integration coverage for settings/startup wiring and replaced raw UI values with design-system tokens. - Removed the PR-only `design/pap-15023-status-cards` wireframe artifacts so the repository contains only production feature assets. ## Verification - `pnpm -r typecheck` — passes on the PR head; includes `ui` `tsc -b` passing. The UI compile gate was also independently recorded as passing at `6d7f3cf96b` on July 23, 2026. - `pnpm build` — passes. - `pnpm check:token-gates` — passes with all three gates clean. - `pnpm test:run` — 2,880 tests passed and 1 skipped; the sole failure was an unrelated 10-second `afterAll` database-cleanup timeout in `execution-workspaces-service.test.ts`. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/execution-workspaces-service.test.ts` — passes on immediate focused rerun (25/25). - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/instance-settings-service.test.ts src/__tests__/server-startup-feedback-export.test.ts` — passes (31/31). - `pnpm --filter @paperclipai/ui exec vitest run src/pages/StatusCards/StatusCardSettingsForm.test.tsx src/pages/StatusCards/StatusCardTile.test.tsx src/pages/StatusCards/format.test.ts src/lib/status-card-state.test.ts` — passes (26/26). - Recorded pre-PR QA: compile-pipeline e2e PASS; full lifecycle and cost QA PASS; security re-review PASS after write-back hardening; UX approved. - `pnpm exec vitest run packages/db/src/status-card-migrations.test.ts` — passes; reapplies migrations `0185`–`0189` against an already-migrated embedded Postgres database. - `pnpm --filter /db check:migrations` — passes migration numbering and safety checks. - `pnpm --filter /db typecheck` — passes. - Merged current `origin/master` on July 24, 2026 with no conflicts; migrations `0185`–`0189` remain unclaimed on master. ## Risks - The feature introduces five database migrations and a new background update path; all new DDL is repeat-safe after partial application, migration numbering/safety checks pass, and update execution is company-scoped and change-gated. - Natural-language compilation can produce invalid or overly broad queries; compile provenance, query validation, debug visibility, and version history make failures inspectable and recoverable. - Reactive or interval refresh could increase spend; active hours, max refresh frequency, daily token caps, per-update cost records, and budget-paused states bound and expose that risk. - The branch name contains an internal execution identifier because it is a fixed handoff branch; it was intentionally not renamed or rebased per the release handoff instructions. - Overall rollout risk is limited because the route, navigation, services, and UI are disabled by default behind `enableStatusCards`. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using GPT-5.5 with reasoning, repository tool use, shell execution, GitHub CLI, and test/build execution. The runtime did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change; the fixed execution-workspace identifier is documented as an authorized handoff exception - [x] I have run tests locally and they pass, with the one cleanup timeout passing on focused rerun - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-24 12:26:43 -05:00
DottaandPaperclip	148a5b11f5	Route blocked transitions to explicit unblock owners (#10112) ... ## Thinking Path > - Paperclip is the open source control plane people use to coordinate AI-agent companies. > - Issue status transitions determine whether work keeps moving or silently stalls. > - A blocked issue previously could rely on prose alone, leaving the intended unblock owner unstructured and unnotified. > - Existing blocker-attention classification could identify stalled chains, but the signal was not delivered to the board attention feed. > - Blocked transitions also need rollout-safe deduplication so upgrades do not notify for historical issues and repeated processing does not create notification storms. > - This pull request adds structured unblock descriptors, prospective transition timestamps, owner delivery, and board attention routing with focused authorization controls. > - The benefit is that newly blocked work has an explicit, routable next action without weakening company boundaries or allowing agents to inject arbitrary human attention items. ## Linked Issues or Issue Description Related documentation PR: #10094. ### Subsystem affected Cross-cutting: `server/`, `packages/db`, and `packages/shared`. ### Problem or motivation An issue can enter `blocked` without a machine-readable unblock path. Prose-only ownership does not reliably wake the responsible agent or surface human-owned work, while the existing `blockerAttention` classifier is not delivered to an operator-facing attention feed. ### Proposed solution Require new transitions into `blocked` to have unresolved blockers, a pending interaction/approval, or a structured `{ owner, action }` descriptor. Notify an allowed owner once per prospective transition, route human-owned cases to board attention, and leave pre-rollout blocked issues untouched. ### Alternatives considered - Keep prose-only blockers: rejected because ownership remains unroutable. - Backfill all historical blocked issues: rejected because upgrades would create notification storms. - Let agents target arbitrary users or the board: rejected after security review because it creates an attention-injection channel. ### Roadmap alignment Aligns with `ROADMAP.md` → “Enforced Outcomes (watchdogs, recovery actions, review gates)” by making blocked work carry an explicit continuation path. ### Additional context The implementation is prospective-only and deduplicated per blocked transition. Agent-authored descriptors are limited to the acting agent; board actors retain human-owner routing. ## What Changed - Added persisted unblock descriptors and prospective blocked-transition delivery timestamps with an idempotent migration. - Added shared types and validation for board, user, and agent unblock owners. - Enforced valid blocked transitions and same-company owner validation in the issue update route. - Restricted agent-authored descriptors to the acting agent itself, preventing board/user attention injection by compromised agents. - Added one-per-transition agent wake delivery and prospective-only rollout gating. - Routed human-owned blocker attention into the board attention feed. - Added focused tests for validation, prospective delivery, flap deduplication, attention routing, route authorization, and stop-relay compatibility. ## Verification - `pnpm -r typecheck` - `pnpm exec vitest run server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts server/src/__tests__/routable-blocked.test.ts server/src/__tests__/attention-service.test.ts packages/shared/src/validators/issue.test.ts` - `AWS_ACCESS_KEY_ID= AWS_SECRET_ACCESS_KEY= pnpm test:run` - `pnpm build` - `pnpm --filter @paperclipai/db check:migrations` ## Risks - Behavioral shift: new `blocked` transitions without a real blocker, pending governed action, or structured descriptor now return `422`. - Notification abuse is constrained by same-company validation, agent self-only routing, prospective rollout gating, and transition-scoped deduplication. - Migration risk is low: columns are additive, nullable, and use `IF NOT EXISTS`; historical blocked issues are not backfilled or notified. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex CLI with GPT-5.4, reasoning-enabled tool use and code execution. The runtime did not expose a context-window value. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-23 15:49:28 -05:00
3 people	b57aa9950c	fix(test): stop flaky server-suite afterAll hook timeouts (#10024) ... ## Thinking Path > - Paperclip is an open-source AI agent management platform; its test suite spans a `server` package that mounts real embedded Postgres databases in `beforeAll`/`afterAll` hooks > - The `server` package CI shard runs all ~93 suites serially (`maxWorkers=1`) on a loaded CI host; each suite boots and tears down its own embedded Postgres in hook callbacks > - vitest's default `hookTimeout` is 10 seconds; under load, graceful embedded-Postgres shutdown occasionally crosses that threshold > - This produces intermittent `Error: Hook timed out in 10000ms` failures in `afterAll` hooks — not test assertion failures — and the suites pass on re-run, making them textbook flaky tests > - Inspecting `embedded-postgres@18.1.0-beta.16` shows that `stop()` takes no argument (no fast-shutdown mode), SIGINTs postgres (already PostgreSQL "fast shutdown"), and resolves only on the child's `exit` event with no internal time bound > - Two targeted fixes: (1) raise `hookTimeout` and `teardownTimeout` to 30 s in `server/vitest.config.ts` — one config change that eliminates the flake for all ~93 suites at once; (2) wrap `stop()` in a 5 s bounded `Promise.race` in the test helper so a slow shutdown can never hang the hook regardless of OS scheduling variance > - This PR changes only test-infra and test-config; no production-code behavior changes ## Linked Issues or Issue Description No public GitHub issue exists for this flake. Inline bug description (bug report template): **What happened?** The `General tests (server (N/3))` CI shards intermittently fail with `Error: Hook timed out in 10000ms` in `afterAll` hooks and pass on re-run. Every test assertion passes; only the teardown hook exceeds vitest's default timeout. **Expected behavior** CI passes reliably. Teardown timeouts should not be a source of flake. **Steps to reproduce** Run the server test suite repeatedly on a loaded host or in CI with `maxWorkers=1` — the shard occasionally crosses 10 s in `afterAll` during embedded-Postgres shutdown. **Paperclip version** `master`, any build that includes `server/vitest.config.ts` without an explicit `hookTimeout`. **Deployment mode** Self-hosted (CI). ## What Changed - **`server/vitest.config.ts`** — added `hookTimeout: 30000` and `teardownTimeout: 30000`. Removes flake across all ~93 server suites at once. 30 s gives generous headroom over observed worst-case teardown while still catching a genuinely hung hook. - **`packages/db/src/test-embedded-postgres.ts`** — added `stopEmbeddedPostgresBounded()`, a 5 s `Promise.race` wrapper around `stop()`. Applied at all three call sites inside `cleanup()`. Data dir is still removed unconditionally; errors are still swallowed; the null-instance guard is preserved. Existing behavior unchanged except the shutdown can no longer block indefinitely. ## Verification - `tsc --noEmit` clean on `packages/db` (built against worktree-local `shared`) - `packages/db` `client.test.ts` passes 14/14 — boots embedded Postgres and exercises the bounded teardown via `cleanup()` in `afterEach` - Standalone bounded-race semantics verified: hang resolves at the 5 s bound; late or immediate `stop()` rejection swallowed; no unhandled rejection; null-instance path safe - CI: all 3 server shards + split-verify lane (Async-Verification Gate) expected green after this PR ```bash # Reproduce the teardown test locally: cd packages/db && npx vitest run src/client.test.ts # Type-check packages/db: npx tsc --noEmit -p packages/db/tsconfig.json ``` ## Risks Low risk. No product-code changes — test-infra and test-config only. The vitest timeout increase is additive (raises the ceiling; never lowers it). The bounded race wrapper preserves prior teardown behavior exactly: data dir always removed, errors always swallowed, stop is still attempted. A worst-case outcome is that a genuinely hung `stop()` now surfaces as a test timeout at 30 s instead of 10 s — still caught, just later. ## Model Used - **Provider:** Anthropic - **Model:** Claude Sonnet 4.6 (`claude-sonnet-4-6`) - **Context window:** 200 k tokens - **Capabilities:** tool use, code execution, extended reasoning - **Mode:** Paperclip agent heartbeat (autonomous execution with human board oversight) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-22 11:51:30 -07:00
DottaandPaperclip	cac3c0fa1a	feat(connections): add runtime subjects and grants (#9982) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Connections is the subsystem that lets operators connect external apps and govern which subjects may use those credentials > - #9958 established the v3 schema foundation and #9981 adds the AppDefinition catalog layer > - The runtime still needs subject-aware authorization state, scoped key handling, and API/OpenAPI routes so connected apps can actually be granted and used safely > - This pull request adds the runtime grants/authorization behavior on top of the catalog branch, while keeping unrelated dependency and workflow sync commits out of the stack > - The benefit is a reviewable runtime layer that can land after the catalog PR, then unblock the wizard and orchestrator cutover work ## Linked Issues or Issue Description Refs #9958 and #9981. Refs #9981. No public GitHub issue exists for this branch. This is the runtime layer for the Connections v3 stack and is rebased onto `master` after #9981 landed. ## What Changed - Adds the connection user authorization state migration and schema wiring. - Adds shared runtime subject/grant types and validators. - Adds runtime grant and scoped key behavior in the tool-access service. - Adds runtime route coverage and registers the routes in OpenAPI. - Replays only the Connections runtime commits on top of the catalog branch, dropping unrelated sync/dependency history from the prior closed runtime PR. ## Verification - `pnpm run preflight:workspace-links` - `pnpm exec vitest run packages/shared/src/validators/tool-access.test.ts server/src/__tests__/tool-access-service.test.ts` ## Risks - Medium: runtime grant enforcement is security-sensitive and must fail closed for unknown key scopes. - Migration ordering depends on the schema and catalog layers already merged through #9958 and #9981. - This PR is rebased and retargeted to `master` with runtime-only commits. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 Codex coding agent with repository tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-21 16:17:02 -05:00
DottaandPaperclip	7e00f67138	feat(connections): add v3 schema core (#9958) ... ## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their governed access to external systems. > - Connected Apps build on the existing Apps and MCP gateway substrate so companies can configure reusable, auditable integrations. > - The current connection record does not yet have a stable public address, explicit ownership/auth method fields, or subject-specific credential grants. > - Without that schema core, later OAuth, per-user authorization, token brokering, triggers, and connector-service phases cannot enforce tenant and subject boundaries consistently. > - This pull request adds the forward-compatible Connections v3 schema core while preserving the existing connection lifecycle and directly migrating the remote MCP transport name. > - The benefit is a company-scoped, least-privilege foundation for one-click integrations without bypassing Paperclip secrets, profiles, rules, or audit controls. ## Linked Issues or Issue Description No matching public issue was found. **Problem** Paperclip's current app connections need a durable identity and authorization substrate before Connected Apps can safely support multiple setup methods, per-user credentials, provider tenants, and managed connector services. The existing schema only models a single connection-level credential set and uses legacy transport terminology. **Proposed solution** Add a stable company-scoped connection UID, explicit ownership/auth/transport fields, a subject-aware `connection_grants` table, and multi-key credential annotations. Backfill existing connections and workspace grants in a reversible migration, then update shared/server/UI contracts to the new `mcp_remote` transport name. **Related work** - Related foundation: #9534 - Roadmap: Connected Apps (one-click integrations) ## What Changed - Added company-scoped connection `uid`, `ownership`, `authKind`, and canonical transport fields across database, shared contracts, validators, services, and UI fixtures. - Added `connection_grants` with workspace/user subject rules, provider tenant metadata, credential secret refs, revocation state, company scoping, and uniqueness constraints. - Added migration `0182_connections_v3_schema_core` to backfill stable UIDs, rename `remote_http` to `mcp_remote`, infer auth kinds, create default workspace grants, and support rollback coverage. - Added multi-key credential annotations and updated gateway/access services without changing the existing lifecycle behavior. - Updated the connection glossary, connector playbook, and security threat model for the new identity, grant, and relay boundaries. - Added explicit test UIDs to direct database fixtures so the new non-null invariant is exercised across affected server suites. ## Verification - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/db typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts server/src/__tests__/tool-gateway-service.test.ts server/src/__tests__/tool-gateway.test.ts server/src/__tests__/heartbeat-runtime-skills.test.ts server/src/__tests__/tool-oauth-legacy-backfill.test.ts server/src/__tests__/tool-access-policy-service.test.ts server/src/__tests__/heartbeat-runtime-mcp-servers.test.ts packages/db/src/connections-v3-schema-core-migration.test.ts packages/shared/src/validators/tool-access.test.ts --config vitest.config.ts` — 9 files, 218 tests passed. - Latest-head GitHub Actions: build, typecheck, general/serialized suites, backup/worktree restore coverage, both e2e shards, canary, policy, and security scans pass. - Greptile: 5/5 with zero unresolved threads. - `pnpm check:token-gates` remains red only on five pre-existing `#9627` color literals outside this change. ## Risks - **Migration risk:** UID backfill and default-grant creation touch every existing connection. The migration uses company-scoped uniqueness, deterministic legacy UIDs with ID suffixes, and seeded up/rollback coverage. - **Authorization risk:** Grant rows carry credential references. Constraints enforce workspace-vs-user subject shape, company/connection lookup indexes, one default grant per connection, and one user grant per connection/subject. Security review is requested specifically for this design. - **Compatibility risk:** `remote_http` is renamed directly to `mcp_remote`; all repository call sites and fixtures are updated in the same change. - **Future-phase risk:** Subject-bound token issuance, triggers, and connector-service relay verification remain fail-closed requirements documented for later phases; this PR does not expose those capabilities. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex CLI coding agent. The runtime did not expose an exact underlying model ID or context-window size; capabilities used include repository inspection, code editing, shell execution, test execution, Git/GitHub CLI operations, and structured reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-21 15:16:26 -05:00
DottaandPaperclip	a090c09ee5	feat: add decision training snapshot foundation (#9702) ... ## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their work > - Human approvals, issue interactions, and execution decisions already capture high-value decision moments > - Those moments are currently transient and cannot be reused as stable evaluation or training examples > - Reusable examples need a server-owned, immutable snapshot so later comments or runs cannot leak into the recorded state > - Human notes need to remain editable and auditable without changing the captured state > - This pull request adds the database model, snapshot capture service, API, export format, and attention-feed enrichment for decision training > - The benefit is a durable, inspectable foundation for evaluating whether agents can reproduce good human decisions from only the context available at decision time ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting (`server/`, `packages/db`, and `packages/shared`). ### Problem or motivation Paperclip has no durable dataset for converting human decisions into evaluation-ready examples. Teams need to capture pending or resolved decisions with the exact issue context, comments, runs, and repository evidence available at a cutoff, while preventing future context from leaking into the example. ### Proposed solution Store immutable, schema-versioned snapshots anchored to durable interaction, approval, or execution-decision records; keep notes separately editable with history; expose human-only CRUD, list, and JSONL export APIs. ### Alternatives considered Client-generated snapshots were rejected because they duplicate cutoff logic and cannot reliably enforce no-leakage boundaries. Automatic outcome backfill was deferred so captured examples remain faithful to what was known at capture time. ### Roadmap alignment Supports the roadmap direction of turning completed work and decision patterns into reusable organizational knowledge. ### Additional context The implementation records explicit commit-resolution confidence (`exact`, `nearest_run`, `workspace`, or `none`) so downstream evaluation can distinguish evidence quality. ## What Changed - Added the `decision_training_examples` schema and idempotent migration with company, issue, and source/author indexes. - Added shared types for decision-training records, notes history, and versioned snapshots. - Added a single server-side snapshot capture path with inclusive comment cutoffs, pre-cutoff run capture, durable decision payloads, and explicit commit-resolution confidence. - Added create, list, detail, notes-only update, delete, and JSONL export routes with human-only write authorization and activity logging that skips no-op note submissions. - Added per-user `trainingExampleId` enrichment to attention items. - Added focused embedded-Postgres tests for cutoff boundaries, post-cutoff leakage, immutable snapshots, human-only writes, duplicate prevention, notes history, attention enrichment, and export shape. - Updated UI test and Storybook attention-item factories for the new required `trainingExampleId` contract. ## Verification - `pnpm exec vitest run server/src/__tests__/decision-training.test.ts` — 10 tests passed. - `pnpm --filter @paperclipai/db typecheck` — passed, including migration numbering and safety checks. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. ## Risks - The migration adds a new table and indexes only; it does not rewrite existing rows or install resolve-time hooks. - Snapshot JSON can grow with long comment threads and run histories; v1 intentionally favors complete, inspectable examples over aggressive truncation. - Commit SHA resolution is evidence-based and records `exact`, `nearest_run`, or `none` so downstream consumers can account for confidence. - The API is additive, but future UI work must continue to treat the snapshot as immutable and use notes-only updates. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using `gpt-5.3-codex`, with repository tool use, terminal execution, and code-editing capabilities; context-window size is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-17 12:17:34 -05:00
3 people	1f1f545238	feat: add built-in summarizer and summary slots (#9713) ... ## Thinking Path > - Paperclip is the open source control plane people use to organize, govern, and understand AI-agent work > - Operators need concise, current status views across projects and execution workspaces without manually reading every issue and run > - Paperclip already has auditable issues, documents, built-in agents, routines, and live run events, but no first-class summary-slot workflow connecting those systems > - A built-in Summarizer can generate status prose through ordinary governed tasks while summary slots provide stable, revisioned destinations for that output > - The UI needs to show current summaries, generation progress, failures, revisions, and streaming draft status in the places operators already work > - This pull request adds the end-to-end summary-slot data, API, agent, orchestration, and UI surfaces behind an experimental setting > - The benefit is decision-oriented status context that remains company-scoped, auditable, retryable, and inexpensive by default ## Linked Issues or Issue Description No public GitHub issue exists for this feature. **Problem** Operators currently have to reconstruct project and workspace status by reading many issues, runs, and comments. This makes it hard to identify decisions, review queues, recent work, and the next event worth watching. **Proposed capability** Add an experimental summary system with revisioned summary slots for projects and workspaces, a paused-by-default built-in Summarizer agent, governed generation tasks, live draft status, and reusable UI cards. **Expected behavior** - Summary data remains company-scoped and revisions remain auditable. - Generation runs through normal issue/agent orchestration and deduplicates active requests. - Only the linked built-in Summarizer generation task can author a slot revision. - Operators can generate, retry, inspect revisions, and follow draft progress from project and workspace views. - The feature remains opt-in and background generation remains paused by default. ## What Changed - Added summary-slot schema, idempotent migrations, shared contracts, validators, API paths, and service tests. - Added company-scoped summary-slot routes for reading revisions, requesting generation, and guarded Summarizer writes with activity logging. - Added terminal generation finalization, failure reasons, assignment wakeups, and orchestration integration. - Added the paused-by-default built-in Summarizer bundle, low-cost runtime defaults, status-summarization skill, and stale-summary routine. - Added summary cards, revision selection, retry/configuration states, live draft streaming, transcript chunk handling, and project/workspace integrations. - Updated Claude local parsing for streamed status output and expanded server, adapter, shared, database, catalog, and UI coverage. ## Verification - `pnpm -r typecheck` - `pnpm exec vitest run packages/db/src/summary-slots-schema.test.ts packages/shared/src/summary-slot.test.ts server/src/__tests__/summary-slot-routes.test.ts server/src/__tests__/summary-slots.test.ts server/src/__tests__/built-in-agents.test.ts ui/src/components/SummarySlotCard.test.tsx ui/src/components/SummarySlotCard.status.test.tsx ui/src/components/useSummaryDraftStream.test.tsx ui/src/lib/summary-draft-stream.test.ts ui/src/lib/run-log-chunks.test.ts ui/src/context/LiveUpdatesProvider.hook.test.tsx` — 113 tests passed - `pnpm test:run` — server and UI suites passed; one CLI AWS doctor test was affected by inherited `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY`, and passed when those host credentials were removed - `pnpm exec vitest run cli/src/__tests__/secrets.test.ts` with inherited AWS credential variables removed — 8 tests passed - `pnpm build` - `pnpm check:token-gates` currently reports nine `#9627` comment references introduced by current `master`; none are in this PR diff ## Risks - Database risk is limited by incrementally ordered, idempotent migrations and migration safety checks. - Summary generation creates normal issues/runs, so misconfiguration can produce failed slots; the UI exposes retryable failure reasons and agent configuration entry points. - Streaming draft parsing depends on the documented `STATUS:` protocol; final persisted revisions remain the source of truth. - The feature is experimental, opt-in, and its built-in routine is paused with no background token spend by default. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-5.3 Codex with reasoning, repository tool use, code execution, GitHub CLI, and Paperclip control-plane integration. Earlier branch commits also record Claude model co-authorship where applicable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-17 11:03:07 -05:00
DottaandPaperclip	b07b2994cc	feat: stamp responsible users on activity logs (#9731) ... ## Thinking Path > - Paperclip is the control plane people use to manage AI-agent companies and their work > - The activity log is the generic audit spine for mutations across the control plane > - Activity rows identify agents and runs, but they do not persist the responsible human upstream > - Relying only on run joins loses attribution after run pruning and misses agent API-key actions outside a run > - This pull request resolves responsible-user attribution when each activity row is written and stores it directly > - The benefit is durable, queryable agent audit feeds without rewriting historical provenance ## Linked Issues or Issue Description ### Problem or motivation Agent activity records do not persist the responsible user, so attribution can disappear when runs are pruned and no-run API-key mutations cannot be attributed correctly. ### Proposed solution Resolve attribution for each new activity row from the run, related issue, active agent API key, or company default, in that order, and persist the result directly. ### Alternatives considered Read-time joins alone were rejected because pruned runs lose durable attribution and out-of-run agent-key actions have no run to join. Historical backfill was rejected because it would invent provenance. ### Roadmap alignment This strengthens the durable audit-trail direction described in `ROADMAP.md` without adding a new product surface. ## What Changed - Added nullable `activity_log.responsible_user_id` plus company/agent/time and company/responsible-user/time indexes. - Added an idempotent forward-only migration with no historical backfill. - Added centralized write-time resolution: heartbeat run → issue attribution → active agent API key → company default. - Propagated authenticated API-key IDs through existing request-backed `logActivity()` calls. - Added unit coverage for every fallback and an embedded-Postgres assertion for the no-run API-key stamping path. ## Verification - `pnpm --filter @paperclipai/db typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/activity-log-responsible-user.test.ts src/__tests__/authz-company-access.test.ts` - Initial focused verification: 25 tests passed; migration safety passed. - Follow-up regression verification: 54 focused attribution/company-skill/environment/issue-tree/tool-gateway tests passed; server typecheck passed. - GitHub: full build, typecheck, server shards, serialized suites, e2e, security, and policy checks passed. ## Risks - Adding two indexes to an existing large table can hold a write lock while the transactional migration runs. The migration safety suppressions document why `CONCURRENTLY` is unavailable under the current Drizzle migration runner. - Historical rows remain nullable by design; this avoids inventing provenance and keeps the migration forward-only. - API-key attribution requires request-backed activity call sites to pass the authenticated key ID; this PR mechanically updates the existing actor-based activity calls and covers the no-run path with integration testing. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, `gpt-5.4`; context-window size was not exposed by the runtime. Medium reasoning with repository editing, terminal execution, and test execution capabilities was used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-16 20:54:55 -05:00
DottaandPaperclip	53f09cb818	fix: prevent duplicate task creation and recovery loops (#9648) ... ## Thinking Path > - Paperclip is the open source control plane people use to coordinate AI-agent companies > - Agents, routines, productivity review, and recovery services can all create or re-trigger work > - Repeated heartbeats or catch-up cycles can produce duplicate tasks or repeat recovery actions before prior work is visible > - The base duplicate-create and routine catch-up coalescing work has now landed on `master` via related PRs while this PR was being prepared > - This pull request carries the remaining hardening: bounded idempotency retention, recovery cooldown/throttle fixes, productivity-review query batching and ordering fixes, and regression coverage > - The benefit is fewer duplicate tasks, safer retries, and enough provenance to diagnose any future recurrence ## Linked Issues or Issue Description Agents can retry issue creation after ambiguous responses or independently recreate the same child title, while recovery and short-interval routine catch-up paths can repeat before prior work settles. This can produce visible duplicate tasks and makes the originating heartbeat difficult to identify. Related work: Refs #8356 for caller-supplied issue-create idempotency and Refs #9224 for plugin-scoped issue-create idempotency. Prior related PR: #6936. The base issue-create deduplication and routine catch-up coalescing pieces have since landed on `master` via #9650 and #9649; this PR remains as the follow-up hardening stack on top of those changes. ## What Changed - Add 7-day retention for issue-create idempotency claims with indexed, batch-limited cleanup so the claim table does not grow forever. - Preserve recovery cooldown intent after terminal recovery actions are closed, and throttle repeated source-scoped recovery work. - Batch productivity-review source-activity checks to avoid repeated per-source queries while keeping the no-action suppression behavior. - Order productivity-review no-action streak windows by review creation time, matching the window semantics even when completion timestamps are out of order. - Preserve generated issue IDs in route mocks used by backlog/assignment contract tests. - Document PR-gardening task deduplication expectations in the company skill. - Add focused regression tests for idempotency retention, liveness recovery cooldowns, and productivity-review batching/suppression/ordering behavior. ## Verification - `pnpm --filter @paperclipai/server typecheck` — passed on latest head. - `pnpm exec vitest run --project @paperclipai/server server/src/__tests__/productivity-review-service.test.ts server/src/__tests__/issue-create-deduplication-routes.test.ts --reporter verbose` — 2 files, 23 tests passed on latest head. - `pnpm --filter @paperclipai/adapter-utils build && pnpm exec vitest run --project @paperclipai/adapter-utils --reporter dot` — 30 files passed, 476 tests passed, 8 skipped. - `pnpm build` — passed on latest head. - `pnpm -r typecheck` — passed on the rebased head before the final productivity-review ordering commit; the latest touched server code is covered by the server typecheck above. - `pnpm check:token-gates` — passed. - `pnpm test:run` — progressed through server, UI, CLI, shared, skills-catalog, and DB sections, then exposed an adapter-utils compiled-test fingerprint mismatch before rebuilding adapter-utils; the adapter-utils project passed after rebuild, and the GitHub split PR checks passed on the pushed head. ## Risks - Caller-supplied idempotency replay is now bounded to 7 days; reusing an old key after retention can create new work, which matches retry-oriented idempotency semantics. - Recovery and productivity-review timing changes may suppress redundant follow-up work; focused tests cover the intended boundaries. - Advisory locking and idempotency cleanup rely on PostgreSQL-compatible transaction semantics already used by the production data layer. - The migration extends the private claim table indexes without rewriting existing issue rows. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex based on GPT-5, with reasoning, repository editing, shell/tool execution, and test execution. Exact model ID and context-window size are not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-16 17:00:53 -05:00
DottaandPaperclip	52aea90263	feat: organize skills with nested folders and My Skills (#9633) ... ## Thinking Path > - Paperclip is the open source control plane people use to organize and govern AI-agent companies > - Company skills are durable resources that users browse, import, assign, and maintain over time > - A flat skill list plus tags does not provide a stable location or hierarchy for personal, company, project-imported, and bundled skills > - Folder paths need to be canonical, company-scoped, safe to move, and preserved across re-imports without changing skill IDs > - The `/skills` UI also needs traversal, breadcrumbs, move/create flows, and a dedicated My Skills namespace that work on desktop and mobile > - This pull request adds the folder data model and APIs, reserved-root lifecycle, project import behavior, and the folder-first skills experience > - The benefit is a predictable filesystem-like organization model while tags remain available for cross-cutting classification ## Linked Issues or Issue Description Refs #9619 — the reviewed folder foundation was intentionally closed and folded into this combined feature PR. Refs #9026 — earlier flat-folder attempt superseded by this integrated implementation. Refs #3281 — related skill organization proposal; this PR uses canonical persisted folders rather than deriving groups from skill keys, and does not add hidden-skill behavior. **Feature request** - **Problem:** Skills currently lack a canonical hierarchical location, making personal skills, project imports, bundled skills, and company-authored skills difficult to traverse and manage at scale. - **Proposed behavior:** Add nested company-scoped folders with stable paths, reserved My/Projects/Bundled roots, subtree queries, safe move/create operations, and a folder-first `/skills` library UI. - **Import behavior:** New project scans file skills under `projects/<project-slug>`; later imports update content without overriding a user-selected folder. - **Alternatives considered:** Tags alone remain useful for cross-cutting classification, but they do not provide canonical location, nesting, reserved namespaces, or stable import placement. - **Roadmap alignment:** Extends the completed Skills Manager and Scheduled Routines capabilities without duplicating an active roadmap item. ## What Changed - Adds `folders` persistence for routine and skill folders, nested canonical paths, parent/slug/system-key fields, migration backfills, and reapply-safe migrations `0174`–`0175` after current master migrations. - Adds company-scoped folder CRUD, cycle/depth/namespace validation, reserved My/Projects/Bundled lifecycle, item moves, subtree filtering, and folder paths on skill results. - Preserves project-import placement: first import files into the project folder, while re-import keeps user-owned placement and stable skill IDs. - Adds the `/skills` folder tree rail, tags facet, breadcrumbs, subfolder browser, move/new-folder dialog, canonical detail location, inline tag editing, and folder-aware Studio creation. - Keeps bundled skills read-only even when their source metadata is incomplete by detecting the reserved Bundled folder and hiding selection/move actions. - Extends routine folder UI and OpenAPI coverage, and adds regression tests across migrations, services, routes, tree helpers, pages, and Studio creation. ## Verification - `pnpm exec vitest run packages/db/src/nested-skill-folders-migration.test.ts server/src/__tests__/folders-routes.test.ts server/src/__tests__/folders-service.test.ts server/src/__tests__/company-skills-service.test.ts server/src/__tests__/routines-service.test.ts ui/src/components/folders/FolderControls.test.tsx ui/src/components/folders/SkillFolderTree.test.tsx ui/src/components/folders/skill-folder-tree.test.ts ui/src/pages/CompanySkills.test.tsx ui/src/pages/Routines.test.tsx ui/src/pages/SkillStudio.test.tsx ui/src/lib/company-skill-routes.test.ts ui/src/lib/skill-create.test.ts` — 13 files, 192 tests passed. - `pnpm exec vitest run ui/src/pages/CompanySkills.test.tsx ui/src/components/folders/SkillFolderTree.test.tsx` — 2 files, 20 tests passed after preserving the existing PR's bundled-skill fixes. - `pnpm -r typecheck` — passed for all workspace packages. - `pnpm test:run` — passed in an isolated CI-like environment with inherited Paperclip runtime identity and static AWS credential variables removed. - `pnpm build` — production build passed for all workspace packages. - Greptile iteration 2 — 5/5 confidence with zero unresolved threads on commit `ff2d67aa71`. - Latest-head GitHub checks — all success, neutral, or skipped; PR is mergeable with a clean merge state. - `pnpm check:token-gates` — reports nine existing `#9627` comment false positives already present on `master`; this PR introduces no new token violation. ## Risks - **Migration/backfill:** `0174` creates the foundation and `0175` adds nested/reserved semantics. Both are ordered after current master migration `0173`, are covered by numbering/safety checks, and are designed to be reapply-safe. - **Reserved namespaces:** My, Projects, and Bundled roots are service-managed. Regression coverage prevents namespace squatting, cross-company folder use, bundled writes, cycles, and excessive depth. - **Behavioral change:** Project scans choose a project folder only on initial creation; existing skills deliberately retain their current folder during refresh. - **UI scope:** The folder rail applies to the Installed library; Catalog retains the discovery-oriented category sidebar. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI `gpt-5.5` in Codex CLI, medium reasoning mode; runtime did not expose a context-window value. Used repository/file tools, terminal execution, Git/GitHub operations, test execution, and code editing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-16 15:50:45 -05:00
DottaandPaperclip	a04a77c9d3	feat(authz): govern agent inbox archive access (#9658) ... ## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their work > - The inbox subsystem must let agents act for a responsible user without silently granting access to every company user's tasks > - Existing authorization had no inbox-specific action, target-user scope, or per-user agent policy > - Inbox archive data also needs company-safe ownership and replay-safe schema changes before API mutations can rely on it > - This pull request adds the database policy foundation and a fail-closed `inbox:manage` authorization decision > - The benefit is a least-privilege core for later inbox archive endpoints, including explicit cross-user grants and low-trust denial ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting (`packages/db`, `packages/shared`, and `server`). ### Problem or motivation Agents need to manage inbox state for the user responsible for their run, but the control plane lacks an inbox-specific permission model and user-targeted grant scope. A generic mutation path would risk cross-user access or inconsistent policy enforcement. ### Proposed solution Add inbox archive ownership and per-user agent policies, introduce `inbox:manage`, and evaluate responsible-user defaults, disabled/allowlist policies, active membership, low-trust presets, and scoped cross-user grants in one authorization decision. ### Alternatives considered Reusing generic issue mutation permissions was rejected because it cannot express user-targeted inbox scope. Requiring grants for all self-user access was rejected because it would make the responsible-user path closed by default instead of using the requested per-user policy model. ### Roadmap alignment `ROADMAP.md` contains no overlapping inbox archive or inbox authorization item; this is incremental control-plane authorization work. ### Additional context This PR provides the authorization and schema foundation. Route and UI behavior can build on this decision without duplicating access-control rules. ## What Changed - Builds on the merged migration `0172_inbox_archive_agent_policies` (#9654) for company/user-scoped inbox archives and per-user agent policy rows. - Added replay-safe migration `0173_inbox_policy_agent_cleanup` with a GIN allowlist index and GIN-backed database cleanup that removes deleted agent IDs from policy allowlists. - Added Drizzle schema exports for inbox agent policies and responsible-user ownership on inbox archives. - Added the shared `inbox:manage` permission key and `scope.userIds` evaluation for user-targeted grants. - Added fail-closed inbox authorization for unresolved targets, inactive memberships, low-trust agents, disabled policies, allowlist misses, and ungranted cross-user access. - Added migration replay coverage and the full inbox authorization decision matrix. ## Verification - `pnpm exec vitest run packages/db/src/inbox-archive-agent-policies-migration.test.ts server/src/__tests__/authorization-service.test.ts` — 50 tests passed. - `pnpm --filter @paperclipai/db typecheck` - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/server typecheck` - `git diff --check origin/master...HEAD` ## Risks - The merged `0172` migration changed inbox archive uniqueness from agent-owned to responsible-user-owned rows; `0173` is additive (index + cleanup trigger) and idempotent, and replay coverage verifies both remain safe for databases that already applied an earlier form. - `scope.userIds` uses the existing JSON grant-scope parser, so malformed privileged grant payloads continue to fail through the shared parsing behavior rather than a dedicated schema. - Cross-user grants intentionally act as board-admin overrides; responsible-user default access remains bounded by disabled and allowlist policies. - The authorization action is not yet wired to public mutation routes, limiting immediate behavioral impact while establishing the contract those routes must use. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with `gpt-5.6-sol`, high reasoning effort, CLI tool use, code execution, GitHub CLI, and Paperclip control-plane integration. Context window size is not exposed by the configured adapter. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-16 09:51:48 -05:00
DottaandPaperclip	da549123cc	feat(db): add inbox archive agent policies (#9654) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The inbox tracks issue visibility separately for each responsible user > - Existing inbox archive records only identify the user whose inbox changed, not the actor that made the change > - Agent-managed inbox cleanup needs durable agent and heartbeat-run attribution for auditability > - Users also need a minimal core policy that can remain open, restrict access to an allowlist, or disable agent inbox management > - This pull request adds those database contracts without changing routes or services > - The benefit is a company-scoped, auditable foundation for later agent inbox-management APIs ## Linked Issues or Issue Description ### Subsystem affected `packages/db` — Drizzle schema and migrations. ### Problem or motivation Agent workflows such as PR gardening can reduce inbox noise after work is complete, but existing inbox archive records only identify the user whose inbox changed. They cannot retain which agent acted or which heartbeat run performed the action, and there is no user-specific policy controlling whether agents may manage that inbox. ### Proposed solution Add database-only contracts for agent-managed inbox archiving: actor type, agent ID, and heartbeat-run attribution on archive rows, plus one company-scoped policy per user with `open`, `allowlist`, or `disabled` mode. Existing archive rows retain `user` attribution by default. Route, service, and UI behavior are intentionally deferred. ### Alternatives considered - Store attribution only in activity logs: rejected because archive state needs durable, directly queryable attribution. - Add richer per-agent rules immediately: rejected because advanced policy rules belong in a later or enterprise layer; the core table stays deliberately minimal. - Implement APIs in the same change: rejected to keep this migration-focused PR independently reviewable and safe to deploy. ### Roadmap alignment `ROADMAP.md` does not currently list this capability. This PR establishes only the database foundation and does not overlap a listed roadmap item. ### Additional context Expected behavior is that legacy archive rows upgrade without backfill, new archive rows can reference an agent and heartbeat run, and each company/user pair has at most one agent policy. ## What Changed - Added actor type, agent ID, and heartbeat run ID attribution columns to `issue_inbox_archives` with enum checks and `SET NULL` foreign keys. - Added the company-scoped `user_inbox_agent_policies` table with mode validation, JSONB agent allowlists, timestamps, and unique company/user ownership. - Added migration `0172_inbox_archive_agent_policies.sql` using idempotent DDL for existing installations. - Added an embedded PostgreSQL migration test covering legacy rows, attribution round-trips, policy JSON, and policy uniqueness. ## Verification - `pnpm --filter @paperclipai/db exec vitest run src/inbox-archive-agent-policies-migration.test.ts` - `pnpm --filter @paperclipai/db typecheck` ## Risks - Low migration risk: adding the non-null actor type uses a constant `user` default so legacy rows upgrade without a data backfill. - Agent and run deletions clear attribution foreign keys by design; the actor type remains available for audit interpretation. - No API behavior changes are included, so the new contracts remain unused until follow-up service work lands. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using GPT-5.4 with reasoning, repository tools, shell execution, and test execution. The runtime did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-16 08:48:39 -05:00
DottaandPaperclip	bd7c0d5f83	fix(issues): deduplicate repeated creates (#9650) ... ## Thinking Path > Paperclip already treats issue creation as a company-scoped mutation, but retries and parallel agent heartbeats can submit the same create more than once. Client instructions cannot provide at-most-once behavior under concurrency, so the guard belongs in the server transaction. This change adds an explicit company-scoped idempotency contract, a conservative fallback for recent open same-parent titles, and run attribution for auditability. Advisory transaction locks serialize competing requests before lookup/insert, avoiding the race that affected the prior attempt. ## Linked Issues or Issue Description Fixes #6529. This is a clean replacement for #6936, which was closed because it mixed unrelated changes and its check-then-insert implementation was not concurrency-safe. Unlike that attempt, this PR is scoped to eight files, uses a dedicated idempotency-key table, and serializes duplicate candidates inside the create transaction. ## What Changed - Accept optional `idempotencyKey` and `allowDuplicate` fields on issue creation. - Replay the existing issue with HTTP 200 and deduplication metadata for a repeated company/key pair. - Deduplicate recent open issues with the same company, parent, and normalized title for 48 hours unless `allowDuplicate: true` is supplied. - Persist idempotency mappings in a company-scoped table and serialize competing creates with transaction advisory locks. - Populate `originRunId` from `X-Paperclip-Run-Id` for agent/manual creates when the body does not provide an origin run. - Add route integration coverage for key replay, title fallback, bypass, closed/old recreation, company scoping, and run attribution. ## Verification - `pnpm exec vitest run server/src/__tests__/issue-create-deduplication-routes.test.ts` — 7 tests passed. - `pnpm --filter @paperclipai/db typecheck` — passed, including migration numbering and safety checks. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `git diff --check origin/master...HEAD` — passed. - `pnpm exec vitest run server/src/__tests__/issue-assigned-backlog-contract-routes.test.ts server/src/__tests__/issue-create-deduplication-routes.test.ts` — 10 tests passed after the service-contract compatibility fix. ## Risks - The title fallback intentionally treats normalized same-parent titles as duplicates for 48 hours; callers creating intentionally repeated titles must send `allowDuplicate: true`. - Advisory locks use hashed duplicate keys, so an extremely unlikely hash collision can serialize unrelated creates but cannot merge their lookup results. - Deleting an issue cascades its idempotency mapping, allowing the same key to create a replacement later. ## Model Used - OpenAI `gpt-5.6-sol`, high reasoning effort, Codex CLI with repository, shell, GitHub CLI, and Paperclip API tool access. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-15 21:45:11 -05:00
3 people	3ae2c30f2f	feat(skills): import skills from projects (#9620) ... ## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies. > - Company skills make reusable agent behavior discoverable and editable from one place. > - Projects already contain skill directories, but operators had to import each skill path manually. > - Copying those skills would break the desired write-through workflow between Skill Studio and the source project. > - The server therefore needs a safe preview/select/import contract that only accepts rediscovered, workspace-contained candidates. > - The UI needs a guided project picker that explains reference semantics, handles conflicts, and remains usable on mobile. > - This pull request adds that end-to-end project skill import flow with authorization, tenant-scope, traversal, and symlink regression coverage. > - The benefit is faster bulk onboarding while keeping project files as the single source of truth. ## Linked Issues or Issue Description **Feature request** **Problem:** Importing several skills already stored in a Paperclip project requires operators to discover and submit each local path individually. This is slow, hides which well-known directories were searched, and makes conflict/already-imported states difficult to evaluate before mutation. **Proposed solution:** Add an “Import skills from project” flow that previews skills from well-known directories, lets operators selectively import eligible candidates, and stores local-path references so Skill Studio edits write through to the project files. **Alternatives considered:** Copying files into company-managed skill storage was rejected because it creates divergent copies. Trusting client-supplied paths was rejected because imports must be constrained to server-rediscovered, workspace-contained candidates. **Additional context:** GitHub duplicate search found no existing issue or PR for this exact workflow. Refs #3799 for related skill-import inventory behavior; this PR does not claim to close that issue. ## What Changed - Extend `scan-projects` with backward-compatible preview and selective-import modes, typed validation, candidate statuses, and OpenAPI coverage. - Discover project skills under `skills`, `.agents/skills`, `.claude/skills`, `.codex/skills`, `.cursor/skills`, `.opencode/skills`, and `.gemini/skills`. - Re-discover selections server-side, enforce company/project/workspace scope, and reject traversal or symlink escapes before creating `local_path` references. - Add the Skills-page menu entry and responsive project import dialog with project selection, grouped candidates, select all/deselect all, conflicts, empty/error/403 states, and import results. - Add route, service, and component regressions for preview authorization, cross-tenant selections, traversal/symlink safety, selection counts, grouping, and result semantics. ### Screenshots **Choose a project**  **Review discovered skills**  **Mobile selection footer**  **Import result**  ## Verification - `pnpm exec vitest run server/src/__tests__/company-skills-service.test.ts server/src/__tests__/company-skills-routes.test.ts ui/src/pages/skills/ImportSkillsFromProjectDialog.test.tsx` — 3 files, 81 tests passed. - `pnpm check:token-gates` — all token gates clean. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - Security review passed after adding tenant-scope and unauthorized-preview regressions; UX re-review approved desktop/mobile surfaces; QA passed all seven acceptance areas including write-through editing, deduplication, conflicts, empty state, and permission denial. ## Risks - Files remain referenced in project workspaces, so moving or deleting a source directory can make an imported skill unavailable; the UI explicitly communicates the reference behavior. - New well-known directory scans may discover more candidates than older versions, but preview mode prevents mutation until the operator confirms a selection. - The endpoint remains backward compatible: omitting `mode` preserves the prior full-import behavior. - No schema migration or telemetry event changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 4.8 with tool use/code execution assisted with the UI implementation and UX polish. OpenAI Codex CLI with tool use/code execution assisted with server implementation, security fixes, regression coverage, integration, and PR preparation; the runtime did not expose Codex's exact backing model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-15 18:01:44 -05:00
4 people	89ce36d7af	feat(skills): open-by-default company skill policy and core UX (#9564) ... ## Thinking Path > - Paperclip uses company skills to make agent capabilities reusable across an organization. > - Skill operations currently mix capability availability with permission checks, which creates avoidable setup friction and inconsistent denial handling. > - The policy contract needs to remain open by default while allowing company-scoped restrictions for governed deployments. > - Core owns the canonical policy actions, persistence, evaluation, API behavior, safe import boundaries, and generic denial/read-only UI. > - Enterprise policy-editor implementation belongs in the separate `paperclip-ee` repository and is intentionally excluded from this PR. ### Problem or motivation Company skill operations can encounter permission dead ends even when no explicit restriction has been configured, and import-source classification can drift between policy evaluation and execution. ### Proposed solution Define eight canonical skill policy actions, default all actions to allowed, persist company-scoped restrictions, expose policy evaluation APIs, normalize import sources at the boundary, and update Skill Studio to present actionable restriction states without embedding Enterprise Edition implementation in the core repository. ### Alternatives considered Keeping capability checks distributed across routes and UI surfaces was rejected because it duplicates policy logic and makes denial behavior inconsistent. Shipping the Enterprise policy editor in this repository was rejected because `paperclip-ee` is a separate repository and must receive its own PR. ### Roadmap alignment Extends the completed **Skills Manager** roadmap area by adding coherent governance and removing workflow dead ends. ### Additional context The core API contract remains suitable for a separate Enterprise Edition editor, but this PR contains no `paperclip-ee` package or EE-specific UI integration code. ## What Changed - Added the company skill policy contract to product and implementation documentation, including the open-by-default rule, eight canonical actions, decision shape, and core/EE ownership boundary. - Added the company-scoped policy schema, migration `0170`, shared validators, policy service, REST routes, OpenAPI coverage, and focused server tests. - Hardened import policy enforcement by normalizing import sources and keeping source classification consistent between policy evaluation and execution. - Updated core Skill Studio behavior to remove generic permission dead ends and show actionable policy/platform denial states only when an operation is actually denied. - Removed the `plugin-paperclip-ee` package, Docker wiring, EE discovery/deep-link helpers, and EE-specific UI tests/stories from this PR so that implementation can be submitted separately to the EE repository. - Preserved open-by-default behavior when no explicit company restriction exists. ## Verification - `pnpm --filter @paperclipai/ui exec vitest run src/components/skill-studio/SkillPolicySurfaces.test.tsx src/lib/skill-policy-denial.test.ts` — 20/20 passed. - `pnpm --filter @paperclipai/ui exec tsc --noEmit` — passed. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/worktree-config.test.ts` — 12/12 passed. - `pnpm check:token-gates` — passed with all gates clean. - `git diff --check` — passed. - `git diff --name-only origin/master | rg 'paperclip-ee|ee-skill-policy'` — no matches. ## Risks - Migration `0170` introduces company policy persistence; rollout depends on the migration applying before policy routes are exercised. - Open-by-default is an intentional behavioral policy: deployments expecting implicit denials must configure explicit restrictions. - Import normalization is security-sensitive and should retain focused review. - The separate EE editor must stay contract-compatible with the core policy API as policy actions evolve. ## Model Used - OpenAI Codex CLI, runtime model identifier and context-window size not exposed by this execution environment; reasoning, repository tool use, shell execution, and code review capabilities enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available to this runtime) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked or described the result above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on the latest head - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Evyatar Bluzer <bluzername@users.noreply.github.com>	2026-07-15 11:42:40 -05:00
DottaandPaperclip	1de0a3bb1e	feat(mcp) [split 2/8]: add governed access contracts (#9557) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Governed MCP access spans contracts, runtime enforcement, adapters, UI surfaces, and operator verification > - The parity reference PR #9534 is too large for effective automated or human review > - The feature therefore needs a linear stack whose individual diffs stay below the 100-file review limit > - This pull request is split 2/8 and focuses on database schema and shared governance contracts > - The benefit is a standalone, testable review boundary while preserving byte-for-byte parity at the top of the stack ## Linked Issues or Issue Description - Related parity reference: #9534 - Problem: The governed access model needs additive persistence and synchronized shared types before server enforcement can compile. - Proposed solution: Adds migrations 0148–0169, tool-access and Smoke Lab schema, shared types/validators/gallery helpers, and the minimal compile-required contract consumers identified by boundary testing. - Alternatives considered: keeping #9534 as one 403-file review, or rewriting the feature to manufacture seams; both were rejected in favor of path extraction plus compile-driven boundary moves. - Roadmap alignment: this advances the existing governed MCP/tool-access work already represented by #9534; it does not introduce a separate roadmap initiative. - Stack position: base branch is `pap10341-split/01-demo-servers`. - Merge policy: merge bottom-up, in order, only after the complete eight-PR stack has been reviewed and the top-of-stack parity gate remains empty. - Requested review: QA for migrations/validators; Greptile on every PR. ## What Changed - Adds migrations 0148–0169, tool-access and Smoke Lab schema, shared types/validators/gallery helpers, and the minimal compile-required contract consumers identified by boundary testing. - Keeps this PR below 100 changed files and independently typecheckable. - Preserves the final tree from #9534 when combined with the other seven stack levels. ## Verification - `pnpm typecheck` — passed, including migration numbering and safety checks - `pnpm --filter @paperclipai/db test` — passed - `pnpm --filter @paperclipai/shared test` — passed ## Risks - Migration or contract mistakes could affect every upper layer; all migrations are additive/idempotent and compile consumers are included in this boundary. - Stack risk: merging out of order can expose incomplete layers; mitigate by following the documented bottom-up merge policy. - Parity risk: later edits to an intermediate branch can drift from #9534; mitigate by re-running the empty top-of-stack diff before merge. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, exact model ID `gpt-5.4`; runtime-managed context window; medium reasoning with repository, shell, Git, GitHub CLI, and code-execution tools enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] Internal references are omitted except the execution-plan link explicitly required for this coordinated split stack - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge ## Stack Coordination - Internal execution plan: [PAP-13874](/PAP/issues/PAP-13874#document-plan) - Parity reference: #9534 - Stack: #9556 → #9557 → #9558 → #9559 → #9560 → #9561 → #9562 → #9563 - Merge bottom-up only after full-stack review and an empty parity diff at #9563. --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-14 12:57:20 -05:00
DottaandPaperclip	efcce9cc8e	fix(adapters): record unpriced CLI usage (#9505) ... ## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies > - Budgets and spend telemetry are control-plane safety features, not just reporting > - Local Codex and Claude adapters can execute through either ACP or their native CLI engines > - The ACP lane records usage and reported cost, but CLI JSON output often reports tokens without a price > - The CLI lane was either losing per-run usage semantics or coercing missing cost to zero, making real usage indistinguishable from a genuinely free run > - This pull request preserves CLI usage as per-run totals and records token-bearing runs without a reported price as explicitly unpriced ledger events > - The benefit is accurate usage accounting and a visible pricing gap instead of silently misleading zero-cost telemetry ## Linked Issues or Issue Description Refs #9471 Refs #9230 **Bug description** A `codex_local` run using the CLI engine can emit a final `turn.completed` event with millions of input tokens and tens of thousands of output tokens while the agent's spend ledger remains indistinguishable from a true zero-usage, zero-cost run. Claude CLI output has the same missing-price edge case. **Expected behavior** Token-bearing CLI runs should persist their usage. If the adapter reports a price, the ledger should record it as reported; if the CLI reports usage but no price, the ledger should explicitly mark the event as unpriced rather than silently treating missing price data as a reported `$0` cost. **Reproduction shape** 1. Configure `codex_local` with `engine: cli`. 2. Run a task that produces a `turn.completed` usage payload. 3. Observe token usage in the run stream. 4. Before this change, missing price data is represented as ordinary zero-cost spend and the CLI usage basis is not consistently propagated. ## What Changed - Mark Codex and Claude native CLI usage totals as `per_run` and propagate that basis through success and failure results. - Stop coercing missing Claude CLI cost to `0`. - Add `cost_status` to cost events with `reported` and `unpriced` values, including an idempotent migration and shared validation/types. - Persist token-bearing runs without a reported price as `unpriced` ledger events while retaining zero cents until an authoritative price exists. - Add parser, execute-path, heartbeat-accounting, and cost-service regression coverage for both local CLI adapters. - Document the cost-status invariant and CLI accounting behavior. ## Verification - `pnpm exec vitest run packages/adapters/codex-local/src/server/parse.test.ts packages/adapters/claude-local/src/server/parse.test.ts server/src/__tests__/codex-local-execute.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/heartbeat-cost-accounting.test.ts server/src/__tests__/costs-service.test.ts` — 6 files / 102 tests passed. - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/db typecheck` — includes migration numbering and safety checks. - `pnpm --filter @paperclipai/adapter-codex-local typecheck` - `pnpm --filter @paperclipai/adapter-claude-local typecheck` - `pnpm --filter @paperclipai/server typecheck` ## Risks - Existing cost rows default to `reported`, preserving current interpretation; only new token-bearing events with absent cost are marked `unpriced`. - This change does not invent model pricing. Budget hard stops still cannot charge an unknown amount, but operators and evals can now distinguish missing pricing from a genuinely reported zero cost. - Consumers that enumerate cost-event fields should tolerate the additive `costStatus` field. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, exact model `gpt-5.3-codex`, with repository tool use and code execution; default reasoning mode. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-13 20:44:38 -05:00
DottaandPaperclip	4a40c0cb13	feat(routines): gate scheduled runs on external activity (#9436) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI-agent companies and their recurring work > - Scheduled routines provide native cron-driven execution for recurring agent tasks > - Watcher-style routines currently dispatch a model run even when the control plane has been quiet since their last useful run > - Existing pause, catch-up, and concurrency policies do not distinguish external work from a routine's own bookkeeping > - This pull request adds a generic activity gate that checks company-scoped activity provenance before scheduled dispatch > - The benefit is backward-compatible zero-token quiet skips while real human, agent, or delegated-child activity still wakes the routine ## Linked Issues or Issue Description - Refs #8534 ## What Changed - Added `activity_gate_policy` and `activity_gate_scope` routine columns with backward-compatible `always` / `company` defaults. - Added a company-bounded `evaluateActivityGate()` predicate that uses the last dispatched run as its open window, excludes the routine's own execution runs and scheduler bookkeeping, ignores pure-read actions, and supports company/project scope. - Integrated the predicate into scheduled ticks after pause/worktree eligibility checks; quiet ticks create visible skipped run-history rows with reason `no_external_activity` and gate-window diagnostics without advancing the activity window. - Kept webhook, manual, and API dispatch paths ungated; catch-up schedules evaluate the gate once per scheduler tick. - Added migration-default, provenance predicate, project-scope, quiet-window, scheduler, and webhook-bypass coverage. ## Verification - `pnpm --filter @paperclipai/db typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm exec vitest run server/src/__tests__/routines-service.test.ts` — 51 tests passed - Embedded Postgres `EXPLAIN` for the company-scope gate scan: ```text Limit (cost=24.56..24.58 rows=1 width=24) -> Incremental Sort (cost=24.56..24.60 rows=2 width=24) Sort Key: activity.created_at, activity.id Presorted Key: activity.created_at -> Nested Loop Anti Join (cost=0.44..24.55 rows=1 width=24) Join Filter: (own_run.id = activity.run_id) -> Index Scan using activity_log_company_created_idx on activity_log activity (cost=0.15..8.19 rows=1 width=40) Index Cond: ((company_id = '00000000-0000-0000-0000-000000000001'::uuid) AND (created_at > (now() - '01:00:00'::interval)) AND (created_at <= now())) ``` ## Risks - The migration adds two non-null text columns, but constant defaults preserve all existing routine behavior and avoid a backfill step. - Project scope resolves activity through issue/run/routine provenance; tests cover in-project and cross-project issue activity, while every top-level and correlated query remains company-bounded. - This is the scheduler/schema foundation. Public API validation and documentation for configuring the new fields are intentionally handled in the next scoped follow-up. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using `gpt-5.4` with medium reasoning, repository/tool access, terminal code execution, and test execution. The runtime did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR extends the existing Scheduled Routines roadmap item - [x] I have searched GitHub for duplicate or related PRs and linked the related efficiency request above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no user-facing configuration is exposed in this scoped foundation PR) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-11 21:52:09 -05:00
3 people	36ec79c196	feat: add attention queue and Decisions surface (#9380) ... ## Thinking Path > - Paperclip is the control plane for autonomous AI companies, where operators need a reliable way to find and act on work awaiting their input. > - The attention and issue-thread interaction subsystems expose those decision points across server APIs and the board UI. > - The previous navigation and interaction presentation left these actions fragmented and did not offer a controlled rollout for the Decisions surface. > - This branch adds the attention feed, richer interaction cards, grouping, dismiss/snooze behavior, and a gated Decisions sidebar entry. > - It also keeps experimental settings and API contracts synchronized, with an idempotent migration for the new dismissal state. > - This pull request delivers the complete, tested attention/Decisions experience as one reviewable unit. ## Linked Issues or Issue Description - Adds an operator-focused attention queue and Decisions experience: grouped decision cards, semantic interaction actions, dismiss/snooze handling, resilient interaction states, and an experimental flag to control the Decisions navigation entry. ## Feature Context ### Problem or Motivation Operators currently have to hunt across approvals, interactions, failed runs, and budget alerts to find decisions that need their action. ### Proposed Solution Provide a gated Decisions attention queue that groups actionable items, supports direct resolution, and preserves operator control through dismiss and snooze actions. ### Alternatives Considered Keep separate, source-specific views only; this leaves cross-cutting operator decisions fragmented and harder to prioritize. ### Roadmap Alignment This improves the V1 control-plane operator workflow by making pending governed actions discoverable in one company-scoped surface. ## What Changed - Added server attention-feed services, routes, interaction handling, dismiss/snooze support, and an idempotent `0145` inbox-dismissal migration. - Added shared attention, inbox-dismissal, and experimental-settings contracts. - Added Decisions/attention UI, interaction-card states, sidebar badge/navigation integration, grouping, keyboard support, and Storybook coverage. - Added tests for attention behavior, thread interactions, settings normalization, dismissals, and API behavior. - Removed generated screenshots from the final PR diff and rebased the branch onto current `master`. ## Verification - `pnpm check:token-gates` — passed. - `pnpm exec vitest run packages/shared/src/issue-thread-interactions.test.ts server/src/__tests__/attention-service.test.ts server/src/__tests__/inbox-dismissals.test.ts server/src/__tests__/issue-thread-interactions-service.test.ts server/src/__tests__/issue-thread-interaction-routes.test.ts ui/src/lib/attention.test.ts ui/src/components/AttentionQueueRow.test.tsx ui/src/components/IssueThreadInteractionCard.test.tsx ui/src/pages/InstanceExperimentalSettings.test.tsx` — passed: 158 tests across 9 focused files. - GitHub Actions for `ad636f560`: build and typecheck/release-registry have passed; remaining general-server and Greptile checks are in progress. ## Risks - Moderate: this is a cross-layer attention/interaction feature with a new migration and navigation behavior. - The `enableDecisions` experimental setting defaults to off, limiting rollout impact. - Existing dismissal data is backfilled to `dismiss`; the migration is idempotent and uses guarded constraint creation. > ROADMAP.md was checked; no duplicate planned core feature was identified. Related open pull requests were searched before opening this PR. ## Model Used - OpenAI GPT-5.5 via Codex CLI, with tool use and local code execution. Context-window size unavailable in this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My public PR branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally; focused tests pass and the remaining unrelated AWS test failure is documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-10 17:09:57 -05:00
3 people	5c85ae64a0	Cases: experimental first-class case object (#9198) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The board currently uses issues for execution, but longer-lived content work needs a separate object that can survive beyond a single task thread. > - The Cases subsystem adds an experimental, company-scoped record for content artifacts and their supporting metadata. > - The backend needs durable storage, API routes, revision history, issue linkage, and company-boundary enforcement before the UI can depend on Cases. > - The UI needs an opt-in navigation surface, list/detail views, reference chips, and issue-page context so operators can inspect Cases without making them the default workflow. > - The agent-facing skills need a contract for creating and updating Cases so automated content workflows can dogfood the feature. > - This pull request ships that experimental end-to-end path behind the `enableCases` flag. > - The benefit is a first-class place to collect content work, references, attachments, revisions, and related execution threads without polluting the core issue model. ## Linked Issues or Issue Description No public GitHub issue exists for this experimental feature. Feature request fields: ### Problem Content-oriented work such as release notes, announcements, docs, and campaigns can span many execution issues, which makes the final artifact hard to find and reason about after the execution thread moves on. ### Proposed solution Add an experimental Cases object that is company-scoped, linked to issues, queryable through the API, inspectable in the board UI, and writable by agent workflows through documented conventions. ### Alternatives considered Continue encoding content artifacts directly in issues or documents only. That keeps the data model smaller, but it does not give operators a stable artifact-centric view or a clean way to link related execution history. ### Roadmap alignment Checked `ROADMAP.md`; this PR does not duplicate an existing planned core roadmap item. ## What Changed - Added the `cases` data model, migration, schema exports, and experimental `enableCases` instance setting. - Added company-scoped Cases API routes for list/detail/update, issue links, revisions, children, activity events, annotations, attachments, and idempotent agent-oriented upserts. - Scoped case and issue lookup helpers before access checks so inaccessible cross-company identifiers resolve as not found rather than leaking existence. - Fixed case PATCH timestamp handling so non-status updates cannot overwrite `completedAt` from a stale pre-transaction row snapshot. - Moved Cases list type/status/project filters into the server request before the server-side limit is applied, including multi-select filters and no-project filtering. - Added backend route coverage for creation, updates, idempotency, issue linking, attribution, company-boundary enforcement, OpenAPI registration, list filtering, timestamp patch behavior, and inaccessible lookup regressions. - Added the experimental Cases UI surface: sidebar entry, gated routes, list filters/grouping, detail overview, activity, revisions, children, attachments, and issue-page case rail. - Added case reference rendering and company-prefixed case href generation so case links resolve directly inside the active company route. - Added Paperclip skill documentation for agent workflows that create or update Cases. - Wired release-content skills to emit Cases for dogfooding. - Rebased onto current `master` and renumbered the Cases migrations to `0143`/`0144` after the latest upstream migration sequence. ## Verification - Current PR head: `ecc13be0d`. - Rebased on current `master` (`606aa4f266`) and pushed to the existing PR branch. - `git diff --check origin/master...HEAD` — passed before the first update push; subsequent committed diffs were also checked with `git diff --check` before commit. - Guardrails checked: no `pnpm-lock.yaml` changes, no `.github/workflows` changes, and changed-file count is below the Greptile 100-file limit. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/cases-routes.test.ts src/__tests__/instance-settings-service.test.ts src/__tests__/openapi-routes.test.ts` — passed, 3 files / 26 tests before review-fix commits. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/cases-routes.test.ts` — passed after each server-side Greptile fix, latest 1 file / 15 tests. - `pnpm --filter @paperclipai/server typecheck` — passed after the timestamp and lookup fixes. - `pnpm --filter @paperclipai/ui exec vitest run src/pages/Cases.test.tsx src/pages/CaseDetail.test.tsx src/pages/CompanySkills.test.tsx src/App.cases-routing.test.tsx` — passed, 4 files / 30 tests before review-fix commits. - `pnpm --filter @paperclipai/ui exec vitest run src/pages/Cases.test.tsx` — passed after the list-filter fix, 1 file / 12 tests. - `pnpm --filter @paperclipai/ui typecheck` — passed after the list-filter fix. - `pnpm check:token-gates` — passed after UI changes. - Remote PR checks on head `ecc13be0d` are green: Paperclip CI, build, typecheck, test matrix, e2e, Canary Dry Run, policy, commit review, Superagent Security Scan, Socket, Snyk, and Greptile passed; Storybook visual regression is skipped and security-review is neutral. - Greptile Review: 5/5 confidence, zero unresolved Greptile threads. ## Risks - Medium feature risk because this introduces a new experimental domain object across database, server, shared contracts, skills, and UI. - The feature is gated behind `enableCases`, which limits default operator exposure while the model is exercised. - Case links now prefer company-prefixed hrefs; the unprefixed redirect remains for externally entered URLs. - Cases list filtering now sends multi-select filters to the server before limiting; the UI still applies the same local filters as a second pass for ancestor/context rows. - Migrations were renumbered on top of current master; the SQL uses guarded `IF NOT EXISTS` / `ADD COLUMN IF NOT EXISTS` patterns where relevant for safer replay. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 Codex in the Paperclip local coding environment was used for this PR curation, rebase verification, review-fix implementation, push, and PR description update. The runtime exposes tool use and shell execution; context-window size is not exposed by this Paperclip adapter. Several implementation commits also include AI co-author trailers recorded in git history. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I searched the GitHub PR list for similar PRs and confirmed this is not a duplicate - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-09 22:11:03 -05:00
3 people	606aa4f266	feat(search): filters, sorting, operators & command-palette parity (#9327) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Company search is the primary way operators find issues, comments, documents, artifacts, agents, and projects across a busy company > - Search previously supported only a bare text query: no way to narrow by status/assignee/project/label/date, no sort control, no typed operators, and weak relevance/snippets meant hunting through noise > - As companies accumulate tens of thousands of items, unfiltered single-sort search stops scaling for day-to-day operator workflows > - This pull request adds a full filtering model (filter bar, chips, mobile sheet, URL state), sort modes, typed query operators (`status:`, `assignee:`, `type:`, …) with command-palette parity, relevance/snippet/deep-link improvements, zero-results recovery, and the supporting shared validators, backend service work, and DB indexes > - The benefit is that operators can go from a vague query to the exact item in a couple of keystrokes, on desktop and mobile, with shareable filtered-search URLs ## Linked Issues or Issue Description No existing public GitHub issue; describing the underlying feature request inline (per feature_request template): - **Problem:** Company search accepted only a plain text query. Users could not filter results by status, assignee, project, label, or recency; could not change result ordering; and got no guidance when filters emptied the result set. - **Desired solution:** Structured search filters (UI controls + typed query operators + URL parameters), selectable sort modes, better relevance and snippets with exact deep links, and parity between the search page and the command palette. - **Alternatives considered:** Client-side filtering of unfiltered results (does not scale past the fetch limit); a separate "advanced search" page (splits the surface and duplicates state handling). Related (not duplicate) PRs found while searching: #4848 (issue search query planning), #8235 (search rate limiting). ## What Changed - **Shared contract:** new search filter/sort/count/zero-results types and validators in `packages/shared` (`validators/search.ts`, types index). - **Backend:** `server/src/services/company-search.ts` supports issue filters, sort modes, per-filter option counts, snippets, artifact visibility, and zero-results loosen suggestions; single-statement match replaces per-scope scans and predicates are trigram-index compatible (~3.7s → ~350ms on a live 14.8k-hit corpus). - **DB:** migration `0142_company_search_sort_indexes.sql` adds the supporting indexes. - **Search page (`ui/src/pages/Search.tsx`):** filter bar, removable chips, mobile filter sheet with result-count preview, sort menu, URL round-tripping, zero-results recovery UI. - **Query operators (`ui/src/lib/search-query-parser.ts`):** typed operators parsed into filters, operator autocomplete, filter pills. - **Command palette:** operator-aware parsing and full-search handoff. - **Stale-operator fix (latest commit):** typed operator filters are no longer folded into persistent URL-filter state, so deleting a token (e.g. removing `status:blocked` from the input) actually removes the filter from subsequent requests; filter-control edits materialize control state and strip typed tokens so a removed chip cannot resurrect from the input. ## Verification - `cd ui && npx vitest run src/pages/Search.test.tsx` — 19 tests including two new red→green regressions for the stale-operator paths (both fail on the previous commit, pass now). - `cd ui && npx vitest run src/components/CommandPalette.test.tsx` and `cd server && npx vitest run src/services/company-search-service.test.ts` — operator parity and backend filter/sort/count coverage. - `cd ui && npx tsc --noEmit` — clean. - Manual: open `/search`, type `auth status:blocked`, confirm the status filter applies; delete `status:blocked`, confirm results are unfiltered again; drive the same filters from the filter bar/chips/mobile sheet and confirm the URL round-trips (reload/back/forward preserves state). - Full end-to-end QA pass (9/9 acceptance checks) against the wireframes on desktop (1280px) and mobile (390px) with a live API and browser automation. ## Risks - Additive migration (indexes only, no data rewrites) — safe to roll forward; index creation cost is paid once at migrate time. - Search request shape gains optional parameters only; old clients keep working. - Behavioral shift: filter-control edits now strip typed operator tokens from the query text (their values persist as filter state) — deliberate, so removed filters stay removed. - Ranking changes alter result ordering for existing queries; covered by service tests and the QA pass. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic, extended thinking + tool use) — stale-operator-filter fix, regression tests, PR preparation. - GPT-5 Codex (`codex_local` adapter) and Claude Opus 4.6 (`claude-opus-4-6`) — earlier implementation phases (backend contract, filter UI, operators, ranking) under agent orchestration. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details — pre-existing branch name retained to avoid closing/reopening the PR - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no user-facing docs affected) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (pending re-run on latest commit) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (pending re-review of the stale-filter fix) - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-09 19:32:58 -05:00
DottaandPaperclip	176645187c	Fix request storm polling and issue-list coalescing (#9190) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The board UI keeps issue, agent, activity, and run state fresh through polling across several pages and sidebar surfaces > - When multiple components or browser tabs poll the same company data at the same time, the API can receive bursts of duplicate issue-list requests > - Those duplicate requests increase database and server load without returning meaningfully different data > - This pull request adds server-side compression/coalescing plus client-side visibility-aware and cross-tab shared polling > - The benefit is lower request volume during normal board usage while preserving fresh UI data for active users ## Linked Issues or Issue Description No exact public GitHub issue was found for this request. Problem: - The board can issue redundant polling requests for the same issue-list data from multiple UI surfaces and tabs. - In busy operator sessions, those bursts can trigger request-storm behavior and unnecessary issue-list load. - Expected behavior is to reuse identical in-flight work server-side and reduce hidden-tab or duplicate-tab polling client-side while preserving normal refresh behavior. Related public context found during duplicate search: - #8206 covers a different board UI 404-storm scope. - #5165 covers separate issue-list behavior around page-size truncation. ## What Changed - Added API compression middleware foundation and a company/created-at index for heartbeat run access. - Added server-side issue-list request storm detection and identical in-flight request coalescing. - Added UI fetch metadata, visibility-aware polling, and request deduplication for issue/activity/client calls. - Added cross-tab shared polling primitives and wired them into the sidebar, inbox, dashboard, issue, project, routine, and agent surfaces. - Resolved the latest `master` migration collision by keeping upstream `0140_built_in_managed_resources.sql` and renumbering this branch's heartbeat-run index migration to `0141_heartbeat_runs_company_created_at_index.sql`; the SQL uses `CREATE INDEX IF NOT EXISTS` for idempotency. - Stabilized server heartbeat cleanup tests exposed by the PR check matrix. - Fixed the Greptile compression follow-up by weakening strong ETags on encoded JSON responses and bypassing compression for streamed/download responses. ## Verification - `pnpm exec vitest run ui/src/components/IssuesList.test.tsx ui/src/pages/Inbox.test.tsx` — passed after resolving the latest `master` conflict in `IssuesList.tsx` and updating the 200-result cap expectations. - `pnpm check:token-gates` — passed after the UI conflict resolution. - `jq -e '.entries | length as $n | (map(.idx) | unique | length == $n) and (map(.tag) | unique | length == $n)' packages/db/src/migrations/meta/_journal.json` — passed after renumbering the migration to `0141`. - `pnpm exec vitest run server/src/__tests__/api-compression.test.ts` - passed after the compression follow-up. - `pnpm exec vitest run server/src/__tests__/issue-list-assignee-filter-routes.test.ts` - passed after the compression follow-up. - `pnpm --filter @paperclipai/server typecheck` - passed after the compression follow-up. - Greptile Review for head `8dbddac41ec273fda404100b4981ddb912fad57b` - passed after the latest conflict/migration fix; all Greptile review threads are resolved. - `pnpm exec vitest run server/src/__tests__/api-compression.test.ts server/src/__tests__/issue-list-assignee-filter-routes.test.ts ui/src/api/client.test.ts ui/src/api/issues.test.ts ui/src/lib/polling.test.ts ui/src/lib/cross-tab-poll.test.ts ui/src/pages/Inbox.test.tsx ui/src/components/SidebarProjects.test.tsx ui/src/components/SidebarAccountMenu.test.tsx ui/src/lib/issueDetailCache.test.ts` — passed. - `pnpm exec vitest run ui/src/api/client.test.ts` — passed. - `pnpm exec vitest run ui/src/pages/Inbox.test.tsx ui/src/components/SidebarProjects.test.tsx ui/src/components/SidebarAccountMenu.test.tsx ui/src/lib/issueDetailCache.test.ts` — passed. - `pnpm exec vitest run server/src/__tests__/heartbeat-worktree-suppression.test.ts` — passed. - `pnpm exec vitest run server/src/__tests__/low-trust-red-team-routes.test.ts` — passed. - `pnpm exec vitest run server/src/__tests__/heartbeat-workspace-finalize-branch.test.ts` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - GitHub PR checks for head `8dbddac41ec273fda404100b4981ddb912fad57b`: all GitHub Actions/status checks passed; Greptile, Superagent, Socket, Snyk, build, typecheck/release registry, general tests, serialized server suites, e2e, canary dry run, policy, and commitperclip review are green; Storybook visual regression and security-review were skipped/neutral by policy. - Confirmed this branch does not include `pnpm-lock.yaml` or `.github/workflows` changes. ## Risks - Medium risk: issue-list coalescing changes request timing and cache semantics for a hot API path. - Medium risk: cross-tab polling uses browser coordination primitives, so older or unusual browser environments need fallback behavior to stay correct. - Low migration risk: the new index migration is ordered after current `master` and uses `CREATE INDEX IF NOT EXISTS`. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex coding agent, GPT-5-based model, tool-enabled with shell/git execution. Exact hosted deployment identifier and context-window size were not surfaced in the agent runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-09 17:38:34 -05:00
DottaandPaperclip	8b6a06ee25	[codex] Add built-in agents and Reflection Coach bundle (#9206) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators need first-party agent capabilities for repeatable company work, not just manually created one-off agents. > - Built-in agents need to behave like normal company-scoped agents while preserving approval gates, permissions, budgets, and audit trails. > - Reflection and coaching work also needs bundled instructions, skill content, and a routine so the feature can be installed and reset predictably. > - The API, database, UI, portability, and tests all need to agree on the built-in lifecycle from not provisioned through setup, approval, ready, paused, and reset. > - This pull request adds built-in agent provisioning and the Reflection Coach bundle end-to-end. > - The benefit is a safer first-party path for Paperclip-managed agents without bypassing the same governance model used for operator-created agents. ## Linked Issues or Issue Description No public GitHub issue was found for this exact built-in agent and Reflection Coach bundle work. Problem/motivation: - Paperclip did not have a first-party built-in agent lifecycle for product-owned agents. - Bundled agent resources such as default instructions, skills, and routines needed managed ownership and reset semantics. - Approval-gated companies needed built-in setup to preserve requested adapter, budget, manager, and permission state through board approval. - The board UI needed clear built-in badges, setup affordances, readiness state, and bundle status without exposing secrets. Proposed solution: - Add a company-scoped built-in agent registry, provisioning/reset/reconcile/status APIs, and Reflection Coach bundled resources. - Track bundled managed resources in the database with idempotent migration behavior. - Reuse existing agent approval, authorization, budget, and activity-log paths instead of creating a bypass. - Add UI setup, badges, gates, bundle panels, and route coverage for built-in agents. Duplicate search: - Searched GitHub PRs for `built-in agents Reflection Coach repo:paperclipai/paperclip`; only this PR was returned. - Searched GitHub issues for the same query; no public issues were returned. ## What Changed - Added built-in agent definitions, lifecycle state derivation, provisioning, reset, reconcile, status, and routine-control routes. - Added the `built_in_managed_resources` migration and schema exports for bundled instructions, skill, and routine ownership. - Added the Reflection Coach built-in bundle with default instructions, skill catalog content, routine template, default permissions, and managed-resource drift handling. - Added approval-aware provisioning behavior that preserves requested adapter config, budgets, manager assignment, and built-in permissions through hire approval. - Added authorization and mutation gates for built-in agent and skill changes, including consented Reflection Coach change paths. - Added UI surfaces for built-in agent setup, roster/detail badges, readiness gates, bundle status, routine controls, and route filtering. - Added company import/export and validator coverage for built-in managed resources and low-trust/red-team presets. - Addressed Greptile follow-ups for pending approval reconciliation, consent-gate error propagation, config-read authorization fallback, approval-path manager preservation, and non-model adapter provisioning. ## Verification Local verification: - `git diff --check public/master..HEAD` passed. - `pnpm check:token-gates` passed with all gates clean. - `pnpm exec vitest run ui/src/components/ConfigureBuiltInAgentModal.test.tsx` passed: 1 file, 4 tests. - `pnpm exec vitest run ui/src/components/EntityRow.test.tsx ui/src/pages/Agents.test.tsx ui/src/components/BuiltInAgentGate.test.tsx ui/src/components/ConfigureBuiltInAgentModal.test.tsx ui/src/components/BuiltInBundlePanel.test.tsx ui/src/pages/InstanceExperimentalSettings.test.tsx ui/src/pages/Routines.test.tsx` passed: 7 files, 64 tests. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/built-in-agents.test.ts src/__tests__/authorization-service.test.ts src/__tests__/company-skills-routes.test.ts` passed: 3 files, 91 tests. - `pnpm --filter @paperclipai/db check:migrations` passed. - `pnpm -r typecheck` passed after the rebase; `pnpm --filter ui typecheck` passed after the final UI review fix. Remote verification on latest head `1c61f693a4ec881d739022b0e75a8ca8bf8c2cd8`: - Merge state: `CLEAN`. - Greptile: `5/5`, zero unresolved Greptile threads. - PR check rollup: all checks successful, neutral, or skipped as expected. - Passing gates include Build, Typecheck + Release Registry, all server shards, all workspace shards, all serialized server suites, e2e, Canary Dry Run, policy, review, verify, Socket, Superagent, and Snyk. ## Risks - This adds a new managed-resource table and migration; the migration uses idempotent create/add/index guards and passed migration safety checks. - Built-in agent provisioning touches approval and authorization paths; tests cover pending approval preservation, stale retry rejection, consent gates, and config-read fallback behavior. - Reflection Coach creates managed instructions, skill, and routine resources; drift/reset behavior is covered by service tests and redacted API responses. - Non-model adapter setup now provisions a `needs_setup` built-in row before command/endpoint fields are complete; this matches the server lifecycle and is covered by the setup modal regression test. ## Model Used OpenAI Codex coding agent based on GPT-5. Exact hosted model ID, context-window size, and reasoning-mode labels are not exposed in this runtime; tool use, shell execution, GitHub CLI/API access, and local code editing were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-09 16:29:30 -05:00
3 people	b13eb5b2b5	Skill Studio: three-pane skill IDE with sandboxed test runs (#9241) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Skills Manager gives operators a reusable skill layer, but iteration still required manual edits, ad hoc prompts, and indirect run inspection. > - Skill authors need a focused workflow for editing skill files, saving representative test inputs, and running those inputs through an agent without exposing harness tasks as normal company work. > - The backend therefore needs durable test inputs, reusable run templates, hidden harness issues, scoped run execution, retention metadata, and read-containment rules around hidden work. > - The frontend needs a three-pane Studio that keeps skill files, saved inputs/templates, and run output/history visible together while preserving the existing design system and token rules. > - This pull request ships that Skill Studio surface end to end: database migrations, shared contracts, server APIs/services, hidden harness execution behavior, UI routes/components, and focused tests. > - The benefit is faster and safer skill iteration, with inspectable outputs and fewer ways for internal harness work to leak into normal task lists, costs, or adjacent read APIs. ## Linked Issues or Issue Description No public GitHub issue exists for this feature. Feature request summary: - Problem: Skill authors need to edit and test company skills in one place instead of switching between the skill detail page, task creation, run output, and manual prompt history. - Proposed solution: Add a Skill Studio workbench with saved inputs, reusable templates, hidden sandboxed test runs, live run status, output inspection, run history, rerun/delete controls, and frontmatter-aware editing. - Expected users: Paperclip operators and agent-company maintainers who create, fork, import, and tune skills. - Related public PRs: Supersedes #9205, which was replaced so the public PR branch name follows contributor policy. - Duplicate search: searched public GitHub issues and PRs for "Skill Studio"; no other active public issue or PR directly covers this feature. ## What Changed - Added database migrations for Skill Studio test inputs, test runs, test run retention, and reusable run templates. - Added shared Skill Studio types, validators, route helpers, frontmatter utilities, and status handling. - Added server services and routes for saved inputs, test runs, templates, reruns, terminal-run deletion, hidden harness issue execution, and run-detail hydration. - Strengthened hidden-issue read containment across issue-adjacent routes and cost rollups used by skill test harness work. - Added the Skill Studio UI with skill file editing, frontmatter editing, saved inputs, templates, run creation/cancel/rerun/delete flows, output rendering, history, route support, and responsive pane behavior. - Added focused backend, shared, and UI tests for the new APIs, routing logic, editor/run behavior, hidden-issue containment, and migration safety. - Rebased onto current `master`, removed the generated lockfile diff from the PR, and verified no workflow files are changed. ## Verification - [x] `pnpm --filter @paperclipai/db check:migrations` - [x] `pnpm check:token-gates` - [x] `pnpm exec vitest run server/src/__tests__/company-skills-service.test.ts server/src/__tests__/company-skills-routes.test.ts server/src/__tests__/company-skill-test-runs-service.test.ts ui/src/lib/skill-studio.test.ts ui/src/pages/SkillStudio.test.tsx` — 5 files, 132 tests passed - [x] Greptile review on the latest PR head - [x] GitHub PR checks on the latest PR head ## Risks - Medium risk because this is a broad feature touching database schema, server orchestration, issue visibility, and a large UI surface. - Hidden harness issue containment is security-sensitive; this PR includes regression coverage for adjacent read paths and cost rollups. - The new migrations are additive and use idempotent guards where applicable, but deployed databases that previously tested draft migration numbers should still be checked carefully. - The UI depends on a new resizable panels package in `ui/package.json`; the lockfile is intentionally left to repository automation. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5 coding agent with shell, git, and GitHub CLI tool use. Earlier feature commits include assistance from other Paperclip coding agents; this PR preparation, rebase, cleanup commit, and PR body were completed by OpenAI Codex in a Paperclip worktree. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-09 13:08:56 -05:00
Devin FoleyandPaperclip	eedc7ddef2	Make ACP the default engine for local adapters (#9238) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Adapter packages are the bridge between the control plane and local agent harnesses such as Claude Code, Codex, and Gemini CLI. > - ACP support was concentrated in a separate `acpx_local` adapter, which made ACP feel like a separate agent choice instead of an execution capability of the harness adapters. > - Claude, Codex, and Gemini now have ACP-capable harnesses, so the native adapter should own ACP selection, fallback, config, transcript parsing, and environment diagnostics. > - The standalone ACPX adapter still needs a compatibility path for existing rows, but it should not be offered as an active adapter for new agents. > - This pull request moves the shared ACP runtime into `@paperclipai/acpx-engine`, wires Claude/Codex/Gemini local adapters to prefer ACP when prerequisites are available, and retires `acpx_local` to a tombstone. > - The benefit is one adapter per harness, richer ACP transcripts by default where possible, and a migration path for existing Claude/Codex ACPX agents. ## Linked Issues or Issue Description Closes #5932 — the broken default `acpx_local` Claude path is replaced by native `claude_local` ACP support, existing Claude/Codex ACPX rows migrate to native adapters, and new agents no longer choose the standalone ACPX adapter. Refs #4893 — original merged ACPX local adapter runtime that this PR replaces with native per-harness ACP engines. Refs #6590 — prior ACPX-Claude seamlessness work folded into the new native Claude ACP path. Refs #197 — related open generic ACP/Kiro adapter work; this PR does not close it because Kiro/custom generic ACP remains a separate adapter decision. Refs #7018 — related Kimi-specific `acpx_local` shell failure; this PR retires the built-in standalone adapter but does not add a native Kimi adapter. Refs #8864 — related ACPX prompt/API guidance PR; this PR moves runtime guidance into the shared/native ACP engine path instead of the old standalone adapter. Refs #8881 — related `acpx_local` POSIX shell failure from the old `acpx` pin; this PR updates ACP dependencies but does not claim custom/OMP ACP support as a first-class native adapter. Refs #8964 — related open `acpx_local` stderr cleanup PR; this PR makes the old runtime path obsolete for new agents but keeps it as a non-closing reference. Problem description: - The standalone `acpx_local` adapter duplicates Claude/Codex agent choices that already have first-class local adapters. - ACP should be an execution engine capability of each harness adapter when the underlying harness supports ACP. - Existing `acpx_local` agents should either migrate to native harness adapters or fail with an explicit retirement message instead of silently falling back to the process adapter. ## What Changed - Added `@paperclipai/acpx-engine` as the shared ACP execution, session-codec, CLI formatter, and UI parser package. - Wired `claude_local`, `codex_local`, and `gemini_local` to auto-select ACP by default when prerequisites pass, with `engine=cli` opt-out and `engine=acp` strict mode. - Added ACP config schema/UI fields, environment checks, session-codec preservation, transcript parsing, and adapter capability metadata for the native adapters. - Retired `acpx_local` to a server tombstone, removed its UI/package/runtime image surface, and added a migration for existing Claude/Codex ACPX agents. - Updated package manifests, lockfile, release tooling, docs, Kubernetes sandbox defaults, and tests. ## Verification - `corepack pnpm --filter @paperclipai/acpx-engine typecheck` - `corepack pnpm --filter @paperclipai/adapter-claude-local typecheck` - `corepack pnpm --filter @paperclipai/adapter-codex-local typecheck` - `corepack pnpm --filter @paperclipai/adapter-gemini-local typecheck` - `corepack pnpm --filter @paperclipai/acpx-engine exec vitest run` - `corepack pnpm --filter @paperclipai/adapter-claude-local exec vitest run src/server/acp.test.ts src/server/execute.acp-fallback.test.ts src/ui/build-config.test.ts` - `corepack pnpm --filter @paperclipai/adapter-codex-local exec vitest run src/server/acp.test.ts src/ui/build-config.test.ts` - `corepack pnpm --filter @paperclipai/adapter-gemini-local exec vitest run src/server/acp.test.ts src/ui/build-config.test.ts src/ui/parse-stdout.test.ts` - `corepack pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && corepack pnpm --filter @paperclipai/server exec tsc --noEmit` - `corepack pnpm --filter @paperclipai/server exec vitest run src/__tests__/adapter-routes.test.ts src/__tests__/adapter-session-codecs.test.ts src/__tests__/adapter-models.test.ts` - `corepack pnpm --filter @paperclipai/ui typecheck` - `corepack pnpm --filter @paperclipai/ui exec vitest run src/adapters/metadata.test.ts src/adapters/adapter-display-registry.test.ts src/components/AgentConfigForm.test.ts src/components/AgentConfigForm.render.test.tsx src/components/transcript/RunTranscriptView.test.tsx` - `node --test scripts/bootstrap-npm-package.test.mjs scripts/release-package-map.test.mjs scripts/verify-release-registry-state.test.mjs` Note: the server typecheck script calls `pnpm` internally; this dev shell exposes pnpm through Corepack only, so I ran the two script steps manually with `corepack pnpm`. ## Risks - Migration changes existing `acpx_local` Claude/Codex agents to native adapter types and clears old ACPX task sessions/runtime state. - Custom ACP commands remain on the retired tombstone and will need a separate future adapter/plugin path. - ACP auto-selection depends on local Node and ACP server command prerequisites; remote and unsupported environments fall back to CLI unless `engine=acp` is explicit. - `@paperclipai/acpx-engine` is a new public package and needs npm trusted-publishing bootstrap before release automation can publish it. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex coding agent. Exact hosted model build and context-window size are not exposed in this runtime. Tool use included shell execution, repository editing, GitHub CLI operations, and local test/typecheck execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-08 19:05:03 -07:00
DottaandPaperclip	0f08c2b526	fix(db): repair responsible user migration timestamps (#9146) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The database migration layer keeps local and deployed instances moving forward safely as schema/data contracts evolve. > - The responsible-user backfill migration could make historical issues look newly updated by bumping user-visible `updated_at` columns during a backfill. > - That timestamp churn can invalidate inbox/archive state and make old work appear fresh even though no user-facing activity happened. > - This pull request moves the responsible-user invariant migration later in the current migration sequence, keeps it from touching user-visible timestamps, and adds a repair sweep for databases that already saw the timestamp bump. > - The benefit is safer migration replay and regression coverage for future backfills that might otherwise mutate visible timestamps. ## Linked Issues or Issue Description No public GitHub issue exists. Inline bug report: **Pre-submission checklist** - [x] I have searched existing open and closed issues and this is not a duplicate. - [x] I am on the latest released version of Paperclip (or can reproduce on `master`). - [x] I have confirmed the error originates in Paperclip itself — not in my agent adapter, API provider, or local configuration. **What happened?** A responsible-user migration backfill could update user-visible `updated_at` columns while filling missing responsible-user data. That makes historical issues/runs/routines appear newer even when no user-facing activity happened. **Expected behavior** Responsible-user backfills should populate ownership metadata without mutating user-visible recency fields, and databases already affected by a timestamp sweep should be repairable. **Steps to reproduce** 1. Start from current `master` with the responsible-user migration sequence. 2. Apply migrations to a database containing historical issues, runs, routines, and companies with older activity timestamps. 3. Replay the responsible-user invariant migration and inspect user-visible `updated_at` values. **Paperclip version or commit** `master` at the PR base. **Deployment mode** Local dev (`pnpm dev`) and deployed instances using the same migrations. **Installation method** Built from source (`pnpm dev` / `pnpm build`). **Agent adapter(s) involved** - [x] Not adapter-specific (core bug) **Database mode** External Postgres and embedded development Postgres migration paths. **Access context** Board and agent-visible issue recency can both be affected. **Relevant logs or output** Covered by the added embedded-Postgres regression tests. **Relevant config (if applicable)** Not applicable. **Additional context** This PR adapts an extracted local migration fix onto the current master migration sequence, where `0133` is already occupied. **Privacy checklist** - [x] I have reviewed all pasted output for PII (usernames, file paths, API keys, tokens, company names) and redacted where necessary. ## What Changed - Renumbered the responsible-user invariant migration onto the current master migration sequence. - Added a repair migration that detects broad timestamp sweeps and restores safer `updated_at` values for issues, heartbeat runs, routines, routine runs, and companies. - Added focused embedded-Postgres regression coverage for the relocated migration, repair migration, and updated-at backfill allowlist. ## Verification - `pnpm --filter @paperclipai/db run check:migrations` - `/srv/paperclip/home/paperclipai/paperclip/node_modules/.bin/vitest run packages/db/src/client.test.ts -t "migration 0134|migration 0135|unallowlisted migration backfills"` ## Risks Migration behavior is the main risk. The PR intentionally changes the active migration sequence by removing the old responsible-user invariant slot and replaying that work later with a repair migration. Reviewers should confirm this matches the intended release/migration policy for installations that may already have applied the earlier migration. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5.5 coding agent with repository tool use and local shell execution. Context window was not surfaced by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-07 05:54:17 -05:00
Nicky LeachandPaperclip	22001bbd2f	feat(db): add migration safety lint ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The `packages/db` module owns all database migrations via a sequential numbering system already validated at build time > - A recent migration introduced an O(n²) batch backfill over a large, unindexed table — it caused the server's `listen` to block for ~5 minutes on databases with millions of rows > - Nothing in the current CI pipeline catches large-table migration risk patterns (DO-loop mutations, batched LIMIT mutations without support indexes, full-table mutations, non-concurrent index creation) before they land > - This PR wires a new static migration-safety checker (`check-migration-safety.ts`) into the existing `check:migrations` gate in `packages/db/package.json`, so risky patterns fail CI before reaching production > - The checker baselines all historical findings already present in the codebase, so the gate fails only on *new* unbaselined risky patterns > - The benefit is that the specific O(n²) backfill shape (and related patterns) will be caught at author time rather than at incident time ## Linked Issues or Issue Description **Feature — static migration safety lint** **Problem or motivation** Migrations against large tables (millions of rows) have caused production startup blocks. The root pattern is a batched `LIMIT`-based backfill iterating via an unindexed column, making each batch a sequential scan — O(n²) overall. No CI gate exists to flag this class of problem before merge. **Proposed solution** A static SQL-level checker that scans new migration files for known dangerous patterns against known-large tables, producing structured findings that are either baselined (suppressed) or fail the build. Patterns detected: `DO $$ loop` mutations on large tables without a same-migration support index, batched `LIMIT` mutations on large tables missing a same-migration support index, unbounded full-table mutations (no `WHERE` clause), and `CREATE INDEX` without `CONCURRENTLY` on large tables. **Alternatives considered** Runtime instrumentation (only catches issues in production), advisory locking in migrations (doesn't prevent the pattern), per-migration code review (doesn't scale consistently). **Roadmap alignment** Defensive infrastructure / operational reliability — keeps migrations from blocking production startups. Not a user-facing feature. ## What Changed - **`packages/db/package.json`** — extended `check:migrations` script to run `check-migration-safety.ts` after the existing numbering check - **`packages/db/src/check-migration-safety.ts`** — new static checker: SQL pattern matching, rule detection for four dangerous patterns, baseline diffing, and structured exit with findings summary - **`packages/db/src/migration-safety-baseline.ts`** — baseline of all existing historical findings (suppressed from failing the gate); new migrations matching these patterns without a baseline entry will fail - **`packages/db/src/table-size-estimates.ts`** — rough table size estimates from the local dev database; drives `isKnownLargeTable()` used by the safety rules - **`packages/db/src/check-migration-safety.test.ts`** — Vitest coverage for the O(n²) backfill failure mode, suppression via baseline, and each rule type ## Verification ```bash # Run the migration safety checker directly cd packages/db tsx src/check-migration-safety.ts # Run tests cd packages/db npx vitest run src/check-migration-safety.test.ts # Run the full migration check gate (numbering + safety) cd packages/db pnpm run check:migrations ``` - Tests cover the core O(n²) backfill pattern (the motivating incident), baseline suppression, and all four rule types - `check:migrations` now exits non-zero for any new unbaselined large-table migration risk pattern ## Risks - **False positives:** Table-size estimates are from a local dev database snapshot — a table small in dev but large in production would be missed. Best-effort heuristic. - **Baseline drift:** If a baselined finding's SQL changes significantly, the baseline ID (content-hash-based) will no longer match and the finding will re-surface. Intentional but may surprise authors doing incremental fixes. - **SQL parsing limitations:** Regex-based pattern matching rather than a full AST parser — complex SQL may not be detected. Acceptable for an initial gate. - **Low risk to existing behavior:** The gate only fails on *new* findings not present in the baseline. All existing migrations are baselined. ## Model Used - **Provider:** Anthropic - **Model ID:** `claude-sonnet-4-6` - **Context window:** 200K - **Tool use:** yes (file reading, bash, git operations) - **Reasoning mode:** standard (no extended thinking) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-07 01:43:12 +00:00
DottaandPaperclip	d2e3f7dce5	fix(db): correct 0130 responsible-user backfill in place (inbox resurface) (#9111) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Users triage agent work through the issue inbox, which suppresses archived issues by comparing issue `updated_at` against the archive timestamp > - Migration `0130_run_responsible_user_invariant` backfilled responsible-user columns but also set `updated_at = now()` on every row it touched (companies, issues, routines, routine_runs, heartbeat_runs) > - That blanket timestamp bump made every archived issue look newly updated, resurfacing thousands of archived issues into every user's inbox > - This pull request corrects the 0130 backfill in place so it only fills `NULL` responsible-user columns and never touches timestamps, and adds tests that prevent this class of bug from being reintroduced > - The benefit is that inbox archive suppression stays intact across migrations, and no future migration backfill can silently bump `updated_at` on user-visible tables ## Linked Issues or Issue Description **Bug description (no public issue exists):** - **What happened:** after upgrading a dev instance across migration 0130, every previously archived inbox item resurfaced as unread/new for all users. - **Expected:** data backfills must not alter row modification timestamps; archived issues stay archived unless genuinely updated. - **Root cause:** the 0130 backfill's `UPDATE` statements set `updated_at = now()` alongside the responsible-user columns. ## What Changed - `packages/db/src/migrations/0130_run_responsible_user_invariant.sql`: removed all `updated_at = now()` assignments from the backfill UPDATEs; the migration now only fills `NULL` responsible-user columns. The file is corrected **in place** (no new migration number, journal untouched) because 0130 has never shipped in a published release. - `packages/db/src/client.test.ts`: added a guard test that scans every migration and rejects backfills that bump `updated_at` on user-visible tables (with an explicit allowlist for the pre-existing 0131 repair migration). - `packages/db/src/client.test.ts`: added a replay test that simulates an already-migrated database picking up the corrected file (deletes the 0130 ledger hash, re-applies mid-journal) and asserts issue `updated_at` and inbox-archive suppression ordering are untouched. ### Why an in-place edit is safe - 0130 only exists on master/canary builds; the latest published release (v2026.626.0) predates it. - The migration ledger is content-hash based: databases that already applied the old 0130 keep an orphaned hash row (harmless) and see the corrected file as pending, so they replay the corrected backfill — which is idempotent (fills `NULL`s only, no timestamp writes). - Fresh databases simply run the corrected 0130 in journal order. ## Verification - `pnpm --filter @paperclipai/db run check:migrations` — clean - `cd packages/db && npx vitest run src/client.test.ts` — 11/11 passing against embedded Postgres, including the new guard and mid-journal replay tests ## Risks - Migration safety: the corrected backfill is idempotent and only writes `NULL` columns; replay on already-migrated databases is exercised directly by the new test. No schema changes. - Databases that already ran the old 0130 keep the bumped timestamps from that run; repairing historical damage is intentionally out of scope here (no released build ever contained the bug). ## Model Used - Claude Opus 4.7 (`claude-opus-4-7`, extended thinking, tool use) via Claude Code / Paperclip agent runtime ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (backup health alert work split into #9113; no other related open PRs) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (none needed for a migration content fix) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-06 16:56:30 -05:00
DottaandPaperclip	903886bc79	[codex] Add starred resource sidebar controls (#9085) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The board UI is the main daily navigation surface for agents, projects, and their related resources. > - Operators need a lightweight way to keep frequently used agents and projects close without changing company-wide ordering or ownership. > - Resource memberships already model per-user relationships to projects and agents, so they are the right place to store user-specific starred state. > - This pull request extends that membership contract with a starred timestamp and exposes star controls in list/detail views. > - The sidebar then uses those starred memberships to show compact, user-specific shortcuts. > - The benefit is faster navigation without introducing a separate favorites system or leaking preferences across users. ## Linked Issues or Issue Description No public GitHub issue exists. Feature request: ## Problem or motivation Users cannot pin frequently used agents or projects into the main sidebar. Returning to important resources requires scanning full project/agent lists or navigating through detail pages, which adds friction to repeated daily workflows. ## Proposed solution Store a per-user `starred_at` timestamp on agent and project memberships, expose API actions to set or clear that state, add star toggle controls to list/detail pages, and render starred projects and agents as compact sidebar shortcuts. ## Alternatives considered A separate favorites table would work, but it would duplicate membership scoping and require another resource relationship model. Keeping starred state on memberships preserves existing company/user boundaries and avoids a second source of truth. ## Roadmap alignment Checked `ROADMAP.md`; no overlapping planned core work for starred resource/sidebar navigation was found. ## Additional context The affected subsystems are `packages/db`, `packages/shared`, `server/`, and `ui/`. The migration is idempotent with `IF NOT EXISTS` guards so environments that saw an earlier local migration name can still apply the final ordered migration safely. ## What Changed - Added idempotent migration `0133_resource_membership_stars` for `starred_at` columns and lookup indexes on agent/project memberships. - Extended shared resource membership types and validators with starred metadata and actions. - Updated server resource membership services/routes to read and mutate starred resource state. - Added reusable star toggle UI and resource membership hook support for starred state. - Added starred projects and agents sidebar rendering, plus star controls on list and detail pages. - Added focused shared, server, and UI coverage for starred membership behavior and sidebar rendering. ## Verification - Rebased and force-with-lease pushed current PR head `a086fc965391c9e50a51b5b83b5b44a797b2a6f4` onto current `paperclipai/paperclip:master`; `gh pr view` reports `MERGEABLE` with no merge conflicts. GitHub checks are green for this fresh head. - `pnpm exec vitest run packages/shared/src/resource-memberships.test.ts server/src/__tests__/resource-memberships-routes.test.ts server/src/__tests__/workspace-runtime.test.ts ui/src/components/Sidebar.test.tsx ui/src/components/SidebarAgents.test.tsx ui/src/components/SidebarStarredProjects.test.tsx ui/src/components/StarToggle.test.tsx ui/src/pages/InstanceExperimentalSettings.test.tsx` passed after the rebase: 8 files, 143 tests. - Greptile re-review is 5/5; the remaining screenshot thread was resolved as non-blocking because this task explicitly requested no screenshots/images in the PR. - `pnpm exec vitest run ui/src/components/SidebarStarredProjects.test.tsx` passed after the mobile pending-spinner fix. - `pnpm exec vitest run packages/shared/src/resource-memberships.test.ts server/src/__tests__/resource-memberships-routes.test.ts ui/src/components/Sidebar.test.tsx ui/src/components/SidebarAgents.test.tsx ui/src/components/SidebarStarredProjects.test.tsx ui/src/components/StarToggle.test.tsx ui/src/pages/InstanceExperimentalSettings.test.tsx` passed: 7 files, 68 tests. - `pnpm --filter @paperclipai/db typecheck && pnpm --filter @paperclipai/shared typecheck && pnpm --filter @paperclipai/server typecheck && pnpm --filter @paperclipai/ui typecheck` passed db/shared/server, then failed in pre-existing UI code outside this PR: `src/pages/CompanyEnvironments.tsx` missing `@xterm/*` type declarations and `previous` possibly null. - Checked that the PR diff does not include `pnpm-lock.yaml` or `.github/workflows` changes. - Checked `ROADMAP.md` and found no overlapping planned core work for starred resource/sidebar navigation. - Searched existing GitHub PRs for duplicate starred-resource/sidebar work and found none. ## Risks - Migration touches membership tables. The SQL uses `IF NOT EXISTS` for columns and indexes so environments that saw an earlier local migration name can still apply this safely. - Sidebar ordering and visibility changes could affect users who rely on the previous flat sidebar layout. - Starred state is per-user membership metadata; code paths must continue preserving company/user scoping around memberships. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 Codex, tool-enabled coding agent with shell/GitHub access. Context window not disclosed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-06 14:09:11 -05:00
Nicky LeachandPaperclip	c5e03c6d01	fix(db): relocate slow 0126 issue-comment attribution backfill to fast idempotent 0132 (#9108) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The database layer runs migrations during server startup via `server.listen`; migrations that block this path delay instance availability > - Migration `0126_issue_comment_derived_attribution.sql` backfills derived attribution columns on `issue_comments` using a LIMIT-5000 loop with no index or keyset cursor — it re-scans the full table from the start each batch, giving O(n²) complexity > - On instances with millions of issue comments this blocked `server.listen` for ~5 minutes during upgrade, causing CPU pegs and unavailability > - Editing 0126 in place is unsafe: the migration runner keys its applied-set on file **content hash**, so any edit changes the hash, causing the runner to re-apply the migration on already-migrated databases and blocking startup > - The safe remedy is delete-and-relocate: remove 0126 and add a new forward migration 0132 that uses a temporary partial index + keyset pagination (`id > last_comment_id ORDER BY id LIMIT 5000`) so every row is visited exactly once — O(n) > - This pull request implements that delete-and-relocate with idempotency guards (IF NOT EXISTS DDL, backfill WHERE clause that skips already-attributed rows) so it is a safe near-noop on already-migrated, partially-migrated, and fresh databases alike ## Linked Issues or Issue Description No pre-existing public GitHub issue. Inline bug report: **What happened?** The `0126_issue_comment_derived_attribution` migration runs during server startup and uses a LIMIT-5000 batch loop that re-scans `issue_comments` from row 1 each iteration (no index, no keyset cursor). The result is O(n²) I/O that blocked `server.listen` on large instances. **Expected behavior** Backfill migrations should advance with a keyset cursor so each batch reads a new slice; total work is O(n) and startup is not blocked. **Steps to reproduce** Run a Paperclip upgrade on an instance with ≥200k issue comments; observe `server.listen` blocked for several minutes and CPU peg during migration. **Paperclip version or commit** Reproduced on the current `master` branch prior to this fix. **Deployment mode** All deployment modes that run the migration runner at startup. ## What Changed - **Deleted** `packages/db/src/migrations/0126_issue_comment_derived_attribution.sql` — the O(n²) LIMIT-5000 loop with no index/cursor - **Added** `packages/db/src/migrations/0132_issue_comment_derived_attribution_fast.sql`: - Creates a temporary partial index over the eligible predicate before backfilling - Uses keyset pagination (`id > last_comment_id ORDER BY id LIMIT 5000`) — each batch advances to the batch-max id, so every row is visited once - Drops the temporary index at the end - Columns/FKs guarded with `IF NOT EXISTS`; Option-A timing-tier cleanup preserved; human-authored comments never touched - `WHERE` clause in the backfill excludes rows already attributed (safe near-noop on already-migrated DBs) - **Updated** `packages/db/src/migrations/meta/_journal.json` — dropped 0126 entry, appended 0132 - **Added** `packages/db/src/issue-comment-derived-attribution-migration.test.ts` (345 lines) — covers fresh-install, already-0126-migrated idempotency, and partial-backfill completion scenarios using embedded Postgres ## Verification ```bash # Migration numbering guard pnpm --filter @paperclipai/db check:migrations # Migration tests (embedded Postgres, 3 scenarios) pnpm --filter @paperclipai/db vitest run issue-comment-derived-attribution-migration.test.ts ``` Both pass locally. CI results will appear on this PR. ## Risks **Migration safety — already-migrated databases:** Deleting 0126 leaves an orphan row in the runner's applied-set. The runner only checks the set for "has this been applied" — orphan rows are never re-applied. 0132 runs as a near-noop: IF NOT EXISTS DDL is skipped, and the backfill WHERE clause excludes rows that already have attribution. **Migration safety — partially-migrated databases:** Keyset pagination is idempotent. 0132 picks up from the highest attributed row id, so a partial prior run is completed correctly. **No data loss:** The migration never deletes or overwrites user-authored content. It only writes to derived attribution columns on rows where attribution is absent. **Rollback:** 0132 is a forward-only migration. If a rollback is needed, the attribution columns remain (no harm) and can be ignored or cleaned up in a subsequent migration. ## Model Used Claude Sonnet 4.6 (`claude-sonnet-4-6`) via the Paperclip AI agent harness, with tool use and extended context enabled. Implementation authored by Priya Raman; PR opened via the Paperclip Git Expert agent. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details _(branch is the preserved implementation branch from the authoring engineer; the internal task id is present in the branch name by workflow convention — not a content risk)_ - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green _(pending — CI running)_ - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups _(pending — will be driven to terminal-green before merge)_ - [ ] I will address all Greptile and reviewer comments before requesting merge _(pending — will action all findings)_ Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-06 11:41:24 -07:00
DottaandPaperclip	1e81bd188b	Fix heartbeat run responsible user migration for identifier refs (#9107) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip stores heartbeat run ownership context so operators can audit which user was responsible for agent work > - A migration backfills missing heartbeat run `responsible_user_id` values from issue references in each run's context snapshot > - Some context snapshots can store issue identifiers as public ticket strings rather than UUIDs > - The migration needs to resolve both UUID issue ids and issue identifiers without trying to cast identifier strings to UUID > - This pull request tightens the migration query so UUID matching only casts validated UUID-shaped values and identifier matching remains a separate fallback > - A companion repair migration is needed so installations that already recorded `0130` still get the corrected heartbeat-run backfill > - The benefit is that existing installations can apply the responsible-user invariant migrations without failing on non-UUID issue references and without leaving already-migrated databases unrepaired ## Linked Issues or Issue Description No public GitHub issue was found in a quick search for this migration failure. Bug report context following `.github/ISSUE_TEMPLATE/bug_report.yml`: **Pre-submission checklist** - I searched existing open and closed issues and did not find a duplicate. - I can reproduce against current `master` plus the responsible-user invariant migration path. - The error originates in Paperclip's database migration, not in an adapter, API provider, or local configuration. **What happened?** Applying the heartbeat run responsible-user backfill migration could fail when `heartbeat_runs.context_snapshot->>'issueId'` or `taskId` contained an issue identifier such as `PAP-123` instead of a UUID. The migration attempted to use issue refs for UUID matching and identifier fallback, but the UUID path needed to avoid casting non-UUID identifier strings. Because `0130` may already have been applied in some installations, a follow-up repair migration is needed as well. **Expected behavior** The migration should backfill from UUID issue ids when present, from issue identifiers when present, and fall back to the company default responsible user without unsafe UUID casts. Already-migrated installations should receive the repaired heartbeat-run context-ref backfill through a new migration. **Steps to reproduce** 1. Use a migrated database with a company, issue, agent, and heartbeat run. 2. Store a null `heartbeat_runs.responsible_user_id` and a `context_snapshot` like `{"issueId":"PAP-123"}`. 3. Replay/apply the run responsible-user repair migration. 4. Observe that the migration must not cast `PAP-123` to UUID and should backfill from the matching issue identifier. **Paperclip version or commit** Current `master` plus this migration fix branch. **Deployment mode** Database migration during server startup or explicit migration command. **Installation method** Built from source / self-hosted migration path. **Agent adapter(s) involved** Not adapter-specific; core database migration bug. **Database mode** Postgres migration path, including embedded Postgres in development. **Access context** Not applicable; migration-time data backfill. **Relevant logs or output** Unsafe UUID casts can surface as Postgres invalid input syntax errors when a context snapshot issue ref is an identifier rather than a UUID. **Privacy checklist** No private logs, paths, API keys, tokens, company names, or internal Paperclip issue links are included. ## What Changed - Split heartbeat run context issue reference extraction into reusable CTEs. - Only cast `issueId` / `taskId` values to UUID after a UUID-shape regex check. - Preserve fallback matching by issue identifier within the same company. - Keep deterministic candidate priority with `issueId` before `taskId` and UUID matches before identifier matches. - Added `0131_repair_run_responsible_user_context_refs.sql` so installations that already applied `0130` still receive the corrected heartbeat-run backfill. - Added a DB migration regression test that replays the repair migration with an identifier-style heartbeat run issue ref. ## Verification - `pnpm --filter @paperclipai/db typecheck` - `pnpm --filter @paperclipai/db exec vitest run src/client.test.ts` - Isolated embedded Postgres migration run with a temporary `PAPERCLIP_CONFIG`: `pnpm --filter @paperclipai/db migrate` applied all pending migrations successfully. - GitHub Actions PR workflow is green on commit `1c2655bc457cef3716a43e66463e1e5bc2fdcfab`. - Greptile is 5/5 with no unresolved threads on commit `1c2655bc457cef3716a43e66463e1e5bc2fdcfab`. - Searched for duplicate public issues/PRs with GitHub search; no direct duplicate found. - Checked `ROADMAP.md` for overlap; no related roadmap item found. ## Risks - Low-to-medium migration risk because this modifies an existing data backfill migration and adds a companion repair migration. - The query still relies on `context_snapshot` containing either issue UUIDs or identifiers that match issues in the same company. - Installations with unusual malformed context snapshots now skip unsafe UUID casts and fall through to identifier/default backfill behavior. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 Codex coding agent with tool use and local command execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes, or confirmed no docs update is needed for this migration-only fix - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-06 13:14:18 -05:00
3 people	ad961227f5	feat(secrets): add user-specific runtime secrets (#8825) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent runs often need provider credentials, API tokens, and other environment-bound secrets. > - Company-level secrets work for shared credentials, but they do not model values that should differ by human operator. > - Without a user-scoped model, a run can dispatch without knowing whether the responsible human has supplied the needed value. > - Paperclip also needs run attribution to make those user-scoped runtime checks deterministic and auditable. > - This pull request adds user-specific secret definitions, per-user values, environment bindings, responsible-user attribution, and runtime resolution gates. > - The benefit is that teams can define the secret once, let each user provide their own value, and block runs before dispatch when required user secrets or active definitions are unavailable. ## Linked Issues or Issue Description Refs #224 Refs #6057 This PR implements user-specific secret support as a core secret-management capability rather than a one-off adapter setting. It is related to existing public work on company secrets UI and runtime secret refs, but is distinct because the value is owned by the responsible user and resolved at run dispatch time. Related PR search before opening found existing secrets work such as #1550, #8256, #8614, #8634, and #8647; none of those add the full user-secret definition/value/runtime gate covered here. ## What Changed - Added user-secret definitions and per-user "My secrets" values, keeping stored values out of access metadata. - Added `user_secret_ref` environment bindings and UI affordances to pick them alongside existing secret refs. - Added responsible-user runtime resolution so user-secret refs resolve against the human responsible for the run. - Added pre-dispatch missing-secret gates so runs fail before adapter dispatch when required user values are absent or definitions are inactive. - Added low-trust allowlist hardening for user-secret runtime access. - Added issue, routine, run, and agent API key responsible-user attribution and fail-closed dispatch behavior when attribution cannot be resolved. - Added denial-copy mapping so responsible-user authorization failures surface as actionable run outcomes instead of opaque setup failures. - Added OpenAPI documentation for the user-secret routes. - Rebases cleanly on current `master`; migrations were renumbered incrementally as `0128_user_specific_secrets`, `0129_agent_api_key_responsible_user`, and `0130_run_responsible_user_invariant` after upstream `0126`/`0127` migrations. - Removed previously committed local design screenshots so the PR contains code/docs/tests only. ## Verification - PASS: PR head `2527febd106bcf3ca264ca0da7fca491084192d6` is based on `paperclipai/paperclip:master`. - PASS: `git diff --check` - PASS: `git diff --name-only public/master...HEAD | rg '^(pnpm-lock\\.yaml|\\.github/workflows/|screenshots/)' || true` produced no files. - PASS: migration journal audit confirmed unique indexes through `130` with tail entries `0126_issue_comment_derived_attribution`, `0127_environment_custom_images_instance_scoped`, `0128_user_specific_secrets`, `0129_agent_api_key_responsible_user`, and `0130_run_responsible_user_invariant`. - PASS: `pnpm --filter @paperclipai/ui typecheck` - PASS: `pnpm --filter @paperclipai/server typecheck` - PASS: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-responsible-user-invariant.test.ts` - PASS: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-active-run-output-watchdog.test.ts src/__tests__/heartbeat-stale-queue-invalidation.test.ts src/__tests__/heartbeat-workspace-finalize-branch.test.ts src/__tests__/issue-monitor-scheduler.test.ts` - PASS: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-comment-wake-batching.test.ts src/__tests__/heartbeat-retry-scheduling.test.ts src/__tests__/heartbeat-accepted-plan-workspace-refresh.test.ts src/__tests__/heartbeat-plugin-environment.test.ts` - PASS: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/low-trust-red-team-routes.test.ts` - PASS: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/secrets-service.test.ts` (55 tests) - PASS: `pnpm vitest run server/src/__tests__/secrets-routes.test.ts server/src/__tests__/secrets-service.test.ts` (89 tests after final Greptile cleanup fixes) - PASS: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-issue-liveness-escalation.test.ts` (17 tests after the final rebase CI fix) - PASS: focused server Vitest batches covering heartbeat recovery, project env, plugin env, routines, low-trust, pipelines, monitors, watchdog, and stale queue paths. - PASS: GitHub checks are green on `2527febd106bcf3ca264ca0da7fca491084192d6`, including Typecheck + Release Registry, Build, General tests, serialized server suites, e2e, Canary Dry Run, verify, security checks, and Greptile Review. - PASS: Greptile Review completed successfully on `2527febd106bcf3ca264ca0da7fca491084192d6` with Confidence Score 5/5, and GraphQL review-thread audit returned zero unresolved non-outdated threads. ## Risks - Runtime behavior now depends on a run having a correct responsible user; missing or incorrect responsibility assignment can block runs before adapter dispatch. - `user_secret_ref` bindings intentionally expose metadata without values, but UI/API callers may need to handle the new binding kind explicitly. - External secret providers and IAM policies are not automatically provisioned by this PR; operators still need to configure provider-side access for non-local vaults. - The PR is broad across db/shared/server/UI/runtime paths, so release validation should include both API and UI secret workflows before merge. - The migration renumbering is intentionally incremental after upstream migrations; the branch migrations use guarded column/table/index/constraint creation so users who tested the older draft numbering should not hit duplicate DDL for the existing objects. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5-based coding agent (`gpt-5`), Codex local adapter with shell/tool use and code execution. Context window and internal reasoning mode are not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-05 05:58:20 -05:00
Devin FoleyandPaperclip	b4815bf964	Scope environment custom images to instance environments (#8850) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Environments are now managed as instance-level runtime resources rather than per-company rows > - The custom environment image setup tables were introduced with their own `company_id` columns and route query parameters > - That split made one saved environment image state depend on an extra company context even though the environment itself is the durable owner > - It also made saved-environment probes harder because applying the active custom image template could require a company context when no secret-backed config needed one > - This pull request scopes custom image templates and setup sessions directly to the saved environment > - The benefit is that reusable environment images follow the same instance-scoped model as environments while secret resolution still uses company context only when secrets require it ## Linked Issues or Issue Description No matching public GitHub issue was found. Bug report: ### What happened? saved environment custom-image routes and persistence required a `companyId` even though environments are instance-scoped, and saved sandbox probes did not opt into active custom-image template application unless a company context was present. ### Expected behavior custom-image templates and setup sessions should be owned by the saved environment, and saved sandbox probes should apply the active template while still requiring a company context only for secret-backed runtime config. ### Steps to reproduce 1. Configure an instance-scoped sandbox environment with custom-image setup support. 2. Start or inspect a custom-image session or template for that saved environment. 3. Probe the saved environment without a custom-image-specific `companyId` query parameter. ### Paperclip version or commit current `master` after the environment custom-image template migration. ### Deployment mode Local dev (pnpm dev) or authenticated local Paperclip instance. ### Installation method Built from source (pnpm dev / pnpm build). ### Agent adapter(s) involved Not adapter-specific (core bug). ### Database mode Embedded PGlite/Postgres dev database. ### Access context Board human operator. ### Privacy checklist No logs, secrets, tokens, private URLs, or local machine paths are included. Duplicate search performed: - `gh search prs "environment custom image companyId repo:paperclipai/paperclip" --state open --limit 20` - `gh search prs "custom image environment scoped repo:paperclipai/paperclip" --state open --limit 20` - `gh search issues "environment custom image repo:paperclipai/paperclip" --state open --limit 20` The returned results were unrelated adapter, Docker, auth, or stale-workspace items. ## What Changed - Removed redundant `company_id` columns from environment custom-image templates and setup sessions. - Added migration `0127_environment_custom_images_instance_scoped` to collapse duplicate active rows per environment before dropping the old company-scoped indexes/columns. - Updated custom-image services, route handlers, shared validators, and UI API/query keys to use environment-scoped custom-image state. - Kept runtime secret resolution company-aware only when secret refs or bindings require a company context. - Made saved sandbox environment probes opt into active custom-image template application. - Updated DB, shared, server, and UI tests for the new environment-scoped contract. ## Verification - `pnpm --filter @paperclipai/db run check:migrations` - `pnpm exec vitest run packages/db/src/environment-custom-images-schema.test.ts packages/shared/src/environment-custom-images.test.ts server/src/__tests__/environment-custom-image-routes.test.ts server/src/__tests__/environment-custom-images-service.test.ts server/src/__tests__/environment-routes.test.ts ui/src/pages/CompanyEnvironments.test.tsx` - `pnpm -r typecheck` - `pnpm test:run` before rebasing onto latest `master`; after the rebase only the migration number changed, and the migration check plus focused suite, typecheck, and build were rerun. - `pnpm build` ## Risks - Migration safety: the migration supersedes duplicate active templates per environment and fails duplicate active setup sessions before adding environment-only unique indexes. Operators with duplicate historical active rows should review which active template is kept. - Behavior shift: plugin custom-image setup calls now receive `companyId: "instance"` when no secret binding determines a concrete company context. - Secret-backed configs still require an explicit or uniquely inferable company context; environments with secret bindings spread across multiple companies continue to fail fast. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex via the `codex_local` adapter, GPT-5-based coding model with tool-enabled repository inspection, editing, testing, git, and GitHub CLI access. Exact context-window metadata was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-01 23:57:31 -07:00