## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Managed worktree services run isolated Paperclip instances with
cloned databases.
> - A reachable service was reported as ready even when its database,
runtime identity, or login path was not usable.
> - The first candidate added verified database seeding and managed
repair in #11665.
> - This pull request consolidates that candidate with signed login
handoff and a complete readiness contract.
> - Post-QA fixes close five defects in repair identity, repair
responses, UI retry, seed journal handling, and seed-source trust.
> - The benefit is a workspace that either opens safely or reports one
accurate recovery action.
## Linked Issues or Issue Description
No public GitHub issue exists for this work, so the problem is described
here.
**What happened**
Managed workspace URLs could return HTTP 200 and report ready while
login failed. QA also found cases where repair used the wrong instance
identity, returned a generic error, left the UI stuck, rejected a safe
journal lag, or trusted a mutable workspace manifest.
**Expected behavior**
Opening a ready workspace signs the board user in to the correct
isolated instance. Provisioning and repair use a registered source and
report a structured recovery state.
**Actual behavior**
Entry depended on a password copied into the clone. Several failure
paths could publish stale readiness, hide the repair precondition, or
trust state that the workspace could modify.
**Additional context**
This pull request includes the commits first published in #11665. That
pull request keeps the original base head for review history. This
consolidated pull request is the merge candidate. Related open readiness
work includes #11575 and #11621.
## What Changed
- Adds a short-lived, signed, single-use login ticket. It binds the
user, workspace, instance, and runtime origin.
- Exchanges the ticket through Better Auth. It creates the session and
cookie through the supported adapter path.
- Adds protected workspace readiness fields for the database, clone
data, login handoff, seed phase, and runtime identity.
- Fails readiness closed when the guest has no company or
execution-workspace binding.
- Binds ticket issuance to the exact cloned user and active company
membership selected for the handoff.
- Verifies every current active board identity through the exact-user
handoff before publication or reuse.
- Gates managed runtime publication on the readiness contract and the
recorded worktree instance identity.
- Refreshes runtime work products from the live runtime row after a port
change.
- Adds one workspace access card with ready, degraded, repairing, and
failed states.
- Uses the runtime response identity for repair. It returns structured
repair precondition errors.
- Lets a valid source journal lag converge during provisioning.
- Binds seed and repair manifests to a source registered outside the
agent-writable worktree.
- Clears recovered UI errors so a successful retry can open the
workspace.
- Makes runtime tests register canonical sources and avoid ports owned
by live host listeners.
- Keeps Vitest on source suites when compiled `dist` trees exist.
- Isolates CLI and adapter tests from ambient AWS and runtime API
environment variables.
- Preserves a 404 response for cross-company workspace ID lookups before
runtime authorization.
- Makes concurrent single-flight coverage independent of
path-canonicalization scheduling order.
## Verification
The following checks passed on the integrated head:
```sh
pnpm -r typecheck
pnpm build
pnpm check:token-gates
pnpm --filter @paperclipai/db check:migrations
```
- The server source lane passed 420 files and 4,953 tests. Five tests
were skipped.
- The CLI lane passed 57 files and 385 tests.
- The database lane passed 26 files and 97 tests.
- The shared package passed 58 files and 506 tests.
- The adapter utility lane passed 640 tests. Four tests were skipped.
- The Claude adapter passed 220 tests. One test was skipped.
- The Codex adapter passed 323 tests.
- The OpenClaw adapter passed 13 tests.
- The OpenCode adapter passed 42 tests.
- The plugin SDK passed 45 tests.
- The workspace runtime suite passed 124 tests.
- The caller-scoped readiness and handoff suite passed 52 tests.
- The workspace provisioning shell suite passed 7 tests.
- The runtime exposure suite passed 17 tests while live host mappings
occupied fixed test ports.
- `git diff --check` passed and the worktree is clean.
The serialized route lane will run in GitHub CI with its normal shards.
No deployment or active-workspace migration was performed.
## Risks
- This is a medium-risk authentication and runtime-readiness change.
- The login ticket uses exact origin, workspace, instance, and user
binding. It has a short expiry and a one-time nonce.
- Runtime publication is stricter. A real readiness, identity, per-user
handoff, or control-plane database disagreement now blocks publication.
- This pull request supersedes #11665 as the merge candidate. Close
#11665 after this pull request merges.
- No new database migration is included. The lockfile and workflow files
are unchanged.
- Deployment and active-workspace migration are intentionally outside
this pull request.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
Claude Opus 5 (`claude-opus-5[1m]`), 1M context, extended thinking, tool
use, and code execution produced the main candidate. OpenAI GPT-5
(`gpt-5`) through Codex, with agentic reasoning, tool use, and code
execution, integrated the post-QA fixes and hardened the test gates. The
Codex context-window size was not exposed.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The server stores all state in PostgreSQL through Drizzle and the
postgres.js driver
> - Self-hosted installs run Postgres on localhost, so per-query latency
is near zero; hosted installs often attach Postgres over a network,
sometimes through a transaction-mode pooler
> - The DB client passes no options to the driver, so operators cannot
disable prepared statements or tune the pool without a source edit, and
the deploy docs told them to edit `client.ts`
> - The attention feed also runs its related-data lookups one after
another, so its latency grows as queries × network round trip
> - This pull request adds optional environment configuration for the DB
client and batches the independent attention-feed lookups with
`Promise.all`
> - The benefit is that network-attached deployments get correct pooler
support and a much faster attention feed, while self-hosted behavior
does not change
## Linked Issues or Issue Description
No public issue exists for this; description follows the bug report
template:
**What happened?**
On deployments where PostgreSQL is network-attached (managed providers,
pooled endpoints), the attention feed endpoint is slow:
`attentionService.list()` awaits ~15–20 queries strictly in sequence, so
a 70ms round trip turns into more than one second of pure network wait
per call. Separately, connecting through a transaction-mode pooler
(pgbouncer, Supavisor port 6543, Neon `-pooler` hosts) requires
disabling prepared statements, and the only documented way was to
hand-edit `packages/db/src/client.ts` — which `doc/DATABASE.md` itself
tells operators not to do.
**Expected behavior**
The DB client is configurable from the environment (prepared statements,
pool size, timeouts) with driver defaults when unset, and hot read paths
do not multiply network latency by issuing independent queries
sequentially.
**Steps to reproduce**
1. Run the server with `DATABASE_URL` pointing at a Postgres instance
with ~70ms round-trip latency.
2. Open the attention feed (`GET /companies/:companyId/attention`) and
measure response time — it exceeds one second even with little data.
3. Try to connect through a transaction-mode pooler: there is no
supported configuration to disable prepared statements.
## What Changed
- `packages/db/src/client.ts`: `createDb` accepts a
`DatabaseClientOptions` argument and reads optional env config —
`DATABASE_PREPARED_STATEMENTS`, `DATABASE_POOL_MAX`,
`DATABASE_IDLE_TIMEOUT_SECONDS`, `DATABASE_CONNECT_TIMEOUT_SECONDS`.
When nothing is set, no option is passed to the driver and behavior is
identical to the previous bare `postgres(url)`.
- `packages/db/src/client-options.test.ts` (new): env parsing and
driver-option mapping tests, including malformed-value rejection.
- `server/src/services/attention.ts`: the independent related-data
lookups in each feed section now run under `Promise.all` (issue
summary/image/plan-document maps, decision bundle titles, blocked-issue
maps, the newer-runs scan). Section order, item assembly, and query
shapes are unchanged.
- `doc/DATABASE.md` and `docs/deploy/database.md`: the edit-source
pooling instruction is replaced with the env toggle, plus a short
client-tuning reference.
## Verification
- `pnpm --filter @paperclipai/db exec vitest run
src/client-options.test.ts` — 6 tests pass.
- `pnpm --filter server exec vitest run
src/__tests__/attention-service.test.ts` — 22 tests pass.
- `pnpm --filter server exec vitest run
src/__tests__/decisions-service.test.ts
src/__tests__/decision-training.test.ts` — 45 tests pass; this covers
the call path that runs `attentionService.list()` inside
`db.transaction`, where postgres.js serializes queries on the reserved
connection.
- `tsc` reports no errors in the changed files.
## Risks
- Low risk for self-hosted installs: with no env vars set,
`postgres(url, {})` receives an empty options object, which postgres.js
treats the same as no options — driver defaults throughout.
- The `Promise.all` batches only group queries that had no data
dependency on each other; on the transaction call path the driver still
executes them one at a time on the reserved connection, so transactional
semantics are unchanged.
- Malformed env values now fail fast at startup with a clear message
instead of being silently ignored; this is intentional and only affects
operators who set the new variables.
## Model Used
Claude Fable 5 (`claude-fable-5`), Anthropic — via Claude Code CLI,
extended thinking enabled, tool use (test execution, live latency
measurement against a network-attached Postgres to size the problem).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above (searched "prepared statements", "pgbouncer", "pool",
"attention feed", "lockfile" — closest matches are #10573/#10787
lockfile chores, unrelated to this change)
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Governed MCP access spans contracts, runtime enforcement, adapters,
UI surfaces, and operator verification
> - The parity reference PR #9534 is too large for effective automated
or human review
> - The feature therefore needs a linear stack whose individual diffs
stay below the 100-file review limit
> - This pull request is split 2/8 and focuses on database schema and
shared governance contracts
> - The benefit is a standalone, testable review boundary while
preserving byte-for-byte parity at the top of the stack
## Linked Issues or Issue Description
- Related parity reference: #9534
- Problem: The governed access model needs additive persistence and
synchronized shared types before server enforcement can compile.
- Proposed solution: Adds migrations 0148–0169, tool-access and Smoke
Lab schema, shared types/validators/gallery helpers, and the minimal
compile-required contract consumers identified by boundary testing.
- Alternatives considered: keeping #9534 as one 403-file review, or
rewriting the feature to manufacture seams; both were rejected in favor
of path extraction plus compile-driven boundary moves.
- Roadmap alignment: this advances the existing governed MCP/tool-access
work already represented by #9534; it does not introduce a separate
roadmap initiative.
- Stack position: base branch is `pap10341-split/01-demo-servers`.
- Merge policy: merge bottom-up, in order, only after the complete
eight-PR stack has been reviewed and the top-of-stack parity gate
remains empty.
- Requested review: QA for migrations/validators; Greptile on every PR.
## What Changed
- Adds migrations 0148–0169, tool-access and Smoke Lab schema, shared
types/validators/gallery helpers, and the minimal compile-required
contract consumers identified by boundary testing.
- Keeps this PR below 100 changed files and independently typecheckable.
- Preserves the final tree from #9534 when combined with the other seven
stack levels.
## Verification
- `pnpm typecheck` — passed, including migration numbering and safety
checks
- `pnpm --filter @paperclipai/db test` — passed
- `pnpm --filter @paperclipai/shared test` — passed
## Risks
- Migration or contract mistakes could affect every upper layer; all
migrations are additive/idempotent and compile consumers are included in
this boundary.
- Stack risk: merging out of order can expose incomplete layers;
mitigate by following the documented bottom-up merge policy.
- Parity risk: later edits to an intermediate branch can drift from
#9534; mitigate by re-running the empty top-of-stack diff before merge.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex, exact model ID `gpt-5.4`; runtime-managed context
window; medium reasoning with repository, shell, Git, GitHub CLI, and
code-execution tools enabled.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] Internal references are omitted except the execution-plan link
explicitly required for this coordinated split stack
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Stack Coordination
- Internal execution plan:
[PAP-13874](/PAP/issues/PAP-13874#document-plan)
- Parity reference: #9534
- Stack: #9556 → #9557 → #9558 → #9559 → #9560 → #9561 → #9562 → #9563
- Merge bottom-up only after full-stack review and an empty parity diff
at #9563.
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
On macOS, `initdb` defaults to SQL_ASCII encoding because it infers
locale from the system environment. When `ensurePostgresDatabase()`
creates a database without specifying encoding, the new database
inherits SQL_ASCII from the cluster. This causes string functions like
`left()` to operate on bytes instead of characters, producing invalid
UTF-8 when multi-byte characters are truncated.
Two-part fix:
1. Pass `--encoding=UTF8 --locale=C` via `initdbFlags` to all
EmbeddedPostgres constructors so the cluster defaults to UTF-8.
2. Explicitly set `encoding 'UTF8'` in the CREATE DATABASE statement
with `template template0` (required because template1 may already
have a different encoding) and `C` locale for portability.
Existing databases created with SQL_ASCII are NOT automatically fixed;
users must delete their local `data/db` directory and restart to
re-initialize the cluster.
Relates to #636
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The local migrationsFolder variable in migratePostgresIfEmpty duplicated
the module-level MIGRATIONS_FOLDER constant. Reuse the constant to keep
a single source of truth for the migration path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
URL.pathname returns /C:/... on Windows, causing doubled drive letters
when Node prepends the current drive. fileURLToPath handles this
correctly across platforms.
Fixes#132
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduce company_secrets and company_secret_versions tables for
encrypted secret storage. Add EnvBinding discriminated union (plain vs
secret_ref) to replace raw string env values in adapter configs. Add
hiddenAt column to issues for soft-hiding. Improve migration system
with journal-ordered application and manual fallback when Drizzle
migrator can't reconcile history.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add agent_task_sessions table for per-task session state keyed by
(agent, adapter, taskKey). Add AgentTaskSession type, resetAgentSessionSchema
validator, and sessionDisplayId/sessionParamsJson to AgentRuntimeState.
Rework migration hash-resolution fallback ordering to prefer hash-based
matching over timestamp-based journal matching. Move backup-db.sh logic
into packages/db/src/backup.ts for programmatic use and simplify the shell
script to call the TypeScript implementation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add agent_config_revisions table for tracking agent configuration changes with
rollback support. Add issue_approvals junction table linking issues to approvals.
New migrations (0005, 0006) for permissions column and new tables. Rework migration
client with statement-level idempotency checks (table/column/index/constraint
existence) so migrations can be safely retried against partially-migrated databases.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add issuePrefix/issueCounter to companies and issueNumber/identifier to issues
for human-readable issue IDs (e.g. PAP-42). Add runId to activity_log for
linking activity to heartbeat runs. Rework DB client to support migration
state inspection and interactive pending migration prompts at startup.
Add reopen option to issue comments validator.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch from PGlite (WebAssembly) to embedded-postgres for zero-config
local development — provides a real PostgreSQL server with full
compatibility. Add startup banner with config summary on server boot.
Improve server bootstrap with auto port detection, database creation,
and migration on startup. Update DATABASE.md, DEVELOPING.md, and
SPEC-implementation.md to reflect the change. Update CLI database
check and prompts. Simplify OnboardingWizard database options.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add @electric-sql/pglite so the server can run without an external
Postgres instance. When DATABASE_URL is not set, the server auto-creates
an embedded PGlite database in ./data/pglite with schema push on startup.
- Add createPgliteDb() alongside the existing createDb()
- Make DATABASE_URL optional in server config
- Update drizzle config to glob schema files
- Update migrate script to support both Postgres and PGlite
- Add data/ to .gitignore for local PGlite storage
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Postgres schema via Drizzle ORM for agents, goals, issues, projects,
and activity log tables. Includes migration runner, seed script,
and Drizzle config.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
