fix: fail closed on unusable Codex auth during sandbox auth merge

When reusing a Codex sandbox, the workspace-restore step now performs
an auth-merge check before adopting the sandbox's auth.json. The merge
fails closed when:

- host or sandbox Codex auth is unusable (missing token or account)
- the subscription payload shapes differ between host and sandbox
- subscription account identities do not match

Additional hardening: node absence on PATH during the embedded decision
script now exits non-zero instead of silently falling through. Co-change
comments link parseAuth (the embedded script) to hasUsableAuthPayload
(codex-home.ts) so a future auth-format change cannot silently diverge
between the two implementations.

Co-Authored-By: Paperclip <noreply@paperclip.ing>

docs/deploy/aws-ecs.md

@@ -0,0 +1,580 @@
+---
+title: AWS ECS Fargate
+summary: Deploy Paperclip to AWS using ECS Fargate, RDS Postgres, and EFS
+---
+
+Deploy Paperclip to AWS with ECS Fargate (compute), RDS Postgres 17 (database), and EFS (persistent storage). This guide uses the AWS CLI and produces a single-task ECS service behind an ALB with HTTPS.
+
+## Prerequisites
+
+- AWS CLI v2 configured with a profile that has admin-level permissions
+- Docker installed locally (for building and pushing the image)
+- A registered domain with DNS you control (for the TLS certificate)
+- The Paperclip repo cloned locally
+
+Set these shell variables for the rest of the guide:
+
+```bash
+export AWS_REGION=us-east-1
+export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
+export PAPERCLIP_DOMAIN=paperclip.example.com   # your domain
+export DB_PASSWORD=$(openssl rand -base64 24 | tr -d '/+=' | head -c 32)
+export AUTH_SECRET=$(openssl rand -base64 32)
+```
+
+## 1. Create ECR Repository
+
+```bash
+aws ecr create-repository \
+  --repository-name paperclip-server \
+  --image-scanning-configuration scanOnPush=true \
+  --region $AWS_REGION
+```
+
+## 2. Build and Push Docker Image
+
+```bash
+cd /path/to/paperclip
+
+# Authenticate Docker to ECR
+aws ecr get-login-password --region $AWS_REGION \
+  | docker login --username AWS --password-stdin \
+    $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com
+
+# Build
+docker build -t paperclip-server .
+
+# Tag and push
+docker tag paperclip-server:latest \
+  $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/paperclip-server:latest
+
+docker push \
+  $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/paperclip-server:latest
+```
+
+## 3. Networking (VPC, Subnets, Security Groups)
+
+Use the default VPC or create a dedicated one. The guide assumes the default VPC with public and private subnets in two AZs.
+
+```bash
+# Get default VPC
+VPC_ID=$(aws ec2 describe-vpcs \
+  --filters Name=isDefault,Values=true \
+  --query 'Vpcs[0].VpcId' --output text)
+
+# Get two public subnets (for ALB)
+SUBNET_IDS=$(aws ec2 describe-subnets \
+  --filters Name=vpc-id,Values=$VPC_ID \
+  --query 'Subnets[?MapPublicIpOnLaunch==`true`] | [0:2].SubnetId' \
+  --output text)
+SUBNET_1=$(echo $SUBNET_IDS | awk '{print $1}')
+SUBNET_2=$(echo $SUBNET_IDS | awk '{print $2}')
+```
+
+Create security groups:
+
+```bash
+# ALB security group — inbound HTTPS
+ALB_SG=$(aws ec2 create-security-group \
+  --group-name paperclip-alb \
+  --description "Paperclip ALB" \
+  --vpc-id $VPC_ID \
+  --query 'GroupId' --output text)
+
+aws ec2 authorize-security-group-ingress \
+  --group-id $ALB_SG \
+  --protocol tcp --port 443 --cidr 0.0.0.0/0
+
+# Also open port 80 so the ALB can accept HTTP and redirect to HTTPS
+aws ec2 authorize-security-group-ingress \
+  --group-id $ALB_SG \
+  --protocol tcp --port 80 --cidr 0.0.0.0/0
+
+# ECS task security group — inbound from ALB only
+ECS_SG=$(aws ec2 create-security-group \
+  --group-name paperclip-ecs \
+  --description "Paperclip ECS tasks" \
+  --vpc-id $VPC_ID \
+  --query 'GroupId' --output text)
+
+aws ec2 authorize-security-group-ingress \
+  --group-id $ECS_SG \
+  --protocol tcp --port 3100 \
+  --source-group $ALB_SG
+
+# RDS security group — inbound from ECS only
+RDS_SG=$(aws ec2 create-security-group \
+  --group-name paperclip-rds \
+  --description "Paperclip RDS" \
+  --vpc-id $VPC_ID \
+  --query 'GroupId' --output text)
+
+aws ec2 authorize-security-group-ingress \
+  --group-id $RDS_SG \
+  --protocol tcp --port 5432 \
+  --source-group $ECS_SG
+
+# EFS security group — inbound NFS from ECS only
+EFS_SG=$(aws ec2 create-security-group \
+  --group-name paperclip-efs \
+  --description "Paperclip EFS" \
+  --vpc-id $VPC_ID \
+  --query 'GroupId' --output text)
+
+aws ec2 authorize-security-group-ingress \
+  --group-id $EFS_SG \
+  --protocol tcp --port 2049 \
+  --source-group $ECS_SG
+```
+
+## 4. Create RDS Postgres Instance
+
+```bash
+# Custom VPCs don't come with a default DB subnet group — create one
+# that spans our two subnets so RDS can place the instance.
+aws rds create-db-subnet-group \
+  --db-subnet-group-name paperclip-db-subnet \
+  --db-subnet-group-description "Paperclip RDS subnets" \
+  --subnet-ids $SUBNET_1 $SUBNET_2
+
+aws rds create-db-instance \
+  --db-instance-identifier paperclip-db \
+  --db-instance-class db.t4g.micro \
+  --engine postgres \
+  --engine-version 17 \
+  --master-username paperclip \
+  --master-user-password "$DB_PASSWORD" \
+  --allocated-storage 20 \
+  --storage-type gp3 \
+  --vpc-security-group-ids $RDS_SG \
+  --db-subnet-group-name paperclip-db-subnet \
+  --no-publicly-accessible \
+  --backup-retention-period 7 \
+  --no-multi-az \
+  --db-name paperclip \
+  --region $AWS_REGION
+
+# Wait for it to become available (takes 5-10 min)
+aws rds wait db-instance-available \
+  --db-instance-identifier paperclip-db
+
+# Get the endpoint
+RDS_ENDPOINT=$(aws rds describe-db-instances \
+  --db-instance-identifier paperclip-db \
+  --query 'DBInstances[0].Endpoint.Address' --output text)
+
+DATABASE_URL="postgresql://paperclip:${DB_PASSWORD}@${RDS_ENDPOINT}:5432/paperclip"
+```
+
+## 5. Create EFS Filesystem
+
+```bash
+EFS_ID=$(aws efs create-file-system \
+  --performance-mode generalPurpose \
+  --throughput-mode bursting \
+  --encrypted \
+  --tags Key=Name,Value=paperclip-data \
+  --query 'FileSystemId' --output text)
+
+# Create mount targets in each subnet
+for SUBNET in $SUBNET_1 $SUBNET_2; do
+  aws efs create-mount-target \
+    --file-system-id $EFS_ID \
+    --subnet-id $SUBNET \
+    --security-groups $EFS_SG
+done
+
+# Wait for mount targets
+aws efs describe-mount-targets --file-system-id $EFS_ID
+```
+
+## 6. Store Secrets
+
+```bash
+aws secretsmanager create-secret \
+  --name paperclip/database-url \
+  --secret-string "$DATABASE_URL"
+
+aws secretsmanager create-secret \
+  --name paperclip/anthropic-api-key \
+  --secret-string "YOUR_ANTHROPIC_KEY"
+
+aws secretsmanager create-secret \
+  --name paperclip/better-auth-secret \
+  --secret-string "$AUTH_SECRET"
+
+aws secretsmanager create-secret \
+  --name paperclip/openai-api-key \
+  --secret-string "YOUR_OPENAI_KEY"
+
+aws secretsmanager create-secret \
+  --name paperclip/github-token \
+  --secret-string "YOUR_GITHUB_PAT"
+```
+
+## 7. IAM Roles
+
+Create the ECS task execution role (pulls images, reads secrets) and the task role (application permissions).
+
+```bash
+# Task execution role
+aws iam create-role \
+  --role-name paperclip-ecs-execution \
+  --assume-role-policy-document '{
+    "Version": "2012-10-17",
+    "Statement": [{
+      "Effect": "Allow",
+      "Principal": {"Service": "ecs-tasks.amazonaws.com"},
+      "Action": "sts:AssumeRole"
+    }]
+  }'
+
+aws iam attach-role-policy \
+  --role-name paperclip-ecs-execution \
+  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
+
+# Allow reading secrets
+aws iam put-role-policy \
+  --role-name paperclip-ecs-execution \
+  --policy-name SecretsAccess \
+  --policy-document '{
+    "Version": "2012-10-17",
+    "Statement": [{
+      "Effect": "Allow",
+      "Action": ["secretsmanager:GetSecretValue"],
+      "Resource": "arn:aws:secretsmanager:'$AWS_REGION':'$AWS_ACCOUNT_ID':secret:paperclip/*"
+    }]
+  }'
+
+# Task role (application — add permissions as needed)
+aws iam create-role \
+  --role-name paperclip-ecs-task \
+  --assume-role-policy-document '{
+    "Version": "2012-10-17",
+    "Statement": [{
+      "Effect": "Allow",
+      "Principal": {"Service": "ecs-tasks.amazonaws.com"},
+      "Action": "sts:AssumeRole"
+    }]
+  }'
+```
+
+## 8. ECS Cluster and Task Definition
+
+```bash
+aws ecs create-cluster --cluster-name paperclip
+
+aws logs create-log-group --log-group-name /ecs/paperclip
+```
+
+Register the task definition using the template at `docker/ecs-task-definition.json`. Before registering, replace the placeholder values:
+
+```bash
+sed -e "s|<ACCOUNT_ID>|$AWS_ACCOUNT_ID|g" \
+    -e "s|<REGION>|$AWS_REGION|g" \
+    -e "s|<EFS_ID>|$EFS_ID|g" \
+    -e "s|<DOMAIN>|$PAPERCLIP_DOMAIN|g" \
+    docker/ecs-task-definition.json > /tmp/paperclip-task-def.json
+
+aws ecs register-task-definition \
+  --cli-input-json file:///tmp/paperclip-task-def.json
+```
+
+## 9. ALB and TLS Certificate
+
+Request a certificate (you must validate via DNS):
+
+```bash
+CERT_ARN=$(aws acm request-certificate \
+  --domain-name $PAPERCLIP_DOMAIN \
+  --validation-method DNS \
+  --query 'CertificateArn' --output text)
+
+# Get the CNAME record to add to your DNS
+aws acm describe-certificate \
+  --certificate-arn $CERT_ARN \
+  --query 'Certificate.DomainValidationOptions[0].ResourceRecord'
+```
+
+Add the CNAME to your DNS provider, then wait for validation:
+
+```bash
+aws acm wait certificate-validated --certificate-arn $CERT_ARN
+```
+
+Create the ALB:
+
+```bash
+ALB_ARN=$(aws elbv2 create-load-balancer \
+  --name paperclip-alb \
+  --subnets $SUBNET_1 $SUBNET_2 \
+  --security-groups $ALB_SG \
+  --scheme internet-facing \
+  --type application \
+  --query 'LoadBalancers[0].LoadBalancerArn' --output text)
+
+ALB_DNS=$(aws elbv2 describe-load-balancers \
+  --load-balancer-arns $ALB_ARN \
+  --query 'LoadBalancers[0].DNSName' --output text)
+
+# Target group
+TG_ARN=$(aws elbv2 create-target-group \
+  --name paperclip-tg \
+  --protocol HTTP \
+  --port 3100 \
+  --vpc-id $VPC_ID \
+  --target-type ip \
+  --health-check-path /api/health \
+  --health-check-interval-seconds 30 \
+  --healthy-threshold-count 2 \
+  --unhealthy-threshold-count 3 \
+  --query 'TargetGroups[0].TargetGroupArn' --output text)
+
+# HTTPS listener
+LISTENER_ARN=$(aws elbv2 create-listener \
+  --load-balancer-arn $ALB_ARN \
+  --protocol HTTPS \
+  --port 443 \
+  --certificates CertificateArn=$CERT_ARN \
+  --default-actions Type=forward,TargetGroupArn=$TG_ARN \
+  --query 'Listeners[0].ListenerArn' --output text)
+
+# HTTP listener — redirect all :80 traffic to :443
+HTTP_LISTENER_ARN=$(aws elbv2 create-listener \
+  --load-balancer-arn $ALB_ARN \
+  --protocol HTTP \
+  --port 80 \
+  --default-actions Type=redirect,RedirectConfig='{Protocol=HTTPS,Port=443,StatusCode=HTTP_301}' \
+  --query 'Listeners[0].ListenerArn' --output text)
+```
+
+Point your DNS to the ALB:
+- Create a CNAME or ALIAS record for `$PAPERCLIP_DOMAIN` -> `$ALB_DNS`
+
+## 10. Create ECS Service
+
+```bash
+aws ecs create-service \
+  --cluster paperclip \
+  --service-name paperclip-server \
+  --task-definition paperclip-server \
+  --desired-count 1 \
+  --launch-type FARGATE \
+  --deployment-configuration '{
+    "deploymentCircuitBreaker": {"enable": true, "rollback": true},
+    "maximumPercent": 200,
+    "minimumHealthyPercent": 100
+  }' \
+  --network-configuration '{
+    "awsvpcConfiguration": {
+      "subnets": ["'$SUBNET_1'", "'$SUBNET_2'"],
+      "securityGroups": ["'$ECS_SG'"],
+      "assignPublicIp": "ENABLED"
+    }
+  }' \
+  --load-balancers '[{
+    "targetGroupArn": "'$TG_ARN'",
+    "containerName": "paperclip-server",
+    "containerPort": 3100
+  }]'
+```
+
+> **Note:** `assignPublicIp: ENABLED` is needed if using public subnets without a NAT Gateway. For private subnets, set to `DISABLED` and ensure a NAT Gateway is configured for outbound internet access.
+
+## 11. Verify Deployment
+
+```bash
+# Watch task come up
+aws ecs describe-services \
+  --cluster paperclip \
+  --services paperclip-server \
+  --query 'services[0].{desired:desiredCount,running:runningCount,status:status}'
+
+# Check task health
+aws ecs list-tasks --cluster paperclip --service-name paperclip-server
+TASK_ARN=$(aws ecs list-tasks --cluster paperclip --service-name paperclip-server --query 'taskArns[0]' --output text)
+aws ecs describe-tasks --cluster paperclip --tasks $TASK_ARN \
+  --query 'tasks[0].{status:lastStatus,health:healthStatus}'
+
+# Check logs
+aws logs tail /ecs/paperclip --since 10m --follow
+
+# Hit the health endpoint
+curl -sf https://$PAPERCLIP_DOMAIN/api/health
+```
+
+**Healthy indicators:**
+- ECS task status: `RUNNING`, health: `HEALTHY`
+- Logs show `plugin job coordinator started` and `plugin-loader: loadAll complete`
+- `/api/health` returns 200
+
+## Post-Deploy Security Hardening
+
+After the first user has signed up (which grants admin role), lock down the instance:
+
+```bash
+# Disable public sign-up (prevents unauthorized users from creating accounts)
+# Add to the task definition environment section, then redeploy:
+#   { "name": "PAPERCLIP_AUTH_DISABLE_SIGN_UP", "value": "true" }
+
+# Or update via Secrets Manager / task def override, then force new deployment
+aws ecs update-service \
+  --cluster paperclip \
+  --service paperclip-server \
+  --force-new-deployment
+```
+
+Use the invite flow (added in v2026.416.0) to grant access to additional users after sign-up is disabled.
+
+## Deploying Updates
+
+Build, push, and force a new deployment:
+
+```bash
+# Build and push new image
+docker build -t paperclip-server .
+docker tag paperclip-server:latest \
+  $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/paperclip-server:latest
+docker push \
+  $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/paperclip-server:latest
+
+# Roll out
+aws ecs update-service \
+  --cluster paperclip \
+  --service paperclip-server \
+  --force-new-deployment
+
+# Watch the deployment
+aws ecs describe-services \
+  --cluster paperclip \
+  --services paperclip-server \
+  --query 'services[0].deployments[*].{status:status,running:runningCount,desired:desiredCount,rollout:rolloutState}'
+```
+
+ECS performs a rolling update: starts a new task, waits for it to pass health checks, then drains the old task.
+
+## Rollback
+
+If the new deployment is unhealthy:
+
+```bash
+# ECS automatically rolls back if the new task fails health checks
+# (circuit breaker is enabled in the service configuration above).
+# To force rollback manually:
+
+# 1. Find the previous task definition revision
+aws ecs list-task-definitions \
+  --family-prefix paperclip-server \
+  --sort DESC \
+  --query 'taskDefinitionArns[0:3]'
+
+# 2. Update service to the previous revision
+aws ecs update-service \
+  --cluster paperclip \
+  --service paperclip-server \
+  --task-definition paperclip-server:<PREVIOUS_REVISION>
+```
+
+## Scaling to Zero (Cost Savings)
+
+Scale down when not in use:
+
+```bash
+# Stop
+aws ecs update-service \
+  --cluster paperclip \
+  --service paperclip-server \
+  --desired-count 0
+
+# Start
+aws ecs update-service \
+  --cluster paperclip \
+  --service paperclip-server \
+  --desired-count 1
+```
+
+RDS can also be stopped (auto-restarts after 7 days):
+
+```bash
+aws rds stop-db-instance --db-instance-identifier paperclip-db
+aws rds start-db-instance --db-instance-identifier paperclip-db
+```
+
+## Teardown
+
+Remove all resources in reverse order:
+
+```bash
+# 1. ECS service and cluster
+aws ecs update-service --cluster paperclip --service paperclip-server --desired-count 0
+aws ecs delete-service --cluster paperclip --service paperclip-server --force
+aws ecs delete-cluster --cluster paperclip
+
+# 2. ALB and ACM cert
+aws elbv2 delete-listener --listener-arn $HTTP_LISTENER_ARN
+aws elbv2 delete-listener --listener-arn $LISTENER_ARN
+aws elbv2 delete-target-group --target-group-arn $TG_ARN
+aws elbv2 delete-load-balancer --load-balancer-arn $ALB_ARN
+aws acm delete-certificate --certificate-arn $CERT_ARN
+
+# 3. RDS (creates final snapshot)
+aws rds delete-db-instance \
+  --db-instance-identifier paperclip-db \
+  --final-db-snapshot-identifier paperclip-db-final
+aws rds wait db-instance-deleted --db-instance-identifier paperclip-db
+aws rds delete-db-subnet-group --db-subnet-group-name paperclip-db-subnet
+
+# 4. EFS (mount targets must be deleted first)
+for MT in $(aws efs describe-mount-targets --file-system-id $EFS_ID --query 'MountTargets[*].MountTargetId' --output text); do
+  aws efs delete-mount-target --mount-target-id $MT
+done
+# Mount-target deletion is async; poll until none remain before deleting
+# the filesystem, otherwise delete-file-system fails with FileSystemInUse.
+echo "Waiting for mount targets to delete..."
+while aws efs describe-mount-targets \
+  --file-system-id $EFS_ID \
+  --query 'MountTargets[0].MountTargetId' --output text 2>/dev/null | grep -q 'fsmt-'; do
+  sleep 5
+done
+aws efs delete-file-system --file-system-id $EFS_ID
+
+# 5. Secrets
+for s in database-url anthropic-api-key better-auth-secret openai-api-key github-token; do
+  aws secretsmanager delete-secret --secret-id paperclip/$s --force-delete-without-recovery
+done
+
+# 6. Security groups (after all dependents are gone)
+for sg in $EFS_SG $RDS_SG $ECS_SG $ALB_SG; do
+  aws ec2 delete-security-group --group-id $sg
+done
+
+# 7. ECR
+aws ecr delete-repository --repository-name paperclip-server --force
+
+# 8. IAM roles
+aws iam delete-role-policy --role-name paperclip-ecs-execution --policy-name SecretsAccess
+aws iam detach-role-policy --role-name paperclip-ecs-execution \
+  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
+aws iam delete-role --role-name paperclip-ecs-execution
+aws iam delete-role --role-name paperclip-ecs-task
+
+# 9. Log group
+aws logs delete-log-group --log-group-name /ecs/paperclip
+```
+
+## Cost Reference
+
+| Service | Config | Monthly |
+|---------|--------|---------|
+| ECS Fargate | 2 vCPU, 4 GB, 24/7 | ~$70 |
+| RDS Postgres | db.t4g.micro, 20 GB | ~$15 |
+| ALB | 1 LCU average | ~$22 |
+| NAT Gateway | 1 AZ (if using private subnets) | ~$35 |
+| EFS | 1 GB Standard | ~$0.30 |
+| Secrets Manager | 5 secrets | ~$2 |
+| CloudWatch Logs | ~1 GB/mo | ~$0.50 |
+| ECR | ~1 GB | ~$0.10 |
+| **Total (public subnets, no NAT)** | | **~$110/mo** |
+| **Total (private subnets + NAT)** | | **~$145/mo** |
+
+Use Fargate Spot and scheduled scaling to 0 during off-hours to reduce to ~$60-85/mo.

docs/deploy/database.md

@@ -0,0 +1,77 @@
+---
+title: Database
+summary: Embedded PGlite vs Docker Postgres vs hosted
+---
+
+Paperclip uses PostgreSQL via Drizzle ORM. There are three ways to run the database.
+
+## 1. Embedded PostgreSQL (Default)
+
+Zero config. If you don't set `DATABASE_URL`, the server starts an embedded PostgreSQL instance automatically.
+
+```sh
+pnpm dev
+```
+
+On first start, the server:
+
+1. Creates `~/.paperclip/instances/default/db/` for storage
+2. Ensures the `paperclip` database exists
+3. Runs migrations automatically
+4. Starts serving requests
+
+Data persists across restarts. To reset: `rm -rf ~/.paperclip/instances/default/db`.
+
+The Docker quickstart also uses embedded PostgreSQL by default.
+
+## 2. Local PostgreSQL (Docker)
+
+For a full PostgreSQL server locally:
+
+```sh
+docker compose up -d
+```
+
+This starts PostgreSQL 17 on `localhost:5432`. Set the connection string:
+
+```sh
+cp .env.example .env
+# DATABASE_URL=postgres://paperclip:paperclip@localhost:5432/paperclip
+```
+
+Push the schema:
+
+```sh
+DATABASE_URL=postgres://paperclip:paperclip@localhost:5432/paperclip \
+  npx drizzle-kit push
+```
+
+## 3. Hosted PostgreSQL (Supabase)
+
+For production, use a hosted provider like [Supabase](https://supabase.com/).
+
+1. Create a project at [database.new](https://database.new)
+2. Copy the connection string from Project Settings > Database
+3. Set `DATABASE_URL` in your `.env`
+
+Use the **direct connection** (port 5432) for migrations and the **pooled connection** (port 6543) for the application.
+
+If using connection pooling, disable prepared statements:
+
+```ts
+// packages/db/src/client.ts
+export function createDb(url: string) {
+  const sql = postgres(url, { prepare: false });
+  return drizzlePg(sql, { schema });
+}
+```
+
+## Switching Between Modes
+
+| `DATABASE_URL` | Mode |
+|----------------|------|
+| Not set | Embedded PostgreSQL |
+| `postgres://...localhost...` | Local Docker PostgreSQL |
+| `postgres://...supabase.com...` | Hosted Supabase |
+
+The Drizzle schema (`packages/db/src/schema/`) is the same regardless of mode.

docs/deploy/deployment-modes.md

@@ -0,0 +1,88 @@
+---
+title: Deployment Modes
+summary: local_trusted vs authenticated (private/public)
+---
+
+Paperclip supports two runtime modes with different security profiles. Reachability is configured separately with `bind`.
+
+## `local_trusted`
+
+The default mode. Optimized for single-operator local use.
+
+- **Host binding**: loopback only (localhost)
+- **Bind**: `loopback`
+- **Authentication**: no login required
+- **Use case**: local development, solo experimentation
+- **Board identity**: auto-created local board user
+
+```sh
+# Set during onboard
+pnpm paperclipai onboard
+# Choose "local_trusted"
+```
+
+## `authenticated`
+
+Login required. Supports two exposure policies.
+
+### `authenticated` + `private`
+
+For private network access (Tailscale, VPN, LAN).
+
+- **Authentication**: login required via Better Auth
+- **URL handling**: auto base URL mode (lower friction)
+- **Host trust**: private-host trust policy required
+- **Bind**: choose `loopback`, `lan`, `tailnet`, or `custom`
+
+```sh
+pnpm paperclipai onboard
+# Choose "authenticated" -> "private"
+```
+
+Allow custom Tailscale hostnames:
+
+```sh
+pnpm paperclipai allowed-hostname my-machine
+```
+
+### `authenticated` + `public`
+
+For internet-facing deployment.
+
+- **Authentication**: login required
+- **URL**: explicit public URL required
+- **Security**: stricter deployment checks in doctor
+- **Bind**: usually `loopback` behind a reverse proxy; `lan/custom` is advanced
+
+```sh
+pnpm paperclipai onboard
+# Choose "authenticated" -> "public"
+```
+
+## Board Claim Flow
+
+When migrating from `local_trusted` to `authenticated`, Paperclip emits a one-time claim URL at startup:
+
+```
+/board-claim/<token>?code=<code>
+```
+
+A signed-in user visits this URL to claim board ownership. This:
+
+- Promotes the current user to instance admin
+- Demotes the auto-created local board admin
+- Ensures active company membership for the claiming user
+
+## Changing Modes
+
+Update the deployment mode:
+
+```sh
+pnpm paperclipai configure --section server
+```
+
+Runtime override via environment variable:
+
+```sh
+PAPERCLIP_DEPLOYMENT_MODE=authenticated PAPERCLIP_BIND=lan pnpm paperclipai run
+```

docs/deploy/dev-plane-restart-hygiene.md

@@ -0,0 +1,57 @@
+# Dev-Plane Deploy & Restart Hygiene
+
+Operational runbook for restarting a dev/shared Paperclip control plane without killing in-flight agent work. Written after the 2026-07-06/07 failure spike, where a restart-heavy deploy window was the single largest source of failed tasks.
+
+## Why this matters
+
+Every control-plane restart hard-kills any heartbeat run in flight at that moment. The run is finalized as `failed` with `error_code = 'process_lost'`. There is an automatic single retry (`process_lost_retry`), but it is best-effort: in the 07-06/07 incident window only 12 of 36 lost runs recovered. On the evening of 07-06, **9 restarts in 90 minutes killed 16 in-flight runs**.
+
+## Rules of thumb
+
+1. **Batch your deploys.** Stack up changes and restart once, instead of restart-per-change. A restart storm (several restarts within an hour) multiplies run loss for near-zero benefit.
+2. **Restart during low fleet activity.** Check for active runs before restarting; prefer windows where the fleet is quiet. A quick check against the control-plane DB:
+
+   ```sql
+   SELECT count(*) FROM heartbeat_runs WHERE status = 'running';
+   ```
+
+3. **Drain before restart (upcoming).** Graceful SIGTERM drain — stop accepting new runs, let in-flight runs finish or checkpoint, then exit — is being added in PAP-12930. Once it lands, send SIGTERM and wait for drain instead of hard-restarting. Until then, rule 2 is your drain.
+4. **After any restart, glance at the damage.** See the detection queries below; confirm lost runs either retried successfully or get manual follow-up.
+
+## How to spot a restart burst
+
+Two signals, cross-referenced:
+
+**1. Server start markers in the instance log.** Each boot logs `Server listening on <host>:<port>`. Logs live at `~/.paperclip/instances/<instance>/server.log`, rotated daily to `server.log-YYYYMMDD.gz`.
+
+```bash
+grep -h "Server listening" ~/.paperclip/instances/default/server.log
+zgrep -h "Server listening" ~/.paperclip/instances/default/server.log-20260706.gz
+```
+
+Many markers minutes apart = restart burst.
+
+**2. Same-minute `process_lost` clusters in `heartbeat_runs`.** Runs killed by a restart are finalized together, so they cluster on the same minute:
+
+```sql
+SELECT date_trunc('minute', finished_at) AS minute, count(*)
+FROM heartbeat_runs
+WHERE error_code = 'process_lost'
+GROUP BY 1 HAVING count(*) > 1
+ORDER BY 1 DESC;
+```
+
+If the cluster minutes line up with the `Server listening` timestamps, the failures are restart-inflicted, not a product regression.
+
+**Note:** query the DB directly for this — the `/heartbeat-runs` list API ignores the `status=` filter and omits error fields.
+
+## Checking recovery after a burst
+
+Each `process_lost` run should have triggered one retry wake (`reason = 'process_lost_retry'`). To find lost runs that never recovered, look for `process_lost` failures with no subsequent successful run for the same issue, and re-wake or reassign those issues manually.
+
+## Related failure modes (not restart-caused)
+
+Seen in the same incident window; do not confuse them with restart damage:
+
+- `workspace_validation_failed` — deterministic workspace validation retry loops (self-heal: PAP-12931).
+- Provider quota exhaustion — dominant cause of `claude_transient_upstream` failures (quota-aware handling: PAP-12932).

docs/deploy/docker.md

@@ -0,0 +1,80 @@
+---
+title: Docker
+summary: Docker Compose quickstart
+---
+
+Run Paperclip in Docker without installing Node or pnpm locally.
+
+## Compose Quickstart (Recommended)
+
+```sh
+docker compose -f docker/docker-compose.quickstart.yml up --build
+```
+
+Open [http://localhost:3100](http://localhost:3100).
+
+Defaults:
+
+- Host port: `3100`
+- Data directory: `./data/docker-paperclip`
+
+Override with environment variables:
+
+```sh
+PAPERCLIP_PORT=3200 PAPERCLIP_DATA_DIR=../data/pc \
+  docker compose -f docker/docker-compose.quickstart.yml up --build
+```
+
+**Note:** `PAPERCLIP_DATA_DIR` is resolved relative to the compose file (`docker/`), so `../data/pc` maps to `data/pc` in the project root.
+
+## Manual Docker Build
+
+```sh
+docker build -t paperclip-local .
+docker run --name paperclip \
+  -p 3100:3100 \
+  -e HOST=0.0.0.0 \
+  -e PAPERCLIP_HOME=/paperclip \
+  -v "$(pwd)/data/docker-paperclip:/paperclip" \
+  paperclip-local
+```
+
+## Data Persistence
+
+All data is persisted under the bind mount (`./data/docker-paperclip`):
+
+- Embedded PostgreSQL data
+- Uploaded assets
+- Local secrets key
+- Agent workspace data
+
+## Local Adapter CLIs in Docker
+
+The Docker image pre-installs these agent CLIs so their `*_local` adapters can run inside the container:
+
+- `claude` (Anthropic Claude Code CLI) — `claude_local`
+- `codex` (OpenAI Codex CLI) — `codex_local`
+- `opencode` (OpenCode multi-provider CLI) — `opencode_local`
+- `gemini` (Google Gemini CLI) — `gemini_local` (experimental)
+
+Pass API keys to enable local adapter runs inside the container:
+
+```sh
+docker run --name paperclip \
+  -p 3100:3100 \
+  -e HOST=0.0.0.0 \
+  -e PAPERCLIP_HOME=/paperclip \
+  -e OPENAI_API_KEY=sk-... \
+  -e ANTHROPIC_API_KEY=sk-... \
+  -e GEMINI_API_KEY=... \
+  -v "$(pwd)/data/docker-paperclip:/paperclip" \
+  paperclip-local
+```
+
+Each adapter reads its provider's standard credentials — for example `ANTHROPIC_API_KEY` (Claude), `OPENAI_API_KEY` (Codex), and `GEMINI_API_KEY` or `GOOGLE_API_KEY` (Gemini). OpenCode is multi-provider and uses whichever provider key you supply.
+
+> **Gemini key restrictions:** Google requires Gemini API keys to be *restricted* to the Gemini API (scoped in the Google Cloud console); unrestricted keys are blocked and `gemini_local` runs will fail with an auth error. Create a restricted key, or authenticate with `gemini auth login` (OAuth) and persist `~/.gemini` via the data volume so the credential survives container restarts.
+
+The image sets `GEMINI_SANDBOX=false` so the Gemini CLI does not try to launch its own (Docker-in-Docker) sandbox inside the container. The `gemini_local` adapter already passes `--sandbox=none` per run, so this env var only matters if you invoke `gemini` manually inside the container; override it if you have nested-container support and want CLI-level sandboxing.
+
+Without API keys, the app runs normally — adapter environment checks will surface missing prerequisites.

docs/deploy/environment-variables.md

@@ -0,0 +1,54 @@
+---
+title: Environment Variables
+summary: Full environment variable reference
+---
+
+All environment variables that Paperclip uses for server configuration.
+
+## Server Configuration
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `PORT` | `3100` | Server port |
+| `PAPERCLIP_BIND` | `loopback` | Reachability preset: `loopback`, `lan`, `tailnet`, or `custom` |
+| `PAPERCLIP_BIND_HOST` | (unset) | Required when `PAPERCLIP_BIND=custom` |
+| `HOST` | `127.0.0.1` | Legacy host override; prefer `PAPERCLIP_BIND` for new setups |
+| `DATABASE_URL` | (embedded) | PostgreSQL connection string |
+| `PAPERCLIP_HOME` | `~/.paperclip` | Base directory for all Paperclip data |
+| `PAPERCLIP_INSTANCE_ID` | `default` | Instance identifier (for multiple local instances) |
+| `PAPERCLIP_DEPLOYMENT_MODE` | `local_trusted` | Runtime mode override |
+| `PAPERCLIP_DEPLOYMENT_EXPOSURE` | `private` | Exposure policy when deployment mode is `authenticated` |
+| `PAPERCLIP_API_URL` | (auto-derived) | Paperclip API base URL. When set externally (e.g., via Kubernetes ConfigMap, load balancer, or reverse proxy), the server preserves the value instead of deriving it from the listen host and port. Useful for deployments where the public-facing URL differs from the local bind address. |
+
+## Secrets
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `PAPERCLIP_SECRETS_MASTER_KEY` | (from file) | 32-byte encryption key (base64/hex/raw) |
+| `PAPERCLIP_SECRETS_MASTER_KEY_FILE` | `~/.paperclip/.../secrets/master.key` | Path to key file |
+| `PAPERCLIP_SECRETS_STRICT_MODE` | `false` | Require secret refs for sensitive env vars |
+
+## Agent Runtime (Injected into agent processes)
+
+These are set automatically by the server when invoking agents:
+
+| Variable | Description |
+|----------|-------------|
+| `PAPERCLIP_AGENT_ID` | Agent's unique ID |
+| `PAPERCLIP_COMPANY_ID` | Company ID |
+| `PAPERCLIP_API_URL` | Paperclip API base URL (inherits the server-level value; see Server Configuration above) |
+| `PAPERCLIP_API_KEY` | Short-lived JWT for API auth |
+| `PAPERCLIP_RUN_ID` | Current heartbeat run ID |
+| `PAPERCLIP_TASK_ID` | Issue that triggered this wake |
+| `PAPERCLIP_WAKE_REASON` | Wake trigger reason |
+| `PAPERCLIP_WAKE_COMMENT_ID` | Comment that triggered this wake |
+| `PAPERCLIP_APPROVAL_ID` | Resolved approval ID |
+| `PAPERCLIP_APPROVAL_STATUS` | Approval decision |
+| `PAPERCLIP_LINKED_ISSUE_IDS` | Comma-separated linked issue IDs |
+
+## LLM Provider Keys (for adapters)
+
+| Variable | Description |
+|----------|-------------|
+| `ANTHROPIC_API_KEY` | Anthropic API key (for Claude Code adapter) |
+| `OPENAI_API_KEY` | OpenAI API key (for Codex adapter) |

docs/deploy/local-development.md

@@ -0,0 +1,105 @@
+---
+title: Local Development
+summary: Set up Paperclip for local development
+---
+
+Run Paperclip locally with zero external dependencies.
+
+## Prerequisites
+
+- Node.js 20+
+- pnpm 9+
+
+## Start Dev Server
+
+```sh
+pnpm install
+pnpm dev
+```
+
+This starts:
+
+- **API server** at `http://localhost:3100`
+- **UI** served by the API server in dev middleware mode (same origin)
+
+No Docker or external database required. Paperclip uses embedded PostgreSQL automatically.
+
+## One-Command Bootstrap
+
+For a first-time install:
+
+```sh
+pnpm paperclipai run
+```
+
+This does:
+
+1. Auto-onboards if config is missing
+2. Runs `paperclipai doctor` with repair enabled
+3. Starts the server when checks pass
+
+## Bind Presets In Dev
+
+Default `pnpm dev` stays in `local_trusted` with loopback-only binding.
+
+To open Paperclip to a private network with login enabled:
+
+```sh
+pnpm dev --bind lan
+```
+
+For Tailscale-only binding on a detected tailnet address:
+
+```sh
+pnpm dev --bind tailnet
+```
+
+Legacy aliases still work and map to the older broad private-network behavior:
+
+```sh
+pnpm dev --tailscale-auth
+pnpm dev --authenticated-private
+```
+
+Allow additional private hostnames:
+
+```sh
+pnpm paperclipai allowed-hostname dotta-macbook-pro
+```
+
+For full setup and troubleshooting, see [Tailscale Private Access](/deploy/tailscale-private-access).
+
+## Health Checks
+
+```sh
+curl http://localhost:3100/api/health
+# -> {"status":"ok"}
+
+curl http://localhost:3100/api/companies
+# -> []
+```
+
+## Reset Dev Data
+
+To wipe local data and start fresh:
+
+```sh
+rm -rf ~/.paperclip/instances/default/db
+pnpm dev
+```
+
+## Data Locations
+
+| Data | Path |
+|------|------|
+| Config | `~/.paperclip/instances/default/config.json` |
+| Database | `~/.paperclip/instances/default/db` |
+| Storage | `~/.paperclip/instances/default/data/storage` |
+| Secrets key | `~/.paperclip/instances/default/secrets/master.key` |
+| Logs | `~/.paperclip/instances/default/logs` |
+
+Override with environment variables:
+
+```sh
+PAPERCLIP_HOME=/custom/path PAPERCLIP_INSTANCE_ID=dev pnpm paperclipai run
+```

docs/deploy/overview.md

@@ -0,0 +1,55 @@
+---
+title: Deployment Overview
+summary: Deployment modes at a glance
+---
+
+Paperclip supports three deployment configurations, from zero-friction local to internet-facing production.
+
+## Deployment Modes
+
+| Mode | Auth | Best For |
+|------|------|----------|
+| `local_trusted` | No login required | Single-operator local machine |
+| `authenticated` + `private` | Login required | Private network (Tailscale, VPN, LAN) |
+| `authenticated` + `public` | Login required | Internet-facing cloud deployment |
+
+## Quick Comparison
+
+### Local Trusted (Default)
+
+- Loopback-only host binding (localhost)
+- No human login flow
+- Fastest local startup
+- Best for: solo development and experimentation
+
+### Authenticated + Private
+
+- Login required via Better Auth
+- Binds to all interfaces for network access
+- Auto base URL mode (lower friction)
+- Best for: team access over Tailscale or local network
+
+### Authenticated + Public
+
+- Login required
+- Explicit public URL required
+- Stricter security checks
+- Best for: cloud hosting, internet-facing deployment
+
+## Choosing a Mode
+
+- **Just trying Paperclip?** Use `local_trusted` (the default)
+- **Sharing with a team on private network?** Use `authenticated` + `private`
+- **Deploying to the cloud?** Use `authenticated` + `public` — see [AWS ECS Fargate guide](aws-ecs.md)
+
+Set the mode during onboarding:
+
+```sh
+pnpm paperclipai onboard
+```
+
+Or update it later:
+
+```sh
+pnpm paperclipai configure --section server
+```

docs/deploy/secrets.md

@@ -0,0 +1,514 @@
+---
+title: Secrets Management
+summary: Master key, encryption, and strict mode
+---
+
+Paperclip encrypts secrets at rest using a local master key. Agent environment variables that contain sensitive values (API keys, tokens) are stored as encrypted secret references.
+
+## Custody Boundaries
+
+Paperclip protects secret values up to the moment they are handed to an agent
+or workload:
+
+- Storage: values are encrypted at rest by the active provider. The local
+  provider keeps them encrypted with a key that never leaves the host.
+- Transport: values are decrypted server-side and injected into the agent
+  process environment, SSH command env, sandbox driver, or HTTP request
+  immediately before the call. Paperclip does not return decrypted values to
+  the board UI.
+- Audit: each resolution records a non-sensitive event (secret id, version,
+  provider id, consumer, outcome) without the value or provider credentials.
+
+Once a value reaches the consuming process, Paperclip can no longer guarantee
+secrecy. The agent (or sandbox, or remote host) can read the value, write it to
+its own logs or transcript, or pass it to downstream tools. Treat any secret
+you bind to an agent as exposed to that agent. Limit blast radius with bindings
+(only bind what each agent needs), short-lived provider credentials where the
+provider supports them, and rotation when an agent transcript or downstream
+system might have captured a value.
+
+## Using Secrets In Runs
+
+Creating a company secret does not automatically create an environment variable.
+You use a secret by binding it into an agent, project, environment, or plugin
+configuration field that supports secret references.
+
+For agent and project environment variables:
+
+1. Create or link the secret in `Company Settings > Secrets`.
+2. Open the agent's `Environment variables` field, or the project's `Env`
+   field.
+3. Add the environment variable key the process expects, such as `GH_TOKEN` or
+   `OPENAI_API_KEY`.
+4. Set the row source to `Secret`, select the stored secret, and choose either
+   `latest` or a pinned version.
+
+At runtime, Paperclip resolves the selected secret server-side and injects the
+resolved value under the env key from the binding row. The stored secret name
+can be human-readable; the binding key is what the agent process receives.
+
+Project env applies to every issue run in that project. When a project env key
+matches an agent env key, the project value wins before Paperclip injects its
+own `PAPERCLIP_*` runtime variables.
+
+## User-Specific Secrets
+
+User-specific secrets let a shared agent or project declare a slot such as
+`github_api_token`, then resolve the value owned by the run's responsible user
+at dispatch time. The environment binding stores only the definition key:
+
+```json
+{
+  "env": {
+    "GITHUB_TOKEN": {
+      "type": "user_secret_ref",
+      "key": "github_api_token",
+      "required": true,
+      "allowMissingOverride": false
+    }
+  }
+}
+```
+
+Paperclip stores the feature as value-free metadata plus the user's own secret
+value record:
+
+- `user_secret_definitions`: company-level metadata for the reusable slot.
+- `user_secret_declarations`: target/config-path declarations for
+  `user_secret_ref` bindings, including required/optional policy.
+- `company_secrets` rows with `scope = "user"`, `owner_user_id`, and
+  `user_secret_definition_id`: the current user's actual value record.
+- `company_secret_versions`: encrypted or provider-backed version metadata for
+  the value record.
+
+Board/admin operators manage definitions and coverage. Individual users manage
+their own values. Board/admin coverage views must stay metadata-only: they can
+show missing, configured, inactive, provider, and vault status, but not
+plaintext values, raw external refs, provider credentials, or provider error
+payloads.
+
+Required user-secret refs fail closed when the responsible user is missing,
+the definition is missing, or the responsible user has no active value.
+Optional refs may omit the env var. They must not inject blank credentials or
+fall back to another user's value.
+
+### Vault Placement
+
+User-scoped values can use the same provider families as company secrets:
+
+- `local_encrypted`: the default local path. It is appropriate for local
+  trusted installs and small self-hosted deployments. The same master key
+  protects both company and user-scoped values.
+- `aws_secrets_manager`: the hosted/provider-vault path. Use it when the
+  deployment already relies on AWS Secrets Manager, KMS, CloudTrail, and
+  infrastructure IAM for custody.
+- Dedicated provider vaults: optional. Use a dedicated vault only when you need
+  a separate AWS account, Region, KMS key, prefix, retention posture, or
+  import boundary for user-owned values. Do not create one vault per user by
+  default; prefer one vault per environment or compliance boundary.
+
+A user-secret definition can carry provider and vault defaults. A user's value
+may also carry provider/vault metadata when the implementation path supports
+overrides. In both cases, provider vault config contains non-secret routing
+metadata only. Provider credentials still come from deployment infrastructure
+identity, not from Paperclip secrets.
+
+### External Reference Naming
+
+External vault references are secret-adjacent metadata. Treat paths, ARNs,
+versions, aliases, and tags as operationally sensitive even though they are not
+plaintext secret values.
+
+Recommended external ref shape for operator-owned AWS paths:
+
+```text
+paperclip-ext/{environment}/{company-id}/user-secrets/{definition-key}/{opaque-owner-id}
+```
+
+Guidance:
+
+- Use the user-secret definition key for the credential type, such as
+  `github_api_token`.
+- Use an opaque stable user id or one-way mapped subject id for the owner
+  segment. Avoid emails, personal names, customer names, OAuth scopes, or
+  ticket identifiers in provider paths.
+- Keep provider metadata value-free. Safe metadata examples are provider id,
+  vault id, Region, KMS key id or alias, tag counts, and fingerprint hashes.
+  Do not store raw AWS descriptions, full tag maps, token scopes, provider
+  error bodies, or anything copied from the secret value.
+- Keep Paperclip-managed AWS values under the Paperclip managed namespace.
+  External refs under that namespace are blocked by guardrails; use the
+  Paperclip-managed flow when Paperclip should create and rotate the value.
+
+### IAM Caveats
+
+Paperclip enforces company scoping, responsible-user derivation, declaration
+policy, current-user value APIs, redaction, and access-event metadata. It does
+not replace the IAM policy of an external vault.
+
+For AWS Secrets Manager, the Paperclip runtime role needs `GetSecretValue` and
+any required KMS decrypt permission for every user-scoped value it may resolve.
+If you link user-specific external refs outside the Paperclip managed prefix,
+scope those permissions to the approved external prefixes and KMS keys. AWS
+tag/name filters help operators search, but they are not a reliable permission
+boundary for resolution.
+
+If stronger provider-side isolation is required, split user-secret workloads by
+provider vault, AWS account, Region, prefix, or runtime role before linking the
+refs. Paperclip can prevent an agent from choosing another credential owner,
+but a runtime role with broad external vault read permissions can still read
+what IAM allows if another code path is introduced outside Paperclip.
+
+## Default Provider: `local_encrypted`
+
+Secrets are encrypted with a local master key stored at:
+
+```
+~/.paperclip/instances/default/secrets/master.key
+```
+
+This key is auto-created during onboarding. The key never leaves your machine.
+Paperclip best-effort enforces `0600` permissions when it creates or loads the
+key file. `paperclipai doctor` and the provider health API warn when the file is
+readable by group or other users.
+
+Back up the key file together with database backups. A database backup without
+the key cannot decrypt local secrets, and a key backup without the database
+metadata is not enough to restore named secret versions.
+
+## Configuration
+
+### CLI Setup
+
+Onboarding writes default secrets config:
+
+```sh
+pnpm paperclipai onboard
+```
+
+Update secrets settings:
+
+```sh
+pnpm paperclipai configure --section secrets
+```
+
+Validate secrets config:
+
+```sh
+pnpm paperclipai doctor
+pnpm paperclipai secrets doctor --company-id <company-id>
+```
+
+### Environment Overrides
+
+| Variable | Description |
+|----------|-------------|
+| `PAPERCLIP_SECRETS_MASTER_KEY` | 32-byte key as base64, hex, or raw string |
+| `PAPERCLIP_SECRETS_MASTER_KEY_FILE` | Custom key file path |
+| `PAPERCLIP_SECRETS_STRICT_MODE` | Set to `true` to enforce secret refs |
+
+## Strict Mode
+
+When strict mode is enabled, sensitive env keys (matching `*_API_KEY`, `*_TOKEN`, `*_SECRET`) must use secret references instead of inline plain values.
+
+```sh
+PAPERCLIP_SECRETS_STRICT_MODE=true
+```
+
+Recommended for any deployment beyond local trusted.
+
+Authenticated deployments default strict mode on unless explicitly overridden by
+configuration or `PAPERCLIP_SECRETS_STRICT_MODE=false`.
+
+## External References
+
+Provider-owned secrets can be linked without copying values into Paperclip by
+using `managedMode: "external_reference"` plus a provider `externalRef`.
+Paperclip stores metadata and a non-sensitive fingerprint, never the value.
+Runtime resolution remains server-side and binding-enforced.
+
+The built-in AWS, GCP, and Vault provider IDs currently accept external
+reference metadata, but runtime resolution requires provider configuration in the
+deployment. Their provider health check reports this as a warning until
+configured.
+
+For hosted Paperclip Cloud on AWS, see the AWS Secrets Manager operational
+contract — required env vars, IAM/KMS scoping, naming and tag conventions, and
+backup/rotation/incident runbooks — in `doc/SECRETS-AWS-PROVIDER.md`.
+
+## Provider Vaults
+
+A *provider vault* is a named, company-scoped configuration that points secret
+material at one of the supported provider backends. Each company can configure
+multiple vaults, including more than one vault per provider family, and pick a
+default vault per family for new secret operations. Existing secrets created
+before any vault was configured continue to resolve through the deployment-level
+default provider — no migration is required.
+
+### Where to configure
+
+Open `Company Settings → Secrets` in the board UI and switch to the
+`Provider vaults` tab. From there you can:
+
+- Create a vault for any supported provider family.
+- Edit the non-secret config of an existing vault.
+- Set one ready vault per provider family as the company default.
+- Disable a vault (a soft delete that keeps audit history).
+- Run a health check against a vault and read the latest result inline.
+
+The same operations are exposed under
+`/api/companies/{companyId}/secret-provider-configs` for automation. See the
+[secrets API reference](/api/secrets#provider-vaults) for the full route table.
+
+### Custody Of Provider Credentials
+
+Provider vaults intentionally store only **non-sensitive** configuration:
+region, project id, namespace, prefix, KMS key id, mount path, address, and
+similar routing metadata. The API, UI, and activity log never accept, return,
+or display provider credential values. Submitting fields with names like
+`accessKeyId`, `secretAccessKey`, `token`, `password`, `serviceAccountJson`,
+`privateKey`, `keyFile`, `unsealKey`, or any common credential alias is rejected
+at validation time.
+
+That keeps the bootstrap rule from the AWS provider applicable to every
+provider family: **provider credentials live in deployment infrastructure
+identity, not in Paperclip company secrets**. Allowed credential sources are
+workload identity attached to the Paperclip server (instance profile, IRSA, ECS
+task role), `AWS_PROFILE` / SSO / shared config for local runs, an orchestrator
+secret store that boots the server, or short-lived shell credentials for local
+development. Do not paste long-lived API keys into the vault config.
+
+### Vault Status
+
+Each vault carries a status that drives what the runtime can do with it:
+
+| Status        | Meaning                                                                                       |
+|---------------|-----------------------------------------------------------------------------------------------|
+| `ready`       | Selectable for create/rotate/resolve. Eligible to be the default.                             |
+| `warning`     | Saved config exists but health needs attention (for example missing AWS env). Still selectable. |
+| `coming_soon` | Visible and editable as draft metadata, but locked out of all runtime operations.            |
+| `disabled`    | Soft-deleted. Hidden from the secret create/rotate flow.                                      |
+
+`gcp_secret_manager` and `vault` are pinned to `coming_soon` until their
+runtime modules ship. The settings UI lets you save draft configuration for
+those providers (and surfaces them on the vault list), but secret create,
+rotate, and resolve calls that target a coming-soon vault fail with a clear
+runtime-locked error.
+
+### Default Vault Behavior
+
+A company can mark **one** ready (or warning) vault per provider family as the
+default. The secret create and rotate dialogs preselect the default vault for
+the chosen provider so operators don't have to remember which vault to pick.
+Coming-soon and disabled vaults cannot be marked default; attempting to do so
+returns a validation error. Setting a new default automatically clears the
+previous default for that provider.
+
+If a secret is created without any `providerConfigId` (no vaults exist yet, or
+the operator clears the selector), runtime resolution falls back to the
+deployment-level provider configuration — the same path existing installs use.
+This keeps secrets created before any provider vault was configured working
+without migration. Picking the default in the UI is an explicit selection, not
+a runtime fallback: the create call still sends an explicit `providerConfigId`.
+
+### Multiple Vaults Per Provider
+
+Multiple vaults from the same provider family are first-class. Common patterns:
+
+- Two AWS vaults pointing at different regions or KMS keys for environment
+  separation.
+- A staging Vault address alongside a production address.
+- A dedicated GCP project for a single product line while the rest of the
+  company uses another.
+
+Each vault has its own display name, status, default flag, and health record.
+Operators choose the vault explicitly when creating or rotating a secret; the
+default vault is preselected to avoid accidental routing to the wrong account.
+
+### Per-Vault Health Checks
+
+`POST /api/secret-provider-configs/{id}/health` runs a provider-specific health
+probe and stores the result on the vault row. The settings UI exposes the same
+action and renders the result inline. Health responses include a status,
+operator-facing message, and structured guidance (such as missing env var
+names, expected credential sources, and backup reminders). They never include
+provider credentials or secret values. Coming-soon vaults always return a
+`runtime_locked` health code and never call into provider modules.
+
+### Provider-Specific Notes
+
+**Local encrypted vaults** wrap the existing `local_encrypted` provider. The
+master key path and rotation guidance described above still applies. A local
+vault config is mostly bookkeeping plus an explicit acknowledgement that the
+key file is backed up alongside the database.
+
+**AWS Secrets Manager vaults** read the per-vault `region`, `namespace`,
+`secretNamePrefix`, `kmsKeyId`, `ownerTag`, and `environmentTag` to route
+managed writes and external-reference reads. The vault config supplements (and
+can override) the deployment-level `PAPERCLIP_SECRETS_AWS_*` env. Bootstrap
+credentials still come from the AWS SDK default credential chain — see
+`doc/SECRETS-AWS-PROVIDER.md` for the full IAM and KMS contract.
+
+**GCP Secret Manager** and **HashiCorp Vault** vaults are coming soon. You can
+save draft `projectId`, `location`, `namespace`, `address`, and `mountPath`
+metadata so the company is ready to flip them on when the provider modules
+ship. Vault `address` values must be origin-only `http(s)://host[:port]` URLs;
+addresses with embedded credentials, paths, query strings, or fragments are
+rejected.
+
+### Remote Import From AWS Vaults
+
+AWS provider vaults can import existing AWS Secrets Manager entries as
+Paperclip `external_reference` secrets. This is a metadata-only link: Paperclip
+stores the AWS ARN/path, a fingerprint/version reference, and binding metadata.
+It does not read, copy, store, log, or display the remote plaintext secret
+value during preview or import.
+
+Operator flow in the board UI:
+
+1. Open `Company Settings -> Secrets`.
+2. Confirm at least one AWS provider vault is `ready` or `warning`.
+3. In the `Secrets` tab, choose `Import from vault`.
+4. Select an AWS vault, search the remote inventory, and load more pages as
+   needed.
+5. Check the rows to import, review/edit the Paperclip name and key, then
+   submit.
+6. Review the result summary for created, skipped, and failed rows.
+
+The preview list is intentionally paged and search-first. AWS accounts can have
+large per-Region inventories, and `ListSecrets` returns opaque `NextToken`
+cursors. Do not expect Paperclip to crawl a whole account in the background;
+load pages deliberately and retry throttled requests with backoff.
+
+Remote import exposes AWS secret metadata visible to the Paperclip runtime
+role, including names/ARNs and safe derived fields such as dates, whether a
+description or KMS key exists, and tag count. Treat names, ARNs, tags, and
+search text as operational metadata that may be sensitive. The API and activity
+log must not store raw descriptions, tags, plaintext values, provider
+credentials, or raw AWS error blobs.
+
+Required AWS posture:
+
+- Preview needs optional `secretsmanager:ListSecrets` permission on
+  `Resource: "*"`. AWS does not support constraining `ListSecrets` to
+  individual secret ARNs or tags as an IAM boundary.
+- Preview/import must not call `secretsmanager:GetSecretValue`,
+  `secretsmanager:BatchGetSecretValue`, or KMS decrypt.
+- Runtime resolution of an imported reference still needs
+  `secretsmanager:GetSecretValue` on the selected external ARN/path and KMS
+  decrypt when that secret uses a customer-managed key.
+- Keep managed create/rotate/delete permissions scoped to the Paperclip
+  deployment prefix. Do not broaden managed write/delete permissions just
+  because import inventory is enabled.
+
+Safe scoping comes from deployment posture rather than AWS list filtering:
+dedicated Paperclip runtime roles per environment/account, AWS vaults pointed at
+the intended account and Region, import-enabled roles only where inventory
+exposure is acceptable, and board-only access to the import routes. Tags and
+name filters are search aids, not a permission model.
+
+If import preview fails:
+
+- `AccessDenied` or `not authorized`: the runtime role is missing
+  `secretsmanager:ListSecrets`; add the optional inventory statement only if
+  remote import should be enabled for that vault.
+- Throttling: retry after a short delay and narrow the search before loading
+  more pages.
+- Invalid cursor: refresh the preview; AWS `NextToken` values are opaque and
+  can expire or become stale.
+- Runtime resolution failure after import: verify `GetSecretValue` and KMS
+  decrypt scope for the selected external secret. Being visible in inventory is
+  not proof that the runtime role can read the value.
+
+### Backup And Restore
+
+Each provider family has a different backup story:
+
+- `local_encrypted`: back up the local master key file and the Paperclip
+  database together. Either alone is not enough to restore the encrypted
+  values, and the vault row only records the path and acknowledgement, not the
+  key bytes. This includes user-scoped values: the database holds
+  `user_secret_definitions`, `user_secret_declarations`,
+  `company_secrets.scope = "user"` rows, version metadata, and owner ids; the
+  key file is required to decrypt their local material.
+- `aws_secrets_manager`: back up Paperclip's database for vault metadata
+  (vault id, region, prefix, KMS key id, default flag, bindings, version
+  pointers, user-secret definitions/declarations, owner ids, and access-event
+  metadata). The actual secret values live in AWS Secrets Manager under the
+  configured prefix or operator-owned external refs; restore by pointing the
+  same Paperclip company at the same AWS namespace and confirming the runtime
+  role still has `GetSecretValue` plus KMS decrypt for both managed and linked
+  user-scoped values. The full restore checklist lives in
+  `doc/SECRETS-AWS-PROVIDER.md`.
+- `gcp_secret_manager` and `vault`: while these are coming soon, only the
+  draft vault config exists in Paperclip. Database backups capture it. There
+  is nothing to restore on the provider side until runtime support lands.
+
+### AWS Provider Bootstrap Boundary
+
+The AWS Secrets Manager provider cannot bootstrap itself from Paperclip
+`company_secrets`. Its initial AWS access must be present before the server can
+create or resolve AWS-backed company secrets, regardless of whether you use the
+deployment-level default or a per-company vault.
+
+For Paperclip Cloud, provision the server runtime IAM role/workload identity,
+KMS key, deployment prefix, and non-secret `PAPERCLIP_SECRETS_AWS_*` environment
+configuration before enabling AWS-backed secrets in the board UI. For
+self-hosted and local runs, use the AWS SDK default credential chain: instance
+profile, ECS task role, EKS IRSA/OIDC web identity, AWS SSO/shared config via
+`AWS_PROFILE`, or short-lived shell credentials for local development.
+
+Do not store AWS root credentials or long-lived IAM user access keys in
+Paperclip secrets. Bootstrap material belongs in infrastructure IAM/workload
+identity, the process environment, an AWS profile, or the orchestrator secret
+store.
+
+## Migrating Inline Secrets
+
+If you have existing agents with inline API keys in their config, migrate them to encrypted secret refs:
+
+```sh
+pnpm paperclipai secrets migrate-inline-env --company-id <company-id>
+pnpm paperclipai secrets migrate-inline-env --company-id <company-id> --apply
+
+# low-level script for direct database maintenance
+pnpm secrets:migrate-inline-env         # dry run
+pnpm secrets:migrate-inline-env --apply # apply migration
+```
+
+Use the CLI command for normal operations because it goes through the Paperclip
+API, creates or rotates secret records, and updates agent env bindings with
+audit logging.
+
+## Portable Declarations
+
+Company exports include only environment declarations. They do not include
+secret IDs, provider references, encrypted material, or plaintext values.
+
+```sh
+pnpm paperclipai secrets declarations --company-id <company-id> --kind secret
+```
+
+Before importing a package into another instance, use those declarations to
+create local values or link hosted provider references in the target deployment.
+For hosted providers such as AWS Secrets Manager, the hosted provider remains
+the value custodian; Paperclip stores metadata and provider version references,
+not provider credentials or plaintext secret values.
+
+## Secret References in Agent Config
+
+Agent environment variables use secret references:
+
+```json
+{
+  "env": {
+    "ANTHROPIC_API_KEY": {
+      "type": "secret_ref",
+      "secretId": "8f884973-c29b-44e4-8ea3-6413437f8081",
+      "version": "latest"
+    }
+  }
+}
+```
+
+The server resolves and decrypts these at runtime, injecting the real value into the agent process environment.

docs/deploy/storage.md

@@ -0,0 +1,39 @@
+---
+title: Storage
+summary: Local disk vs S3-compatible storage
+---
+
+Paperclip stores uploaded files (issue attachments, images) using a configurable storage provider.
+
+## Local Disk (Default)
+
+Files are stored at:
+
+```
+~/.paperclip/instances/default/data/storage
+```
+
+No configuration required. Suitable for local development and single-machine deployments.
+
+## S3-Compatible Storage
+
+For production or multi-node deployments, use S3-compatible object storage (AWS S3, MinIO, Cloudflare R2, etc.).
+
+Configure via CLI:
+
+```sh
+pnpm paperclipai configure --section storage
+```
+
+## Configuration
+
+| Provider | Best For |
+|----------|----------|
+| `local_disk` | Local development, single-machine deployments |
+| `s3` | Production, multi-node, cloud deployments |
+
+Storage configuration is stored in the instance config file:
+
+```
+~/.paperclip/instances/default/config.json
+```

docs/deploy/tailscale-private-access.md

@@ -0,0 +1,82 @@
+---
+title: Tailscale Private Access
+summary: Run Paperclip with Tailscale-friendly bind presets and connect from other devices
+---
+
+Use this when you want to access Paperclip over Tailscale (or a private LAN/VPN) instead of only `localhost`.
+
+## 1. Start Paperclip in private authenticated mode
+
+```sh
+pnpm dev --bind tailnet
+```
+
+Recommended behavior:
+
+- `PAPERCLIP_DEPLOYMENT_MODE=authenticated`
+- `PAPERCLIP_DEPLOYMENT_EXPOSURE=private`
+- `PAPERCLIP_BIND=tailnet`
+
+If you want the old broad private-network behavior instead, use:
+
+```sh
+pnpm dev --bind lan
+```
+
+Legacy aliases still map to `authenticated/private + bind=lan`:
+
+pnpm dev --authenticated-private
+pnpm dev --tailscale-auth
+```
+
+## 2. Find your reachable Tailscale address
+
+From the machine running Paperclip:
+
+```sh
+tailscale ip -4
+```
+
+You can also use your Tailscale MagicDNS hostname (for example `my-macbook.tailnet.ts.net`).
+
+## 3. Open Paperclip from another device
+
+Use the Tailscale IP or MagicDNS host with the Paperclip port:
+
+```txt
+http://<tailscale-host-or-ip>:3100
+```
+
+Example:
+
+```txt
+http://my-macbook.tailnet.ts.net:3100
+```
+
+## 4. Allow custom private hostnames when needed
+
+If you access Paperclip with a custom private hostname, add it to the allowlist:
+
+```sh
+pnpm paperclipai allowed-hostname my-macbook.tailnet.ts.net
+```
+
+## 5. Verify the server is reachable
+
+From a remote Tailscale-connected device:
+
+```sh
+curl http://<tailscale-host-or-ip>:3100/api/health
+```
+
+Expected result:
+
+```json
+{"status":"ok"}
+```
+
+## Troubleshooting
+
+- Login or redirect errors on a private hostname: add it with `paperclipai allowed-hostname`.
+- App only works on `localhost`: make sure you started with `--bind lan` or `--bind tailnet` instead of plain `pnpm dev`.
+- Can connect locally but not remotely: verify both devices are on the same Tailscale network and port `3100` is reachable.