diff --git a/README.md b/README.md index f7ade1b389e..42f24cd1a87 100644 --- a/README.md +++ b/README.md @@ -177,6 +177,8 @@ Open source. Self-hosted. No Paperclip account required. npx paperclipai onboard --yes ``` +If you already have Paperclip configured, rerunning `onboard` keeps the existing config in place. Use `paperclipai configure` to edit settings. + Or manually: ```bash diff --git a/cli/README.md b/cli/README.md new file mode 100644 index 00000000000..08f36ad49b3 --- /dev/null +++ b/cli/README.md @@ -0,0 +1,292 @@ +

+ Paperclip — runs your business +

+ +

+ Quickstart · + Docs · + GitHub · + Discord +

+ +

+ MIT License + Stars + Discord +

+ +
+ +
+ +
+ +
+ +## What is Paperclip? + +# Open-source orchestration for zero-human companies + +**If OpenClaw is an _employee_, Paperclip is the _company_** + +Paperclip is a Node.js server and React UI that orchestrates a team of AI agents to run a business. Bring your own agents, assign goals, and track your agents' work and costs from one dashboard. + +It looks like a task manager — but under the hood it has org charts, budgets, governance, goal alignment, and agent coordination. + +**Manage business goals, not pull requests.** + +| | Step | Example | +| ------ | --------------- | ------------------------------------------------------------------ | +| **01** | Define the goal | _"Build the #1 AI note-taking app to $1M MRR."_ | +| **02** | Hire the team | CEO, CTO, engineers, designers, marketers — any bot, any provider. | +| **03** | Approve and run | Review strategy. Set budgets. Hit go. Monitor from the dashboard. | + +
+ +> **COMING SOON: Clipmart** — Download and run entire companies with one click. Browse pre-built company templates — full org structures, agent configs, and skills — and import them into your Paperclip instance in seconds. + +
+ +
+ + + + + + + + + + +
Works
with
OpenClaw
OpenClaw
Claude
Claude Code
Codex
Codex
Cursor
Cursor
Bash
Bash
HTTP
HTTP
+ +If it can receive a heartbeat, it's hired. + +
+ +
+ +## Paperclip is right for you if + +- ✅ You want to build **autonomous AI companies** +- ✅ You **coordinate many different agents** (OpenClaw, Codex, Claude, Cursor) toward a common goal +- ✅ You have **20 simultaneous Claude Code terminals** open and lose track of what everyone is doing +- ✅ You want agents running **autonomously 24/7**, but still want to audit work and chime in when needed +- ✅ You want to **monitor costs** and enforce budgets +- ✅ You want a process for managing agents that **feels like using a task manager** +- ✅ You want to manage your autonomous businesses **from your phone** + +
+ +## Features + + + + + + + + + + + + + + + + + +
+

🔌 Bring Your Own Agent

+Any agent, any runtime, one org chart. If it can receive a heartbeat, it's hired. +
+

🎯 Goal Alignment

+Every task traces back to the company mission. Agents know what to do and why. +
+

💓 Heartbeats

+Agents wake on a schedule, check work, and act. Delegation flows up and down the org chart. +
+

💰 Cost Control

+Monthly budgets per agent. When they hit the limit, they stop. No runaway costs. +
+

🏢 Multi-Company

+One deployment, many companies. Complete data isolation. One control plane for your portfolio. +
+

🎫 Ticket System

+Every conversation traced. Every decision explained. Full tool-call tracing and immutable audit log. +
+

🛡️ Governance

+You're the board. Approve hires, override strategy, pause or terminate any agent — at any time. +
+

📊 Org Chart

+Hierarchies, roles, reporting lines. Your agents have a boss, a title, and a job description. +
+

📱 Mobile Ready

+Monitor and manage your autonomous businesses from anywhere. +
+ +
+ +## Problems Paperclip solves + +| Without Paperclip | With Paperclip | +| ------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | +| ❌ You have 20 Claude Code tabs open and can't track which one does what. On reboot you lose everything. | ✅ Tasks are ticket-based, conversations are threaded, sessions persist across reboots. | +| ❌ You manually gather context from several places to remind your bot what you're actually doing. | ✅ Context flows from the task up through the project and company goals — your agent always knows what to do and why. | +| ❌ Folders of agent configs are disorganized and you're re-inventing task management, communication, and coordination between agents. | ✅ Paperclip gives you org charts, ticketing, delegation, and governance out of the box — so you run a company, not a pile of scripts. | +| ❌ Runaway loops waste hundreds of dollars of tokens and max your quota before you even know what happened. | ✅ Cost tracking surfaces token budgets and throttles agents when they're out. Management prioritizes with budgets. | +| ❌ You have recurring jobs (customer support, social, reports) and have to remember to manually kick them off. | ✅ Heartbeats handle regular work on a schedule. Management supervises. | +| ❌ You have an idea, you have to find your repo, fire up Claude Code, keep a tab open, and babysit it. | ✅ Add a task in Paperclip. Your coding agent works on it until it's done. Management reviews their work. | + +
+ +## Why Paperclip is special + +Paperclip handles the hard orchestration details correctly. + +| | | +| --------------------------------- | ------------------------------------------------------------------------------------------------------------- | +| **Atomic execution.** | Task checkout and budget enforcement are atomic, so no double-work and no runaway spend. | +| **Persistent agent state.** | Agents resume the same task context across heartbeats instead of restarting from scratch. | +| **Runtime skill injection.** | Agents can learn Paperclip workflows and project context at runtime, without retraining. | +| **Governance with rollback.** | Approval gates are enforced, config changes are revisioned, and bad changes can be rolled back safely. | +| **Goal-aware execution.** | Tasks carry full goal ancestry so agents consistently see the "why," not just a title. | +| **Portable company templates.** | Export/import orgs, agents, and skills with secret scrubbing and collision handling. | +| **True multi-company isolation.** | Every entity is company-scoped, so one deployment can run many companies with separate data and audit trails. | + +
+ +## What Paperclip is not + +| | | +| ---------------------------- | -------------------------------------------------------------------------------------------------------------------- | +| **Not a chatbot.** | Agents have jobs, not chat windows. | +| **Not an agent framework.** | We don't tell you how to build agents. We tell you how to run a company made of them. | +| **Not a workflow builder.** | No drag-and-drop pipelines. Paperclip models companies — with org charts, goals, budgets, and governance. | +| **Not a prompt manager.** | Agents bring their own prompts, models, and runtimes. Paperclip manages the organization they work in. | +| **Not a single-agent tool.** | This is for teams. If you have one agent, you probably don't need Paperclip. If you have twenty — you definitely do. | +| **Not a code review tool.** | Paperclip orchestrates work, not pull requests. Bring your own review process. | + +
+ +## Quickstart + +Open source. Self-hosted. No Paperclip account required. + +```bash +npx paperclipai onboard --yes +``` + +If you already have Paperclip configured, rerunning `onboard` keeps the existing config in place. Use `paperclipai configure` to edit settings. + +Or manually: + +```bash +git clone https://github.com/paperclipai/paperclip.git +cd paperclip +pnpm install +pnpm dev +``` + +This starts the API server at `http://localhost:3100`. An embedded PostgreSQL database is created automatically — no setup required. + +> **Requirements:** Node.js 20+, pnpm 9.15+ + +
+ +## FAQ + +**What does a typical setup look like?** +Locally, a single Node.js process manages an embedded Postgres and local file storage. For production, point it at your own Postgres and deploy however you like. Configure projects, agents, and goals — the agents take care of the rest. + +If you're a solo-entreprenuer you can use Tailscale to access Paperclip on the go. Then later you can deploy to e.g. Vercel when you need it. + +**Can I run multiple companies?** +Yes. A single deployment can run an unlimited number of companies with complete data isolation. + +**How is Paperclip different from agents like OpenClaw or Claude Code?** +Paperclip _uses_ those agents. It orchestrates them into a company — with org charts, budgets, goals, governance, and accountability. + +**Why should I use Paperclip instead of just pointing my OpenClaw to Asana or Trello?** +Agent orchestration has subtleties in how you coordinate who has work checked out, how to maintain sessions, monitoring costs, establishing governance - Paperclip does this for you. + +(Bring-your-own-ticket-system is on the Roadmap) + +**Do agents run continuously?** +By default, agents run on scheduled heartbeats and event-based triggers (task assignment, @-mentions). You can also hook in continuous agents like OpenClaw. You bring your agent and Paperclip coordinates. + +
+ +## Development + +```bash +pnpm dev # Full dev (API + UI, watch mode) +pnpm dev:once # Full dev without file watching +pnpm dev:server # Server only +pnpm build # Build all +pnpm typecheck # Type checking +pnpm test:run # Run tests +pnpm db:generate # Generate DB migration +pnpm db:migrate # Apply migrations +``` + +See [doc/DEVELOPING.md](https://github.com/paperclipai/paperclip/blob/master/doc/DEVELOPING.md) for the full development guide. + +
+ +## Roadmap + +- ✅ Plugin system (e.g. add a knowledge base, custom tracing, queues, etc) +- ✅ Get OpenClaw / claw-style agent employees +- ✅ companies.sh - import and export entire organizations +- ✅ Easy AGENTS.md configurations +- ✅ Skills Manager +- ✅ Scheduled Routines +- ✅ Better Budgeting +- ⚪ Artifacts & Deployments +- ⚪ CEO Chat +- ⚪ MAXIMIZER MODE +- ⚪ Multiple Human Users +- ⚪ Cloud / Sandbox agents (e.g. Cursor / e2b agents) +- ⚪ Cloud deployments +- ⚪ Desktop App + +
+ +## Community & Plugins + +Find Plugins and more at [awesome-paperclip](https://github.com/gsxdsm/awesome-paperclip) + +## Contributing + +We welcome contributions. See the [contributing guide](https://github.com/paperclipai/paperclip/blob/master/CONTRIBUTING.md) for details. + +
+ +## Community + +- [Discord](https://discord.gg/m4HZY7xNG3) — Join the community +- [GitHub Issues](https://github.com/paperclipai/paperclip/issues) — bugs and feature requests +- [GitHub Discussions](https://github.com/paperclipai/paperclip/discussions) — ideas and RFC + +
+ +## License + +MIT © 2026 Paperclip + +## Star History + +[![Star History Chart](https://api.star-history.com/image?repos=paperclipai/paperclip&type=date&legend=top-left)](https://www.star-history.com/?repos=paperclipai%2Fpaperclip&type=date&legend=top-left) + +
+ +--- + +

+ +

+ +

+ Open source under MIT. Built for people who want to run companies, not babysit agents. +

diff --git a/cli/src/__tests__/onboard.test.ts b/cli/src/__tests__/onboard.test.ts new file mode 100644 index 00000000000..a5ffe44ad49 --- /dev/null +++ b/cli/src/__tests__/onboard.test.ts @@ -0,0 +1,105 @@ +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { onboard } from "../commands/onboard.js"; +import type { PaperclipConfig } from "../config/schema.js"; + +const ORIGINAL_ENV = { ...process.env }; + +function createExistingConfigFixture() { + const root = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-onboard-")); + const runtimeRoot = path.join(root, "runtime"); + const configPath = path.join(root, ".paperclip", "config.json"); + const config: PaperclipConfig = { + $meta: { + version: 1, + updatedAt: "2026-03-29T00:00:00.000Z", + source: "configure", + }, + database: { + mode: "embedded-postgres", + embeddedPostgresDataDir: path.join(runtimeRoot, "db"), + embeddedPostgresPort: 54329, + backup: { + enabled: true, + intervalMinutes: 60, + retentionDays: 30, + dir: path.join(runtimeRoot, "backups"), + }, + }, + logging: { + mode: "file", + logDir: path.join(runtimeRoot, "logs"), + }, + server: { + deploymentMode: "local_trusted", + exposure: "private", + host: "127.0.0.1", + port: 3100, + allowedHostnames: [], + serveUi: true, + }, + auth: { + baseUrlMode: "auto", + disableSignUp: false, + }, + storage: { + provider: "local_disk", + localDisk: { + baseDir: path.join(runtimeRoot, "storage"), + }, + s3: { + bucket: "paperclip", + region: "us-east-1", + prefix: "", + forcePathStyle: false, + }, + }, + secrets: { + provider: "local_encrypted", + strictMode: false, + localEncrypted: { + keyFilePath: path.join(runtimeRoot, "secrets", "master.key"), + }, + }, + }; + + fs.mkdirSync(path.dirname(configPath), { recursive: true }); + fs.writeFileSync(configPath, `${JSON.stringify(config, null, 2)}\n`, { mode: 0o600 }); + + return { configPath, configText: fs.readFileSync(configPath, "utf8") }; +} + +describe("onboard", () => { + beforeEach(() => { + process.env = { ...ORIGINAL_ENV }; + delete process.env.PAPERCLIP_AGENT_JWT_SECRET; + delete process.env.PAPERCLIP_SECRETS_MASTER_KEY; + delete process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE; + }); + + afterEach(() => { + process.env = { ...ORIGINAL_ENV }; + }); + + it("preserves an existing config when rerun without flags", async () => { + const fixture = createExistingConfigFixture(); + + await onboard({ config: fixture.configPath }); + + expect(fs.readFileSync(fixture.configPath, "utf8")).toBe(fixture.configText); + expect(fs.existsSync(`${fixture.configPath}.backup`)).toBe(false); + expect(fs.existsSync(path.join(path.dirname(fixture.configPath), ".env"))).toBe(true); + }); + + it("preserves an existing config when rerun with --yes", async () => { + const fixture = createExistingConfigFixture(); + + await onboard({ config: fixture.configPath, yes: true, invokedByRun: true }); + + expect(fs.readFileSync(fixture.configPath, "utf8")).toBe(fixture.configText); + expect(fs.existsSync(`${fixture.configPath}.backup`)).toBe(false); + expect(fs.existsSync(path.join(path.dirname(fixture.configPath), ".env"))).toBe(true); + }); +}); diff --git a/cli/src/commands/onboard.ts b/cli/src/commands/onboard.ts index 523484f3c69..d470354f8d1 100644 --- a/cli/src/commands/onboard.ts +++ b/cli/src/commands/onboard.ts @@ -244,11 +244,12 @@ export async function onboard(opts: OnboardOptions): Promise { ), ); + let existingConfig: PaperclipConfig | null = null; if (configExists(opts.config)) { - p.log.message(pc.dim(`${configPath} exists, updating config`)); + p.log.message(pc.dim(`${configPath} exists`)); try { - readConfig(opts.config); + existingConfig = readConfig(opts.config); } catch (err) { p.log.message( pc.yellow( @@ -258,6 +259,76 @@ export async function onboard(opts: OnboardOptions): Promise { } } + if (existingConfig) { + p.log.message( + pc.dim("Existing Paperclip install detected; keeping the current configuration unchanged."), + ); + p.log.message(pc.dim(`Use ${pc.cyan("paperclipai configure")} if you want to change settings.`)); + + const jwtSecret = ensureAgentJwtSecret(configPath); + const envFilePath = resolveAgentJwtEnvFile(configPath); + if (jwtSecret.created) { + p.log.success(`Created ${pc.cyan("PAPERCLIP_AGENT_JWT_SECRET")} in ${pc.dim(envFilePath)}`); + } else if (process.env.PAPERCLIP_AGENT_JWT_SECRET?.trim()) { + p.log.info(`Using existing ${pc.cyan("PAPERCLIP_AGENT_JWT_SECRET")} from environment`); + } else { + p.log.info(`Using existing ${pc.cyan("PAPERCLIP_AGENT_JWT_SECRET")} in ${pc.dim(envFilePath)}`); + } + + const keyResult = ensureLocalSecretsKeyFile(existingConfig, configPath); + if (keyResult.status === "created") { + p.log.success(`Created local secrets key file at ${pc.dim(keyResult.path)}`); + } else if (keyResult.status === "existing") { + p.log.message(pc.dim(`Using existing local secrets key file at ${keyResult.path}`)); + } + + p.note( + [ + "Existing config preserved", + `Database: ${existingConfig.database.mode}`, + existingConfig.llm ? `LLM: ${existingConfig.llm.provider}` : "LLM: not configured", + `Logging: ${existingConfig.logging.mode} -> ${existingConfig.logging.logDir}`, + `Server: ${existingConfig.server.deploymentMode}/${existingConfig.server.exposure} @ ${existingConfig.server.host}:${existingConfig.server.port}`, + `Allowed hosts: ${existingConfig.server.allowedHostnames.length > 0 ? existingConfig.server.allowedHostnames.join(", ") : "(loopback only)"}`, + `Auth URL mode: ${existingConfig.auth.baseUrlMode}${existingConfig.auth.publicBaseUrl ? ` (${existingConfig.auth.publicBaseUrl})` : ""}`, + `Storage: ${existingConfig.storage.provider}`, + `Secrets: ${existingConfig.secrets.provider} (strict mode ${existingConfig.secrets.strictMode ? "on" : "off"})`, + "Agent auth: PAPERCLIP_AGENT_JWT_SECRET configured", + ].join("\n"), + "Configuration ready", + ); + + p.note( + [ + `Run: ${pc.cyan("paperclipai run")}`, + `Reconfigure later: ${pc.cyan("paperclipai configure")}`, + `Diagnose setup: ${pc.cyan("paperclipai doctor")}`, + ].join("\n"), + "Next commands", + ); + + let shouldRunNow = opts.run === true || opts.yes === true; + if (!shouldRunNow && !opts.invokedByRun && process.stdin.isTTY && process.stdout.isTTY) { + const answer = await p.confirm({ + message: "Start Paperclip now?", + initialValue: true, + }); + if (!p.isCancel(answer)) { + shouldRunNow = answer; + } + } + + if (shouldRunNow && !opts.invokedByRun) { + process.env.PAPERCLIP_OPEN_ON_LISTEN = "true"; + const { runCommand } = await import("./run.js"); + await runCommand({ config: configPath, repair: true, yes: true }); + return; + } + + p.outro("Existing Paperclip setup is ready."); + return; + } + let setupMode: SetupMode = "quickstart"; if (opts.yes) { p.log.message(pc.dim("`--yes` enabled: using Quickstart defaults.")); diff --git a/doc/DEVELOPING.md b/doc/DEVELOPING.md index 7864b90edbc..e85de611e95 100644 --- a/doc/DEVELOPING.md +++ b/doc/DEVELOPING.md @@ -39,6 +39,17 @@ This starts: `pnpm dev` runs the server in watch mode and restarts on changes from workspace packages (including adapter packages). Use `pnpm dev:once` to run without file watching. +`pnpm dev:once` auto-applies pending local migrations by default before starting the dev server. + +`pnpm dev` and `pnpm dev:once` are now idempotent for the current repo and instance: if the matching Paperclip dev runner is already alive, Paperclip reports the existing process instead of starting a duplicate. + +Inspect or stop the current repo's managed dev runner: + +```sh +pnpm dev:list +pnpm dev:stop +``` + `pnpm dev:once` now tracks backend-relevant file changes and pending migrations. When the current boot is stale, the board UI shows a `Restart required` banner. You can also enable guarded auto-restart in `Instance Settings > Experimental`, which waits for queued/running local agent runs to finish before restarting the dev server. Tailscale/private-auth dev mode: diff --git a/doc/PUBLISHING.md b/doc/PUBLISHING.md index 32e4b1312a1..83a6c4a7be7 100644 --- a/doc/PUBLISHING.md +++ b/doc/PUBLISHING.md @@ -76,6 +76,45 @@ The `ui` package uses [`scripts/generate-ui-package-json.mjs`](../scripts/genera After packing or publishing, `postpack` restores the development manifest automatically. +### Manual first publish for `@paperclipai/ui` + +If you need to publish only the UI package once by hand, use the real package name: + +- `@paperclipai/ui` + +Recommended flow from the repo root: + +```bash +# optional sanity check: this 404s until the first publish exists +npm view @paperclipai/ui version + +# make sure the dist payload is fresh +pnpm --filter @paperclipai/ui build + +# confirm your local npm auth before the real publish +npm whoami + +# safe preview of the exact publish payload +cd ui +pnpm publish --dry-run --no-git-checks --access public + +# real publish +pnpm publish --no-git-checks --access public +``` + +Notes: + +- Publish from `ui/`, not the repo root. +- `prepack` automatically rewrites `ui/package.json` to the lean publish manifest, and `postpack` restores the dev manifest after the command finishes. +- If `npm view @paperclipai/ui version` already returns the same version that is in [`ui/package.json`](../ui/package.json), do not republish. Bump the version or use the normal repo-wide release flow in [`scripts/release.sh`](../scripts/release.sh). + +If the first real publish returns npm `E404`, check npm-side prerequisites before retrying: + +- `npm whoami` must succeed first. An expired or missing npm login will block the publish. +- For an organization-scoped package like `@paperclipai/ui`, the `paperclipai` npm organization must exist and the publisher must be a member with permission to publish to that scope. +- The initial publish must include `--access public` for a public scoped package. +- npm also requires either account 2FA for publishing or a granular token that is allowed to bypass 2FA. + ## Version formats Paperclip uses calendar versions: diff --git a/docs/cli/setup-commands.md b/docs/cli/setup-commands.md index 7dc5cd6ad03..448ab7bbc4f 100644 --- a/docs/cli/setup-commands.md +++ b/docs/cli/setup-commands.md @@ -33,6 +33,8 @@ Interactive first-time setup: pnpm paperclipai onboard ``` +If Paperclip is already configured, rerunning `onboard` keeps the existing config in place. Use `paperclipai configure` to change settings on an existing install. + First prompt: 1. `Quickstart` (recommended): local defaults (embedded database, no LLM provider, local disk storage, default secrets) @@ -50,6 +52,8 @@ Non-interactive defaults + immediate start (opens browser on server listen): pnpm paperclipai onboard --yes ``` +On an existing install, `--yes` now preserves the current config and just starts Paperclip with that setup. + ## `paperclipai doctor` Health checks with optional auto-repair: diff --git a/docs/docs.json b/docs/docs.json index 90789e06bbc..7faa6c86991 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -47,6 +47,8 @@ "guides/board-operator/org-structure", "guides/board-operator/managing-tasks", "guides/board-operator/delegation", + "guides/board-operator/execution-workspaces-and-runtime-services", + "guides/board-operator/delegation", "guides/board-operator/approvals", "guides/board-operator/costs-and-budgets", "guides/board-operator/activity-log", diff --git a/docs/guides/board-operator/execution-workspaces-and-runtime-services.md b/docs/guides/board-operator/execution-workspaces-and-runtime-services.md new file mode 100644 index 00000000000..285d701a489 --- /dev/null +++ b/docs/guides/board-operator/execution-workspaces-and-runtime-services.md @@ -0,0 +1,68 @@ +--- +title: Execution Workspaces And Runtime Services +summary: How project runtime configuration, execution workspaces, and issue runs fit together +--- + +This guide documents the intended runtime model for projects, execution workspaces, and issue runs in Paperclip. + +## Project runtime configuration + +You can define how to run a project on the project workspace itself. + +- Project workspace runtime config describes how to run services for that project checkout. +- This is the default runtime configuration that child execution workspaces may inherit. +- Defining the config does not start anything by itself. + +## Manual runtime control + +Runtime services are manually controlled from the UI. + +- Project workspace runtime services are started and stopped from the project workspace UI. +- Execution workspace runtime services are started and stopped from the execution workspace UI. +- Paperclip does not automatically start or stop these runtime services as part of issue execution. +- Paperclip also does not automatically restart workspace runtime services on server boot. + +## Execution workspace inheritance + +Execution workspaces isolate code and runtime state from the project primary workspace. + +- An isolated execution workspace has its own checkout path, branch, and local runtime instance. +- The runtime configuration may inherit from the linked project workspace by default. +- The execution workspace may override that runtime configuration with its own workspace-specific settings. +- The inherited configuration answers "how to run the service", but the running process is still specific to that execution workspace. + +## Issues and execution workspaces + +Issues are attached to execution workspace behavior, not to automatic runtime management. + +- An issue may create a new execution workspace when you choose an isolated workspace mode. +- An issue may reuse an existing execution workspace when you choose reuse. +- Multiple issues may intentionally share one execution workspace so they can work against the same branch and running runtime services. +- Assigning or running an issue does not automatically start or stop runtime services for that workspace. + +## Execution workspace lifecycle + +Execution workspaces are durable until a human closes them. + +- The UI can archive an execution workspace. +- Closing an execution workspace stops its runtime services and cleans up its workspace artifacts when allowed. +- Shared workspaces that point at the project primary checkout are treated more conservatively during cleanup than disposable isolated workspaces. + +## Resolved workspace logic during heartbeat runs + +Heartbeat still resolves a workspace for the run, but that is about code location and session continuity, not runtime-service control. + +1. Heartbeat resolves a base workspace for the run. +2. Paperclip realizes the effective execution workspace, including creating or reusing a worktree when needed. +3. Paperclip persists execution-workspace metadata such as paths, refs, and provisioning settings. +4. Heartbeat passes the resolved code workspace to the agent run. +5. Workspace runtime services remain manual UI-managed controls rather than automatic heartbeat-managed services. + +## Current implementation guarantees + +With the current implementation: + +- Project workspace runtime config is the fallback for execution workspace UI controls. +- Execution workspace runtime overrides are stored on the execution workspace. +- Heartbeat runs do not auto-start workspace runtime services. +- Server startup does not auto-restart workspace runtime services. diff --git a/docs/start/quickstart.md b/docs/start/quickstart.md index 1ad30fcdfa5..2abe538b0ec 100644 --- a/docs/start/quickstart.md +++ b/docs/start/quickstart.md @@ -13,6 +13,8 @@ npx paperclipai onboard --yes This walks you through setup, configures your environment, and gets Paperclip running. +If you already have a Paperclip install, rerunning `onboard` keeps your current config and data paths intact. Use `paperclipai configure` if you want to edit settings. + To start Paperclip again later: ```sh diff --git a/package.json b/package.json index 749cc8d0cfb..311a092f1af 100644 --- a/package.json +++ b/package.json @@ -3,9 +3,11 @@ "private": true, "type": "module", "scripts": { - "dev": "node scripts/dev-runner.mjs watch", - "dev:watch": "node scripts/dev-runner.mjs watch", - "dev:once": "node scripts/dev-runner.mjs dev", + "dev": "pnpm --filter @paperclipai/server exec tsx ../scripts/dev-runner.ts watch", + "dev:watch": "pnpm --filter @paperclipai/server exec tsx ../scripts/dev-runner.ts watch", + "dev:once": "pnpm --filter @paperclipai/server exec tsx ../scripts/dev-runner.ts dev", + "dev:list": "pnpm --filter @paperclipai/server exec tsx ../scripts/dev-service.ts list", + "dev:stop": "pnpm --filter @paperclipai/server exec tsx ../scripts/dev-service.ts stop", "dev:server": "pnpm --filter @paperclipai/server dev", "dev:ui": "pnpm --filter @paperclipai/ui dev", "build": "pnpm -r build", @@ -32,7 +34,8 @@ "test:e2e:headed": "npx playwright test --config tests/e2e/playwright.config.ts --headed", "evals:smoke": "cd evals/promptfoo && npx promptfoo@0.103.3 eval", "test:release-smoke": "npx playwright test --config tests/release-smoke/playwright.config.ts", - "test:release-smoke:headed": "npx playwright test --config tests/release-smoke/playwright.config.ts --headed" + "test:release-smoke:headed": "npx playwright test --config tests/release-smoke/playwright.config.ts --headed", + "metrics:paperclip-commits": "tsx scripts/paperclip-commit-metrics.ts" }, "devDependencies": { "@playwright/test": "^1.58.2", diff --git a/packages/adapter-utils/src/server-utils.ts b/packages/adapter-utils/src/server-utils.ts index 12989f72e70..4a5affdff53 100644 --- a/packages/adapter-utils/src/server-utils.ts +++ b/packages/adapter-utils/src/server-utils.ts @@ -201,6 +201,33 @@ export function redactEnvForLogs(env: Record): Record, + options: { + runtimeEnv?: NodeJS.ProcessEnv | Record; + includeRuntimeKeys?: string[]; + resolvedCommand?: string | null; + resolvedCommandEnvKey?: string; + } = {}, +): Record { + const merged: Record = { ...env }; + const runtimeEnv = options.runtimeEnv ?? {}; + + for (const key of options.includeRuntimeKeys ?? []) { + if (key in merged) continue; + const value = runtimeEnv[key]; + if (typeof value !== "string" || value.length === 0) continue; + merged[key] = value; + } + + const resolvedCommand = options.resolvedCommand?.trim(); + if (resolvedCommand) { + merged[options.resolvedCommandEnvKey ?? "PAPERCLIP_RESOLVED_COMMAND"] = resolvedCommand; + } + + return redactEnvForLogs(merged); +} + export function buildPaperclipEnv(agent: { id: string; companyId: string }): Record { const resolveHostForUrl = (rawHost: string): string => { const host = rawHost.trim(); @@ -269,6 +296,10 @@ async function resolveCommandPath(command: string, cwd: string, env: NodeJS.Proc return null; } +export async function resolveCommandForLogs(command: string, cwd: string, env: NodeJS.ProcessEnv): Promise { + return (await resolveCommandPath(command, cwd, env)) ?? command; +} + function quoteForCmd(arg: string) { if (!arg.length) return '""'; const escaped = arg.replace(/"/g, '""'); diff --git a/packages/adapters/claude-local/src/cli/format-event.ts b/packages/adapters/claude-local/src/cli/format-event.ts index 08423d6ea77..13263be6d4c 100644 --- a/packages/adapters/claude-local/src/cli/format-event.ts +++ b/packages/adapters/claude-local/src/cli/format-event.ts @@ -17,6 +17,27 @@ function asErrorText(value: unknown): string { } } +function printToolResult(block: Record): void { + const isError = block.is_error === true; + let text = ""; + if (typeof block.content === "string") { + text = block.content; + } else if (Array.isArray(block.content)) { + const parts: string[] = []; + for (const part of block.content) { + if (typeof part !== "object" || part === null || Array.isArray(part)) continue; + const record = part as Record; + if (typeof record.text === "string") parts.push(record.text); + } + text = parts.join("\n"); + } + + console.log((isError ? pc.red : pc.cyan)(`tool_result${isError ? " (error)" : ""}`)); + if (text) { + console.log((isError ? pc.red : pc.gray)(text)); + } +} + export function printClaudeStreamEvent(raw: string, debug: boolean): void { const line = raw.trim(); if (!line) return; @@ -51,6 +72,9 @@ export function printClaudeStreamEvent(raw: string, debug: boolean): void { if (blockType === "text") { const text = typeof block.text === "string" ? block.text : ""; if (text) console.log(pc.green(`assistant: ${text}`)); + } else if (blockType === "thinking") { + const text = typeof block.thinking === "string" ? block.thinking : ""; + if (text) console.log(pc.gray(`thinking: ${text}`)); } else if (blockType === "tool_use") { const name = typeof block.name === "string" ? block.name : "unknown"; console.log(pc.yellow(`tool_call: ${name}`)); @@ -62,6 +86,22 @@ export function printClaudeStreamEvent(raw: string, debug: boolean): void { return; } + if (type === "user") { + const message = + typeof parsed.message === "object" && parsed.message !== null && !Array.isArray(parsed.message) + ? (parsed.message as Record) + : {}; + const content = Array.isArray(message.content) ? message.content : []; + for (const blockRaw of content) { + if (typeof blockRaw !== "object" || blockRaw === null || Array.isArray(blockRaw)) continue; + const block = blockRaw as Record; + if (typeof block.type === "string" && block.type === "tool_result") { + printToolResult(block); + } + } + return; + } + if (type === "result") { const usage = typeof parsed.usage === "object" && parsed.usage !== null && !Array.isArray(parsed.usage) diff --git a/packages/adapters/claude-local/src/index.ts b/packages/adapters/claude-local/src/index.ts index b28ae18097b..41c0693f8d4 100644 --- a/packages/adapters/claude-local/src/index.ts +++ b/packages/adapters/claude-local/src/index.ts @@ -26,7 +26,7 @@ Core fields: - extraArgs (string[], optional): additional CLI args - env (object, optional): KEY=VALUE environment variables - workspaceStrategy (object, optional): execution workspace strategy; currently supports { type: "git_worktree", baseRef?, branchTemplate?, worktreeParentDir? } -- workspaceRuntime (object, optional): workspace runtime service intents; local host-managed services are realized before Claude starts and exposed back via context/env +- workspaceRuntime (object, optional): reserved for workspace runtime metadata; workspace runtime services are manually controlled from the workspace UI and are not auto-started by heartbeats Operational fields: - timeoutSec (number, optional): run timeout in seconds diff --git a/packages/adapters/claude-local/src/server/execute.ts b/packages/adapters/claude-local/src/server/execute.ts index 8ac1d7ee6d5..c7d6c6a8b22 100644 --- a/packages/adapters/claude-local/src/server/execute.ts +++ b/packages/adapters/claude-local/src/server/execute.ts @@ -14,10 +14,11 @@ import { buildPaperclipEnv, readPaperclipRuntimeSkillEntries, joinPromptSections, - redactEnvForLogs, + buildInvocationEnvForLogs, ensureAbsoluteDirectory, ensureCommandResolvable, ensurePathInEnv, + resolveCommandForLogs, renderTemplate, runChildProcess, } from "@paperclipai/adapter-utils/server-utils"; @@ -68,11 +69,13 @@ interface ClaudeExecutionInput { interface ClaudeRuntimeConfig { command: string; + resolvedCommand: string; cwd: string; workspaceId: string | null; workspaceRepoUrl: string | null; workspaceRepoRef: string | null; env: Record; + loggedEnv: Record; timeoutSec: number; graceSec: number; extraArgs: string[]; @@ -236,6 +239,12 @@ async function buildClaudeRuntimeConfig(input: ClaudeExecutionInput): Promise { if (idx === args.length - 1 && value !== "-") return ``; return value; }), - env: redactEnvForLogs(env), + env: loggedEnv, prompt, promptMetrics, context, diff --git a/packages/adapters/cursor-local/src/server/execute.ts b/packages/adapters/cursor-local/src/server/execute.ts index df33969001b..ed41754a9e1 100644 --- a/packages/adapters/cursor-local/src/server/execute.ts +++ b/packages/adapters/cursor-local/src/server/execute.ts @@ -9,12 +9,13 @@ import { asStringArray, parseObject, buildPaperclipEnv, - redactEnvForLogs, + buildInvocationEnvForLogs, ensureAbsoluteDirectory, ensureCommandResolvable, ensurePaperclipSkillSymlink, ensurePathInEnv, readPaperclipRuntimeSkillEntries, + resolveCommandForLogs, resolvePaperclipDesiredSkillNames, removeMaintainerOnlySkillSymlinks, renderTemplate, @@ -271,6 +272,12 @@ export async function execute(ctx: AdapterExecutionContext): Promise ( index === args.length - 1 ? `` : value )), - env: redactEnvForLogs(env), + env: loggedEnv, prompt, promptMetrics, context, diff --git a/packages/adapters/openclaw-gateway/src/index.ts b/packages/adapters/openclaw-gateway/src/index.ts index 195edfbf8df..1bdf66f2e65 100644 --- a/packages/adapters/openclaw-gateway/src/index.ts +++ b/packages/adapters/openclaw-gateway/src/index.ts @@ -31,7 +31,7 @@ Gateway connect identity fields: Request behavior fields: - payloadTemplate (object, optional): additional fields merged into gateway agent params -- workspaceRuntime (object, optional): desired runtime service intents; Paperclip forwards these in a standardized paperclip.workspaceRuntime block for remote execution environments +- workspaceRuntime (object, optional): reserved workspace runtime metadata; workspace runtime services are manually controlled from the workspace UI and are not auto-started by heartbeats - timeoutSec (number, optional): adapter timeout in seconds (default 120) - waitTimeoutMs (number, optional): agent.wait timeout override (default timeoutSec * 1000) - autoPairOnFirstConnect (boolean, optional): on first "pairing required", attempt device.pair.list/device.pair.approve via shared auth, then retry once (default true) @@ -45,7 +45,7 @@ Standard outbound payload additions: - paperclip (object): standardized Paperclip context added to every gateway agent request - paperclip.workspace (object, optional): resolved execution workspace for this run - paperclip.workspaces (array, optional): additional workspace hints Paperclip exposed to the run -- paperclip.workspaceRuntime (object, optional): normalized runtime service intent config for the workspace +- paperclip.workspaceRuntime (object, optional): reserved workspace runtime metadata when explicitly supplied outside normal heartbeat execution Standard result metadata supported: - meta.runtimeServices (array, optional): normalized adapter-managed runtime service reports diff --git a/packages/adapters/opencode-local/src/server/execute.ts b/packages/adapters/opencode-local/src/server/execute.ts index f3a5ae50881..7c034c692ae 100644 --- a/packages/adapters/opencode-local/src/server/execute.ts +++ b/packages/adapters/opencode-local/src/server/execute.ts @@ -10,11 +10,12 @@ import { parseObject, buildPaperclipEnv, joinPromptSections, - redactEnvForLogs, + buildInvocationEnvForLogs, ensureAbsoluteDirectory, ensureCommandResolvable, ensurePaperclipSkillSymlink, ensurePathInEnv, + resolveCommandForLogs, renderTemplate, runChildProcess, readPaperclipRuntimeSkillEntries, @@ -186,6 +187,12 @@ export async function execute(ctx: AdapterExecutionContext): Promise`], - env: redactEnvForLogs(preparedRuntimeConfig.env), + env: loggedEnv, prompt, promptMetrics, context, diff --git a/packages/adapters/pi-local/src/server/execute.ts b/packages/adapters/pi-local/src/server/execute.ts index a78bc1d4877..b1d9d801d64 100644 --- a/packages/adapters/pi-local/src/server/execute.ts +++ b/packages/adapters/pi-local/src/server/execute.ts @@ -10,12 +10,13 @@ import { parseObject, buildPaperclipEnv, joinPromptSections, - redactEnvForLogs, + buildInvocationEnvForLogs, ensureAbsoluteDirectory, ensureCommandResolvable, ensurePaperclipSkillSymlink, ensurePathInEnv, readPaperclipRuntimeSkillEntries, + resolveCommandForLogs, resolvePaperclipDesiredSkillNames, removeMaintainerOnlySkillSymlinks, renderTemplate, @@ -204,6 +205,12 @@ export async function execute(ctx: AdapterExecutionContext): Promise | null; + runtimeConfig: ProjectWorkspaceRuntimeConfig | null; isPrimary: boolean; runtimeServices?: WorkspaceRuntimeService[]; createdAt: Date; diff --git a/packages/shared/src/types/workspace-runtime.ts b/packages/shared/src/types/workspace-runtime.ts index 47ed9494c22..2b2c4e2d433 100644 --- a/packages/shared/src/types/workspace-runtime.ts +++ b/packages/shared/src/types/workspace-runtime.ts @@ -31,6 +31,22 @@ export type ExecutionWorkspaceStatus = | "archived" | "cleanup_failed"; +export type ExecutionWorkspaceCloseReadinessState = + | "ready" + | "ready_with_warnings" + | "blocked"; + +export type ExecutionWorkspaceCloseActionKind = + | "archive_record" + | "stop_runtime_services" + | "cleanup_command" + | "teardown_command" + | "git_worktree_remove" + | "git_branch_delete" + | "remove_local_directory"; + +export type WorkspaceRuntimeDesiredState = "running" | "stopped"; + export interface ExecutionWorkspaceStrategy { type: ExecutionWorkspaceStrategyType; baseRef?: string | null; @@ -40,6 +56,63 @@ export interface ExecutionWorkspaceStrategy { teardownCommand?: string | null; } +export interface ExecutionWorkspaceConfig { + provisionCommand: string | null; + teardownCommand: string | null; + cleanupCommand: string | null; + workspaceRuntime: Record | null; + desiredState: WorkspaceRuntimeDesiredState | null; +} + +export interface ProjectWorkspaceRuntimeConfig { + workspaceRuntime: Record | null; + desiredState: WorkspaceRuntimeDesiredState | null; +} + +export interface ExecutionWorkspaceCloseAction { + kind: ExecutionWorkspaceCloseActionKind; + label: string; + description: string; + command: string | null; +} + +export interface ExecutionWorkspaceCloseLinkedIssue { + id: string; + identifier: string | null; + title: string; + status: string; + isTerminal: boolean; +} + +export interface ExecutionWorkspaceCloseGitReadiness { + repoRoot: string | null; + workspacePath: string | null; + branchName: string | null; + baseRef: string | null; + hasDirtyTrackedFiles: boolean; + hasUntrackedFiles: boolean; + dirtyEntryCount: number; + untrackedEntryCount: number; + aheadCount: number | null; + behindCount: number | null; + isMergedIntoBase: boolean | null; + createdByRuntime: boolean; +} + +export interface ExecutionWorkspaceCloseReadiness { + workspaceId: string; + state: ExecutionWorkspaceCloseReadinessState; + blockingReasons: string[]; + warnings: string[]; + linkedIssues: ExecutionWorkspaceCloseLinkedIssue[]; + plannedActions: ExecutionWorkspaceCloseAction[]; + isDestructiveCloseAllowed: boolean; + isSharedWorkspace: boolean; + isProjectPrimaryWorkspace: boolean; + git: ExecutionWorkspaceCloseGitReadiness | null; + runtimeServices: WorkspaceRuntimeService[]; +} + export interface ProjectExecutionWorkspacePolicy { enabled: boolean; defaultMode?: ProjectExecutionWorkspaceDefaultMode; @@ -81,7 +154,9 @@ export interface ExecutionWorkspace { closedAt: Date | null; cleanupEligibleAt: Date | null; cleanupReason: string | null; + config: ExecutionWorkspaceConfig | null; metadata: Record | null; + runtimeServices?: WorkspaceRuntimeService[]; createdAt: Date; updatedAt: Date; } diff --git a/packages/shared/src/validators/agent.ts b/packages/shared/src/validators/agent.ts index 8c29150b358..288ae683fd0 100644 --- a/packages/shared/src/validators/agent.ts +++ b/packages/shared/src/validators/agent.ts @@ -4,6 +4,7 @@ import { AGENT_ICON_NAMES, AGENT_ROLES, AGENT_STATUSES, + INBOX_MINE_ISSUE_STATUS_FILTER, } from "../constants.js"; import { envConfigSchema } from "./secret.js"; @@ -93,6 +94,13 @@ export const createAgentKeySchema = z.object({ export type CreateAgentKey = z.infer; +export const agentMineInboxQuerySchema = z.object({ + userId: z.string().trim().min(1), + status: z.string().trim().min(1).optional().default(INBOX_MINE_ISSUE_STATUS_FILTER), +}); + +export type AgentMineInboxQuery = z.infer; + export const wakeAgentSchema = z.object({ source: z.enum(["timer", "assignment", "on_demand", "automation"]).optional().default("on_demand"), triggerDetail: z.enum(["manual", "ping", "callback", "system"]).optional(), diff --git a/packages/shared/src/validators/execution-workspace.ts b/packages/shared/src/validators/execution-workspace.ts index 53a74036c6f..d761c528dd6 100644 --- a/packages/shared/src/validators/execution-workspace.ts +++ b/packages/shared/src/validators/execution-workspace.ts @@ -8,10 +8,113 @@ export const executionWorkspaceStatusSchema = z.enum([ "cleanup_failed", ]); +export const executionWorkspaceConfigSchema = z.object({ + provisionCommand: z.string().optional().nullable(), + teardownCommand: z.string().optional().nullable(), + cleanupCommand: z.string().optional().nullable(), + workspaceRuntime: z.record(z.unknown()).optional().nullable(), + desiredState: z.enum(["running", "stopped"]).optional().nullable(), +}).strict(); + +export const executionWorkspaceCloseReadinessStateSchema = z.enum([ + "ready", + "ready_with_warnings", + "blocked", +]); + +export const executionWorkspaceCloseActionKindSchema = z.enum([ + "archive_record", + "stop_runtime_services", + "cleanup_command", + "teardown_command", + "git_worktree_remove", + "git_branch_delete", + "remove_local_directory", +]); + +export const executionWorkspaceCloseActionSchema = z.object({ + kind: executionWorkspaceCloseActionKindSchema, + label: z.string(), + description: z.string(), + command: z.string().nullable(), +}).strict(); + +export const executionWorkspaceCloseLinkedIssueSchema = z.object({ + id: z.string().uuid(), + identifier: z.string().nullable(), + title: z.string(), + status: z.string(), + isTerminal: z.boolean(), +}).strict(); + +export const executionWorkspaceCloseGitReadinessSchema = z.object({ + repoRoot: z.string().nullable(), + workspacePath: z.string().nullable(), + branchName: z.string().nullable(), + baseRef: z.string().nullable(), + hasDirtyTrackedFiles: z.boolean(), + hasUntrackedFiles: z.boolean(), + dirtyEntryCount: z.number().int().nonnegative(), + untrackedEntryCount: z.number().int().nonnegative(), + aheadCount: z.number().int().nonnegative().nullable(), + behindCount: z.number().int().nonnegative().nullable(), + isMergedIntoBase: z.boolean().nullable(), + createdByRuntime: z.boolean(), +}).strict(); + +export const executionWorkspaceCloseReadinessSchema = z.object({ + workspaceId: z.string().uuid(), + state: executionWorkspaceCloseReadinessStateSchema, + blockingReasons: z.array(z.string()), + warnings: z.array(z.string()), + linkedIssues: z.array(executionWorkspaceCloseLinkedIssueSchema), + plannedActions: z.array(executionWorkspaceCloseActionSchema), + isDestructiveCloseAllowed: z.boolean(), + isSharedWorkspace: z.boolean(), + isProjectPrimaryWorkspace: z.boolean(), + git: executionWorkspaceCloseGitReadinessSchema.nullable(), + runtimeServices: z.array(z.object({ + id: z.string(), + companyId: z.string().uuid(), + projectId: z.string().uuid().nullable(), + projectWorkspaceId: z.string().uuid().nullable(), + executionWorkspaceId: z.string().uuid().nullable(), + issueId: z.string().uuid().nullable(), + scopeType: z.enum(["project_workspace", "execution_workspace", "run", "agent"]), + scopeId: z.string().nullable(), + serviceName: z.string(), + status: z.enum(["starting", "running", "stopped", "failed"]), + lifecycle: z.enum(["shared", "ephemeral"]), + reuseKey: z.string().nullable(), + command: z.string().nullable(), + cwd: z.string().nullable(), + port: z.number().int().nullable(), + url: z.string().nullable(), + provider: z.enum(["local_process", "adapter_managed"]), + providerRef: z.string().nullable(), + ownerAgentId: z.string().uuid().nullable(), + startedByRunId: z.string().uuid().nullable(), + lastUsedAt: z.coerce.date(), + startedAt: z.coerce.date(), + stoppedAt: z.coerce.date().nullable(), + stopPolicy: z.record(z.unknown()).nullable(), + healthStatus: z.enum(["unknown", "healthy", "unhealthy"]), + createdAt: z.coerce.date(), + updatedAt: z.coerce.date(), + }).strict()), +}).strict(); + export const updateExecutionWorkspaceSchema = z.object({ + name: z.string().min(1).optional(), + cwd: z.string().optional().nullable(), + repoUrl: z.string().optional().nullable(), + baseRef: z.string().optional().nullable(), + branchName: z.string().optional().nullable(), + providerRef: z.string().optional().nullable(), status: executionWorkspaceStatusSchema.optional(), cleanupEligibleAt: z.string().datetime().optional().nullable(), cleanupReason: z.string().optional().nullable(), + config: executionWorkspaceConfigSchema.optional().nullable(), metadata: z.record(z.unknown()).optional().nullable(), }).strict(); diff --git a/packages/shared/src/validators/index.ts b/packages/shared/src/validators/index.ts index 9ec8ed7a0e7..eb736d5911c 100644 --- a/packages/shared/src/validators/index.ts +++ b/packages/shared/src/validators/index.ts @@ -85,6 +85,7 @@ export { upsertAgentInstructionsFileSchema, updateAgentInstructionsPathSchema, createAgentKeySchema, + agentMineInboxQuerySchema, wakeAgentSchema, resetAgentSessionSchema, testAdapterEnvironmentSchema, @@ -97,6 +98,7 @@ export { type UpsertAgentInstructionsFile, type UpdateAgentInstructionsPath, type CreateAgentKey, + type AgentMineInboxQuery, type WakeAgent, type ResetAgentSession, type TestAdapterEnvironment, @@ -109,6 +111,7 @@ export { createProjectWorkspaceSchema, updateProjectWorkspaceSchema, projectExecutionWorkspacePolicySchema, + projectWorkspaceRuntimeConfigSchema, type CreateProject, type UpdateProject, type CreateProjectWorkspace, @@ -151,8 +154,15 @@ export { } from "./work-product.js"; export { + executionWorkspaceConfigSchema, updateExecutionWorkspaceSchema, executionWorkspaceStatusSchema, + executionWorkspaceCloseActionKindSchema, + executionWorkspaceCloseActionSchema, + executionWorkspaceCloseGitReadinessSchema, + executionWorkspaceCloseLinkedIssueSchema, + executionWorkspaceCloseReadinessSchema, + executionWorkspaceCloseReadinessStateSchema, type UpdateExecutionWorkspace, } from "./execution-workspace.js"; diff --git a/packages/shared/src/validators/issue.ts b/packages/shared/src/validators/issue.ts index 3715e4e6a76..e6d1a34ac3d 100644 --- a/packages/shared/src/validators/issue.ts +++ b/packages/shared/src/validators/issue.ts @@ -32,6 +32,7 @@ export const createIssueSchema = z.object({ projectWorkspaceId: z.string().uuid().optional().nullable(), goalId: z.string().uuid().optional().nullable(), parentId: z.string().uuid().optional().nullable(), + inheritExecutionWorkspaceFromIssueId: z.string().uuid().optional().nullable(), title: z.string().min(1), description: z.string().optional().nullable(), status: z.enum(ISSUE_STATUSES).optional().default("backlog"), @@ -66,6 +67,7 @@ export type CreateIssueLabel = z.infer; export const updateIssueSchema = createIssueSchema.partial().extend({ comment: z.string().min(1).optional(), reopen: z.boolean().optional(), + interrupt: z.boolean().optional(), hiddenAt: z.string().datetime().nullable().optional(), }); diff --git a/packages/shared/src/validators/project.ts b/packages/shared/src/validators/project.ts index cf5aba8af15..89308ff41dc 100644 --- a/packages/shared/src/validators/project.ts +++ b/packages/shared/src/validators/project.ts @@ -27,6 +27,11 @@ export const projectExecutionWorkspacePolicySchema = z }) .strict(); +export const projectWorkspaceRuntimeConfigSchema = z.object({ + workspaceRuntime: z.record(z.unknown()).optional().nullable(), + desiredState: z.enum(["running", "stopped"]).optional().nullable(), +}).strict(); + const projectWorkspaceSourceTypeSchema = z.enum(["local_path", "git_repo", "remote_managed", "non_git_path"]); const projectWorkspaceVisibilitySchema = z.enum(["default", "advanced"]); @@ -44,6 +49,7 @@ const projectWorkspaceFields = { remoteWorkspaceRef: z.string().optional().nullable(), sharedWorkspaceKey: z.string().optional().nullable(), metadata: z.record(z.unknown()).optional().nullable(), + runtimeConfig: projectWorkspaceRuntimeConfigSchema.optional().nullable(), }; function validateProjectWorkspace(value: Record, ctx: z.RefinementCtx) { diff --git a/scripts/dev-runner.ts b/scripts/dev-runner.ts new file mode 100644 index 00000000000..aed49c1ba33 --- /dev/null +++ b/scripts/dev-runner.ts @@ -0,0 +1,656 @@ +#!/usr/bin/env -S node --import tsx +import { spawn } from "node:child_process"; +import { existsSync, mkdirSync, readdirSync, rmSync, statSync, writeFileSync } from "node:fs"; +import path from "node:path"; +import { createInterface } from "node:readline/promises"; +import { stdin, stdout } from "node:process"; +import { shouldTrackDevServerPath } from "./dev-runner-paths.mjs"; +import { createDevServiceIdentity, repoRoot } from "./dev-service-profile.ts"; +import { + findAdoptableLocalService, + removeLocalServiceRegistryRecord, + touchLocalServiceRegistryRecord, + writeLocalServiceRegistryRecord, +} from "../server/src/services/local-service-supervisor.ts"; + +const mode = process.argv[2] === "watch" ? "watch" : "dev"; +const cliArgs = process.argv.slice(3); +const scanIntervalMs = 1500; +const autoRestartPollIntervalMs = 2500; +const gracefulShutdownTimeoutMs = 10_000; +const changedPathSampleLimit = 5; +const devServerStatusFilePath = path.join(repoRoot, ".paperclip", "dev-server-status.json"); + +const watchedDirectories = [ + "cli", + "scripts", + "server", + "packages/adapter-utils", + "packages/adapters", + "packages/db", + "packages/plugins/sdk", + "packages/shared", +].map((relativePath) => path.join(repoRoot, relativePath)); + +const watchedFiles = [ + ".env", + "package.json", + "pnpm-workspace.yaml", + "tsconfig.base.json", + "tsconfig.json", + "vitest.config.ts", +].map((relativePath) => path.join(repoRoot, relativePath)); + +const ignoredDirectoryNames = new Set([ + ".git", + ".turbo", + ".vite", + "coverage", + "dist", + "node_modules", + "ui-dist", +]); + +const ignoredRelativePaths = new Set([ + ".paperclip/dev-server-status.json", +]); + +const tailscaleAuthFlagNames = new Set([ + "--tailscale-auth", + "--authenticated-private", +]); + +let tailscaleAuth = false; +const forwardedArgs: string[] = []; + +for (const arg of cliArgs) { + if (tailscaleAuthFlagNames.has(arg)) { + tailscaleAuth = true; + continue; + } + forwardedArgs.push(arg); +} + +if (process.env.npm_config_tailscale_auth === "true") { + tailscaleAuth = true; +} +if (process.env.npm_config_authenticated_private === "true") { + tailscaleAuth = true; +} + +const env: NodeJS.ProcessEnv = { + ...process.env, + PAPERCLIP_UI_DEV_MIDDLEWARE: "true", +}; + +if (mode === "dev") { + env.PAPERCLIP_DEV_SERVER_STATUS_FILE = devServerStatusFilePath; + env.PAPERCLIP_MIGRATION_AUTO_APPLY ??= "true"; +} + +if (mode === "watch") { + env.PAPERCLIP_MIGRATION_PROMPT ??= "never"; + env.PAPERCLIP_MIGRATION_AUTO_APPLY ??= "true"; +} + +if (tailscaleAuth) { + env.PAPERCLIP_DEPLOYMENT_MODE = "authenticated"; + env.PAPERCLIP_DEPLOYMENT_EXPOSURE = "private"; + env.PAPERCLIP_AUTH_BASE_URL_MODE = "auto"; + env.HOST = "0.0.0.0"; + console.log("[paperclip] dev mode: authenticated/private (tailscale-friendly) on 0.0.0.0"); +} else { + console.log("[paperclip] dev mode: local_trusted (default)"); +} + +const serverPort = Number.parseInt(env.PORT ?? process.env.PORT ?? "3100", 10) || 3100; +const devService = createDevServiceIdentity({ + mode, + forwardedArgs, + tailscaleAuth, + port: serverPort, +}); + +const existingRunner = await findAdoptableLocalService({ + serviceKey: devService.serviceKey, + cwd: repoRoot, + envFingerprint: devService.envFingerprint, + port: serverPort, +}); +if (existingRunner) { + console.log( + `[paperclip] ${devService.serviceName} already running (pid ${existingRunner.pid}${typeof existingRunner.metadata?.childPid === "number" ? `, child ${existingRunner.metadata.childPid}` : ""})`, + ); + process.exit(0); +} + +const pnpmBin = process.platform === "win32" ? "pnpm.cmd" : "pnpm"; +let previousSnapshot = collectWatchedSnapshot(); +let dirtyPaths = new Set(); +let pendingMigrations: string[] = []; +let lastChangedAt: string | null = null; +let lastRestartAt: string | null = null; +let scanInFlight = false; +let restartInFlight = false; +let shuttingDown = false; +let childExitWasExpected = false; +let child: ReturnType | null = null; +let childExitPromise: Promise<{ code: number; signal: NodeJS.Signals | null }> | null = null; +let scanTimer: ReturnType | null = null; +let autoRestartTimer: ReturnType | null = null; + +function toError(error: unknown, context = "Dev runner command failed") { + if (error instanceof Error) return error; + if (error === undefined) return new Error(context); + if (typeof error === "string") return new Error(`${context}: ${error}`); + + try { + return new Error(`${context}: ${JSON.stringify(error)}`); + } catch { + return new Error(`${context}: ${String(error)}`); + } +} + +process.on("uncaughtException", async (error) => { + await removeLocalServiceRegistryRecord(devService.serviceKey); + const err = toError(error, "Uncaught exception in dev runner"); + process.stderr.write(`${err.stack ?? err.message}\n`); + process.exit(1); +}); + +process.on("unhandledRejection", async (reason) => { + await removeLocalServiceRegistryRecord(devService.serviceKey); + const err = toError(reason, "Unhandled promise rejection in dev runner"); + process.stderr.write(`${err.stack ?? err.message}\n`); + process.exit(1); +}); + +function formatPendingMigrationSummary(migrations: string[]) { + if (migrations.length === 0) return "none"; + return migrations.length > 3 + ? `${migrations.slice(0, 3).join(", ")} (+${migrations.length - 3} more)` + : migrations.join(", "); +} + +function exitForSignal(signal: NodeJS.Signals) { + if (signal === "SIGINT") { + process.exit(130); + } + if (signal === "SIGTERM") { + process.exit(143); + } + process.exit(1); +} + +function toRelativePath(absolutePath: string) { + return path.relative(repoRoot, absolutePath).split(path.sep).join("/"); +} + +function readSignature(absolutePath: string) { + const stats = statSync(absolutePath); + return `${Math.trunc(stats.mtimeMs)}:${stats.size}`; +} + +function addFileToSnapshot(snapshot: Map, absolutePath: string) { + const relativePath = toRelativePath(absolutePath); + if (ignoredRelativePaths.has(relativePath)) return; + if (!shouldTrackDevServerPath(relativePath)) return; + snapshot.set(relativePath, readSignature(absolutePath)); +} + +function walkDirectory(snapshot: Map, absoluteDirectory: string) { + if (!existsSync(absoluteDirectory)) return; + + for (const entry of readdirSync(absoluteDirectory, { withFileTypes: true })) { + if (ignoredDirectoryNames.has(entry.name)) continue; + + const absolutePath = path.join(absoluteDirectory, entry.name); + if (entry.isDirectory()) { + walkDirectory(snapshot, absolutePath); + continue; + } + if (entry.isFile() || entry.isSymbolicLink()) { + addFileToSnapshot(snapshot, absolutePath); + } + } +} + +function collectWatchedSnapshot() { + const snapshot = new Map(); + + for (const absoluteDirectory of watchedDirectories) { + walkDirectory(snapshot, absoluteDirectory); + } + for (const absoluteFile of watchedFiles) { + if (!existsSync(absoluteFile)) continue; + addFileToSnapshot(snapshot, absoluteFile); + } + + return snapshot; +} + +function diffSnapshots(previous: Map, next: Map) { + const changed = new Set(); + + for (const [relativePath, signature] of next) { + if (previous.get(relativePath) !== signature) { + changed.add(relativePath); + } + } + for (const relativePath of previous.keys()) { + if (!next.has(relativePath)) { + changed.add(relativePath); + } + } + + return [...changed].sort(); +} + +function ensureDevStatusDirectory() { + mkdirSync(path.dirname(devServerStatusFilePath), { recursive: true }); +} + +function writeDevServerStatus() { + if (mode !== "dev") return; + + ensureDevStatusDirectory(); + const changedPaths = [...dirtyPaths].sort(); + writeFileSync( + devServerStatusFilePath, + `${JSON.stringify({ + dirty: changedPaths.length > 0 || pendingMigrations.length > 0, + lastChangedAt, + changedPathCount: changedPaths.length, + changedPathsSample: changedPaths.slice(0, changedPathSampleLimit), + pendingMigrations, + lastRestartAt, + }, null, 2)}\n`, + "utf8", + ); +} + +function clearDevServerStatus() { + if (mode !== "dev") return; + rmSync(devServerStatusFilePath, { force: true }); +} + +async function updateDevServiceRecord(extra?: Record) { + await writeLocalServiceRegistryRecord({ + version: 1, + serviceKey: devService.serviceKey, + profileKind: "paperclip-dev", + serviceName: devService.serviceName, + command: "dev-runner.ts", + cwd: repoRoot, + envFingerprint: devService.envFingerprint, + port: serverPort, + url: `http://127.0.0.1:${serverPort}`, + pid: process.pid, + processGroupId: null, + provider: "local_process", + runtimeServiceId: null, + reuseKey: null, + startedAt: lastRestartAt ?? new Date().toISOString(), + lastSeenAt: new Date().toISOString(), + metadata: { + repoRoot, + mode, + childPid: child?.pid ?? null, + url: `http://127.0.0.1:${serverPort}`, + ...extra, + }, + }); +} + +async function runPnpm(args: string[], options: { + stdio?: "inherit" | ["ignore", "pipe", "pipe"]; + env?: NodeJS.ProcessEnv; + cwd?: string; +} = {}) { + return await new Promise<{ code: number; signal: NodeJS.Signals | null; stdout: string; stderr: string }>((resolve, reject) => { + const spawned = spawn(pnpmBin, args, { + stdio: options.stdio ?? ["ignore", "pipe", "pipe"], + env: options.env ?? process.env, + cwd: options.cwd, + shell: process.platform === "win32", + }); + + let stdoutBuffer = ""; + let stderrBuffer = ""; + + if (spawned.stdout) { + spawned.stdout.on("data", (chunk) => { + stdoutBuffer += String(chunk); + }); + } + if (spawned.stderr) { + spawned.stderr.on("data", (chunk) => { + stderrBuffer += String(chunk); + }); + } + + spawned.on("error", reject); + spawned.on("exit", (code, signal) => { + resolve({ + code: code ?? 0, + signal, + stdout: stdoutBuffer, + stderr: stderrBuffer, + }); + }); + }); +} + +async function getMigrationStatusPayload() { + const status = await runPnpm( + ["--filter", "@paperclipai/db", "exec", "tsx", "src/migration-status.ts", "--json"], + { env }, + ); + if (status.code !== 0) { + process.stderr.write( + status.stderr || + status.stdout || + `[paperclip] Command failed with code ${status.code}: pnpm --filter @paperclipai/db exec tsx src/migration-status.ts --json\n`, + ); + process.exit(status.code); + } + + try { + return JSON.parse(status.stdout.trim()) as { status?: string; pendingMigrations?: string[] }; + } catch (error) { + process.stderr.write( + status.stderr || + status.stdout || + "[paperclip] migration-status returned invalid JSON payload\n", + ); + throw toError(error, "Unable to parse migration-status JSON output"); + } +} + +async function refreshPendingMigrations() { + const payload = await getMigrationStatusPayload(); + pendingMigrations = + payload.status === "needsMigrations" && Array.isArray(payload.pendingMigrations) + ? payload.pendingMigrations.filter((entry) => typeof entry === "string" && entry.trim().length > 0) + : []; + writeDevServerStatus(); + return payload; +} + +async function maybePreflightMigrations(options: { interactive?: boolean; autoApply?: boolean; exitOnDecline?: boolean } = {}) { + const interactive = options.interactive ?? mode === "watch"; + const autoApply = options.autoApply ?? env.PAPERCLIP_MIGRATION_AUTO_APPLY === "true"; + const exitOnDecline = options.exitOnDecline ?? mode === "watch"; + + const payload = await refreshPendingMigrations(); + if (payload.status !== "needsMigrations" || pendingMigrations.length === 0) { + return; + } + + let shouldApply = autoApply; + + if (!autoApply && interactive) { + if (!stdin.isTTY || !stdout.isTTY) { + shouldApply = true; + } else { + const prompt = createInterface({ input: stdin, output: stdout }); + try { + const answer = ( + await prompt.question( + `Apply pending migrations (${formatPendingMigrationSummary(pendingMigrations)}) now? (y/N): `, + ) + ) + .trim() + .toLowerCase(); + shouldApply = answer === "y" || answer === "yes"; + } finally { + prompt.close(); + } + } + } + + if (!shouldApply) { + if (exitOnDecline) { + process.stderr.write( + `[paperclip] Pending migrations detected (${formatPendingMigrationSummary(pendingMigrations)}). Refusing to start watch mode against a stale schema.\n`, + ); + process.exit(1); + } + return; + } + + const exit = await runPnpm(["db:migrate"], { + stdio: "inherit", + env, + cwd: repoRoot, + }); + if (exit.signal) { + exitForSignal(exit.signal); + return; + } + if (exit.code !== 0) { + process.exit(exit.code); + } + + await refreshPendingMigrations(); +} + +async function buildPluginSdk() { + console.log("[paperclip] building plugin sdk..."); + const result = await runPnpm( + ["--filter", "@paperclipai/plugin-sdk", "build"], + { stdio: "inherit" }, + ); + if (result.signal) { + exitForSignal(result.signal); + return; + } + if (result.code !== 0) { + console.error("[paperclip] plugin sdk build failed"); + process.exit(result.code); + } +} + +async function markChildAsCurrent() { + previousSnapshot = collectWatchedSnapshot(); + dirtyPaths = new Set(); + lastChangedAt = null; + lastRestartAt = new Date().toISOString(); + await refreshPendingMigrations(); + await updateDevServiceRecord(); +} + +async function scanForBackendChanges() { + if (mode !== "dev" || scanInFlight || restartInFlight) return; + scanInFlight = true; + try { + const nextSnapshot = collectWatchedSnapshot(); + const changed = diffSnapshots(previousSnapshot, nextSnapshot); + previousSnapshot = nextSnapshot; + if (changed.length === 0) return; + + for (const relativePath of changed) { + dirtyPaths.add(relativePath); + } + lastChangedAt = new Date().toISOString(); + await refreshPendingMigrations(); + } finally { + scanInFlight = false; + } +} + +async function getDevHealthPayload() { + const response = await fetch(`http://127.0.0.1:${serverPort}/api/health`); + if (!response.ok) { + throw new Error(`Health request failed (${response.status})`); + } + return await response.json(); +} + +async function waitForChildExit() { + if (!childExitPromise) { + return { code: 0, signal: null }; + } + return await childExitPromise; +} + +async function stopChildForRestart() { + if (!child) return { code: 0, signal: null }; + childExitWasExpected = true; + child.kill("SIGTERM"); + const killTimer = setTimeout(() => { + if (child) { + child.kill("SIGKILL"); + } + }, gracefulShutdownTimeoutMs); + try { + return await waitForChildExit(); + } finally { + clearTimeout(killTimer); + } +} + +async function startServerChild() { + await buildPluginSdk(); + + const serverScript = mode === "watch" ? "dev:watch" : "dev"; + child = spawn( + pnpmBin, + ["--filter", "@paperclipai/server", serverScript, ...forwardedArgs], + { stdio: "inherit", env, shell: process.platform === "win32" }, + ); + + childExitPromise = new Promise((resolve, reject) => { + child?.on("error", reject); + child?.on("exit", (code, signal) => { + const expected = childExitWasExpected; + childExitWasExpected = false; + child = null; + childExitPromise = null; + void touchLocalServiceRegistryRecord(devService.serviceKey, { + metadata: { + repoRoot, + mode, + childPid: null, + url: `http://127.0.0.1:${serverPort}`, + }, + }); + resolve({ code: code ?? 0, signal }); + + if (restartInFlight || expected || shuttingDown) { + return; + } + if (signal) { + exitForSignal(signal); + return; + } + process.exit(code ?? 0); + }); + }); + + await markChildAsCurrent(); +} + +async function maybeAutoRestartChild() { + if (mode !== "dev" || restartInFlight || !child) return; + if (dirtyPaths.size === 0 && pendingMigrations.length === 0) return; + + restartInFlight = true; + let health: { devServer?: { enabled?: boolean; autoRestartEnabled?: boolean; activeRunCount?: number } } | null = null; + try { + health = await getDevHealthPayload(); + } catch { + restartInFlight = false; + return; + } + + const devServer = health?.devServer; + if (!devServer?.enabled || devServer.autoRestartEnabled !== true) { + restartInFlight = false; + return; + } + if ((devServer.activeRunCount ?? 0) > 0) { + restartInFlight = false; + return; + } + + try { + await maybePreflightMigrations({ + autoApply: true, + interactive: false, + exitOnDecline: false, + }); + await stopChildForRestart(); + await startServerChild(); + } catch (error) { + const err = toError(error, "Auto-restart failed"); + process.stderr.write(`${err.stack ?? err.message}\n`); + process.exit(1); + } finally { + restartInFlight = false; + } +} + +function installDevIntervals() { + if (mode !== "dev") return; + + scanTimer = setInterval(() => { + void scanForBackendChanges(); + }, scanIntervalMs); + autoRestartTimer = setInterval(() => { + void maybeAutoRestartChild(); + }, autoRestartPollIntervalMs); +} + +function clearDevIntervals() { + if (scanTimer) { + clearInterval(scanTimer); + scanTimer = null; + } + if (autoRestartTimer) { + clearInterval(autoRestartTimer); + autoRestartTimer = null; + } +} + +async function shutdown(signal: NodeJS.Signals) { + if (shuttingDown) return; + shuttingDown = true; + clearDevIntervals(); + clearDevServerStatus(); + await removeLocalServiceRegistryRecord(devService.serviceKey); + + if (!child) { + exitForSignal(signal); + return; + } + + childExitWasExpected = true; + child.kill(signal); + const exit = await waitForChildExit(); + if (exit.signal) { + exitForSignal(exit.signal); + return; + } + process.exit(exit.code ?? 0); +} + +process.on("SIGINT", () => { + void shutdown("SIGINT"); +}); +process.on("SIGTERM", () => { + void shutdown("SIGTERM"); +}); + +await maybePreflightMigrations(); +await startServerChild(); +installDevIntervals(); + +if (mode === "watch") { + const exit = await waitForChildExit(); + await removeLocalServiceRegistryRecord(devService.serviceKey); + if (exit.signal) { + exitForSignal(exit.signal); + } + process.exit(exit.code ?? 0); +} diff --git a/scripts/dev-service-profile.ts b/scripts/dev-service-profile.ts new file mode 100644 index 00000000000..9c129b34a28 --- /dev/null +++ b/scripts/dev-service-profile.ts @@ -0,0 +1,44 @@ +import { createHash } from "node:crypto"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import { createLocalServiceKey } from "../server/src/services/local-service-supervisor.ts"; + +export const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), ".."); + +export function createDevServiceIdentity(input: { + mode: "watch" | "dev"; + forwardedArgs: string[]; + tailscaleAuth: boolean; + port: number; +}) { + const envFingerprint = createHash("sha256") + .update( + JSON.stringify({ + mode: input.mode, + forwardedArgs: input.forwardedArgs, + tailscaleAuth: input.tailscaleAuth, + port: input.port, + }), + ) + .digest("hex"); + + const serviceName = input.mode === "watch" ? "paperclip-dev-watch" : "paperclip-dev-once"; + const serviceKey = createLocalServiceKey({ + profileKind: "paperclip-dev", + serviceName, + cwd: repoRoot, + command: "dev-runner.ts", + envFingerprint, + port: input.port, + scope: { + repoRoot, + mode: input.mode, + }, + }); + + return { + serviceKey, + serviceName, + envFingerprint, + }; +} diff --git a/scripts/dev-service.ts b/scripts/dev-service.ts new file mode 100644 index 00000000000..978607ec3f0 --- /dev/null +++ b/scripts/dev-service.ts @@ -0,0 +1,44 @@ +#!/usr/bin/env -S node --import tsx +import { listLocalServiceRegistryRecords, removeLocalServiceRegistryRecord, terminateLocalService } from "../server/src/services/local-service-supervisor.ts"; +import { repoRoot } from "./dev-service-profile.ts"; + +function toDisplayLines(records: Awaited>) { + return records.map((record) => { + const childPid = typeof record.metadata?.childPid === "number" ? ` child=${record.metadata.childPid}` : ""; + const url = typeof record.metadata?.url === "string" ? ` url=${record.metadata.url}` : ""; + return `${record.serviceName} pid=${record.pid}${childPid} cwd=${record.cwd}${url}`; + }); +} + +const command = process.argv[2] ?? "list"; +const records = await listLocalServiceRegistryRecords({ + profileKind: "paperclip-dev", + metadata: { repoRoot }, +}); + +if (command === "list") { + if (records.length === 0) { + console.log("No Paperclip dev services registered for this repo."); + process.exit(0); + } + for (const line of toDisplayLines(records)) { + console.log(line); + } + process.exit(0); +} + +if (command === "stop") { + if (records.length === 0) { + console.log("No Paperclip dev services registered for this repo."); + process.exit(0); + } + for (const record of records) { + await terminateLocalService(record); + await removeLocalServiceRegistryRecord(record.serviceKey); + console.log(`Stopped ${record.serviceName} (pid ${record.pid})`); + } + process.exit(0); +} + +console.error(`Unknown dev-service command: ${command}`); +process.exit(1); diff --git a/scripts/ensure-workspace-package-links.ts b/scripts/ensure-workspace-package-links.ts new file mode 100644 index 00000000000..d249454565a --- /dev/null +++ b/scripts/ensure-workspace-package-links.ts @@ -0,0 +1,113 @@ +#!/usr/bin/env -S node --import tsx +import { spawn } from "node:child_process"; +import { existsSync, readFileSync, realpathSync } from "node:fs"; +import path from "node:path"; +import { repoRoot } from "./dev-service-profile.ts"; + +type WorkspaceLinkMismatch = { + packageName: string; + expectedPath: string; + actualPath: string | null; +}; + +function readJsonFile(filePath: string): Record { + return JSON.parse(readFileSync(filePath, "utf8")) as Record; +} + +function resolveWorkspacePackagePath(packageName: string): string | null { + if (packageName === "@paperclipai/adapter-utils") { + return path.join(repoRoot, "packages", "adapter-utils"); + } + if (packageName === "@paperclipai/db") { + return path.join(repoRoot, "packages", "db"); + } + if (packageName === "@paperclipai/shared") { + return path.join(repoRoot, "packages", "shared"); + } + if (packageName === "@paperclipai/plugin-sdk") { + return path.join(repoRoot, "packages", "plugins", "sdk"); + } + if (packageName.startsWith("@paperclipai/adapter-")) { + return path.join(repoRoot, "packages", "adapters", packageName.slice("@paperclipai/adapter-".length)); + } + return null; +} + +function findServerWorkspaceLinkMismatches(): WorkspaceLinkMismatch[] { + const serverPackageJson = readJsonFile(path.join(repoRoot, "server", "package.json")); + const dependencies = { + ...(serverPackageJson.dependencies as Record | undefined), + ...(serverPackageJson.devDependencies as Record | undefined), + }; + const mismatches: WorkspaceLinkMismatch[] = []; + + for (const [packageName, version] of Object.entries(dependencies)) { + if (typeof version !== "string" || !version.startsWith("workspace:")) continue; + + const expectedPath = resolveWorkspacePackagePath(packageName); + if (!expectedPath) continue; + + const linkPath = path.join(repoRoot, "server", "node_modules", ...packageName.split("/")); + const actualPath = existsSync(linkPath) ? path.resolve(realpathSync(linkPath)) : null; + if (actualPath === path.resolve(expectedPath)) continue; + + mismatches.push({ + packageName, + expectedPath: path.resolve(expectedPath), + actualPath, + }); + } + + return mismatches; +} + +function runCommand(command: string, args: string[], cwd: string) { + return new Promise((resolve, reject) => { + const child = spawn(command, args, { + cwd, + env: process.env, + stdio: "inherit", + }); + + child.on("error", reject); + child.on("exit", (code, signal) => { + if (code === 0) { + resolve(); + return; + } + reject( + new Error( + `${command} ${args.join(" ")} failed with ${signal ? `signal ${signal}` : `exit code ${code ?? "unknown"}`}`, + ), + ); + }); + }); +} + +async function ensureServerWorkspaceLinksCurrent() { + const mismatches = findServerWorkspaceLinkMismatches(); + if (mismatches.length === 0) return; + + console.log("[paperclip] detected stale workspace package links for server; relinking dependencies..."); + for (const mismatch of mismatches) { + console.log( + `[paperclip] ${mismatch.packageName}: ${mismatch.actualPath ?? "missing"} -> ${mismatch.expectedPath}`, + ); + } + + const pnpmBin = process.platform === "win32" ? "pnpm.cmd" : "pnpm"; + await runCommand( + pnpmBin, + ["install", "--force", "--config.confirmModulesPurge=false"], + repoRoot, + ); + + const remainingMismatches = findServerWorkspaceLinkMismatches(); + if (remainingMismatches.length === 0) return; + + throw new Error( + `Workspace relink did not repair all server package links: ${remainingMismatches.map((item) => item.packageName).join(", ")}`, + ); +} + +await ensureServerWorkspaceLinksCurrent(); diff --git a/scripts/kill-dev.sh b/scripts/kill-dev.sh index 2cb946e29bb..9a53498a4ff 100755 --- a/scripts/kill-dev.sh +++ b/scripts/kill-dev.sh @@ -8,64 +8,199 @@ # set -euo pipefail +shopt -s nullglob DRY_RUN=false if [[ "${1:-}" == "--dry" || "${1:-}" == "--dry-run" || "${1:-}" == "-n" ]]; then DRY_RUN=true fi -# Collect PIDs of node processes running from any paperclip directory. -# Matches paths like /Users/*/paperclip/... or /Users/*/paperclip-*/... -# Excludes postgres-related processes. -pids=() -lines=() +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" +REPO_PARENT="$(dirname "$REPO_ROOT")" + +node_pids=() +node_lines=() +pg_pids=() +pg_pidfiles=() +pg_data_dirs=() + +is_pid_running() { + local pid="$1" + kill -0 "$pid" 2>/dev/null +} + +read_pidfile_pid() { + local pidfile="$1" + local first_line + first_line="$(head -n 1 "$pidfile" 2>/dev/null | tr -d '[:space:]' || true)" + if [[ "$first_line" =~ ^[0-9]+$ ]] && (( first_line > 0 )); then + printf '%s\n' "$first_line" + return 0 + fi + return 1 +} + +command_for_pid() { + local pid="$1" + ps -o command= -p "$pid" 2>/dev/null || true +} + +append_postgres_from_pidfile() { + local pidfile="$1" + local pid cmd data_dir + pid="$(read_pidfile_pid "$pidfile" || true)" + [[ -n "$pid" ]] || return 0 + is_pid_running "$pid" || return 0 + cmd="$(command_for_pid "$pid")" + [[ "$cmd" == *postgres* ]] || return 0 + + for existing_pid in "${pg_pids[@]:-}"; do + [[ "$existing_pid" == "$pid" ]] && return 0 + done + + data_dir="$(dirname "$pidfile")" + pg_pids+=("$pid") + pg_pidfiles+=("$pidfile") + pg_data_dirs+=("$data_dir") +} + +wait_for_pid_exit() { + local pid="$1" + local timeout_sec="$2" + local waited=0 + while is_pid_running "$pid"; do + if (( waited >= timeout_sec * 10 )); then + return 1 + fi + sleep 0.1 + ((waited += 1)) + done + return 0 +} while IFS= read -r line; do [[ -z "$line" ]] && continue - # skip postgres processes [[ "$line" == *postgres* ]] && continue pid=$(echo "$line" | awk '{print $2}') - pids+=("$pid") - lines+=("$line") + node_pids+=("$pid") + node_lines+=("$line") done < <(ps aux | grep -E '/paperclip(-[^/]+)?/' | grep node | grep -v grep || true) -if [[ ${#pids[@]} -eq 0 ]]; then +candidate_pidfiles=() +candidate_pidfiles+=( + "$HOME"/.paperclip/instances/*/db/postmaster.pid + "$REPO_ROOT"/.paperclip/instances/*/db/postmaster.pid + "$REPO_ROOT"/.paperclip/runtime-services/instances/*/db/postmaster.pid +) + +for sibling_root in "$REPO_PARENT"/paperclip*; do + [[ -d "$sibling_root" ]] || continue + candidate_pidfiles+=( + "$sibling_root"/.paperclip/instances/*/db/postmaster.pid + "$sibling_root"/.paperclip/runtime-services/instances/*/db/postmaster.pid + ) +done + +for pidfile in "${candidate_pidfiles[@]:-}"; do + [[ -f "$pidfile" ]] || continue + append_postgres_from_pidfile "$pidfile" +done + +if [[ ${#node_pids[@]} -eq 0 && ${#pg_pids[@]} -eq 0 ]]; then echo "No Paperclip dev processes found." exit 0 fi -echo "Found ${#pids[@]} Paperclip dev process(es):" -echo "" +if [[ ${#node_pids[@]} -gt 0 ]]; then + echo "Found ${#node_pids[@]} Paperclip dev node process(es):" + echo "" -for i in "${!pids[@]}"; do - line="${lines[$i]}" - pid=$(echo "$line" | awk '{print $2}') - start=$(echo "$line" | awk '{print $9}') - cmd=$(echo "$line" | awk '{for(i=11;i<=NF;i++) printf "%s ", $i; print ""}') - # Shorten the command for readability - cmd=$(echo "$cmd" | sed "s|$HOME/||g") - printf " PID %-7s started %-10s %s\n" "$pid" "$start" "$cmd" -done + for i in "${!node_pids[@]:-}"; do + line="${node_lines[$i]}" + pid=$(echo "$line" | awk '{print $2}') + start=$(echo "$line" | awk '{print $9}') + cmd=$(echo "$line" | awk '{for(i=11;i<=NF;i++) printf "%s ", $i; print ""}') + cmd=$(echo "$cmd" | sed "s|$HOME/||g") + printf " PID %-7s started %-10s %s\n" "$pid" "$start" "$cmd" + done -echo "" + echo "" +fi + +if [[ ${#pg_pids[@]} -gt 0 ]]; then + echo "Found ${#pg_pids[@]} embedded PostgreSQL master process(es):" + echo "" + + for i in "${!pg_pids[@]:-}"; do + pid="${pg_pids[$i]}" + data_dir="${pg_data_dirs[$i]}" + pidfile="${pg_pidfiles[$i]}" + short_data_dir="${data_dir/#$HOME\//}" + short_pidfile="${pidfile/#$HOME\//}" + printf " PID %-7s data %-55s pidfile %s\n" "$pid" "$short_data_dir" "$short_pidfile" + done + + echo "" +fi if [[ "$DRY_RUN" == true ]]; then echo "Dry run — re-run without --dry to kill these processes." exit 0 fi -echo "Sending SIGTERM..." -for pid in "${pids[@]}"; do - kill "$pid" 2>/dev/null && echo " killed $pid" || echo " $pid already gone" -done +if [[ ${#node_pids[@]} -gt 0 ]]; then + echo "Sending SIGTERM to Paperclip node processes..." + for pid in "${node_pids[@]}"; do + kill -TERM "$pid" 2>/dev/null && echo " signaled $pid" || echo " $pid already gone" + done + echo "Waiting briefly for node processes to exit..." + sleep 2 +fi -# Give processes a moment to exit, then SIGKILL any stragglers -sleep 2 -for pid in "${pids[@]}"; do - if kill -0 "$pid" 2>/dev/null; then - echo " $pid still alive, sending SIGKILL..." - kill -9 "$pid" 2>/dev/null || true +leftover_pg_pids=() +leftover_pg_data_dirs=() +for i in "${!pg_pids[@]:-}"; do + pid="${pg_pids[$i]}" + if is_pid_running "$pid"; then + leftover_pg_pids+=("$pid") + leftover_pg_data_dirs+=("${pg_data_dirs[$i]}") fi done +if [[ ${#leftover_pg_pids[@]} -gt 0 ]]; then + echo "Sending SIGTERM to leftover embedded PostgreSQL processes..." + for i in "${!leftover_pg_pids[@]:-}"; do + pid="${leftover_pg_pids[$i]}" + data_dir="${leftover_pg_data_dirs[$i]}" + kill -TERM "$pid" 2>/dev/null \ + && echo " signaled $pid ($data_dir)" \ + || echo " $pid already gone" + done + echo "Waiting up to 15s for PostgreSQL to shut down cleanly..." + for pid in "${leftover_pg_pids[@]:-}"; do + if wait_for_pid_exit "$pid" 15; then + echo " postgres $pid exited cleanly" + fi + done +fi + +if [[ ${#node_pids[@]} -gt 0 ]]; then + for pid in "${node_pids[@]:-}"; do + if kill -0 "$pid" 2>/dev/null; then + echo " node $pid still alive, sending SIGKILL..." + kill -KILL "$pid" 2>/dev/null || true + fi + done +fi + +if [[ ${#pg_pids[@]} -gt 0 ]]; then + for pid in "${pg_pids[@]:-}"; do + if kill -0 "$pid" 2>/dev/null; then + echo " postgres $pid still alive, sending SIGKILL..." + kill -KILL "$pid" 2>/dev/null || true + fi + done +fi + echo "Done." diff --git a/scripts/paperclip-commit-metrics.ts b/scripts/paperclip-commit-metrics.ts new file mode 100644 index 00000000000..5f989ca1db2 --- /dev/null +++ b/scripts/paperclip-commit-metrics.ts @@ -0,0 +1,872 @@ +#!/usr/bin/env npx tsx + +import { execFile } from "node:child_process"; +import { promises as fs } from "node:fs"; +import path from "node:path"; +import { promisify } from "node:util"; + +const execFileAsync = promisify(execFile); + +const DEFAULT_QUERY = "\"Co-Authored-By: Paperclip \""; +const DEFAULT_CACHE_FILE = path.resolve("data/paperclip-commit-metrics-cache.json"); +const DEFAULT_SEARCH_START = "2008-01-01T00:00:00Z"; +const SEARCH_WINDOW_LIMIT = 900; +const MIN_WINDOW_MS = 60_000; +const DEFAULT_STATS_FETCH_LIMIT = 250; +const DEFAULT_STATS_CONCURRENCY = 4; +const DEFAULT_SEARCH_FIELD = "committer-date"; +const PAPERCLIP_EMAIL = "noreply@paperclip.ing"; +const PAPERCLIP_NAME = "paperclip"; + +interface CliOptions { + cacheFile: string; + end: Date; + excludeOwners: string[]; + exportFormat: "csv" | "json"; + includePrivate: boolean; + json: boolean; + output: string | null; + query: string; + refreshSearch: boolean; + refreshStats: boolean; + searchField: "author-date" | "committer-date"; + start: Date; + statsConcurrency: number; + statsFetchLimit: number; + skipStats: boolean; +} + +interface SearchCommitItem { + author: { + login?: string; + } | null; + commit: { + author: { + date: string; + email: string | null; + name: string | null; + } | null; + message: string; + }; + html_url: string; + repository: { + full_name: string; + html_url: string; + }; + sha: string; +} + +interface CommitStats { + additions: number; + deletions: number; + total: number; +} + +interface CachedCommit { + authorEmail: string | null; + authorLogin: string | null; + authorName: string | null; + committedAt: string | null; + contributors: ContributorRecord[]; + htmlUrl: string; + repositoryFullName: string; + repositoryUrl: string; + sha: string; +} + +interface CachedCommitStats extends CommitStats { + fetchedAt: string; +} + +interface ContributorRecord { + displayName: string; + email: string | null; + key: string; + login: string | null; +} + +interface WindowCacheEntry { + completedAt: string; + key: string; + shas: string[]; + totalCount: number; +} + +interface CacheFile { + commits: Record; + queryKey: string; + searchField: CliOptions["searchField"]; + stats: Record; + updatedAt: string | null; + version: number; + windows: Record; +} + +interface SearchResponse { + incomplete_results: boolean; + items: SearchCommitItem[]; + total_count: number; +} + +interface SearchWindowResult { + shas: Set; + totalCount: number; +} + +interface Summary { + cacheFile: string; + contributors: { + count: number; + sample: ContributorRecord[]; + }; + detectedQuery: string; + lineStats: { + additions: number; + complete: boolean; + coveredCommits: number; + deletions: number; + missingCommits: number; + totalChanges: number; + }; + range: { + end: string; + searchField: CliOptions["searchField"]; + start: string; + }; + filters: { + excludedOwners: string[]; + }; + repos: { + count: number; + sample: string[]; + }; + statsFetch: { + fetchedThisRun: number; + skipped: boolean; + }; + totals: { + commits: number; + }; +} + +async function main() { + const options = parseArgs(process.argv.slice(2)); + const cache = await loadCache(options.cacheFile, options); + const client = new GitHubClient(await resolveGitHubToken()); + + const { shas } = await searchWindow(client, cache, options, options.start, options.end); + const sortedShas = [...shas].sort(); + + let fetchedThisRun = 0; + if (!options.skipStats) { + fetchedThisRun = await enrichCommitStats(client, cache, options, sortedShas); + } + + cache.updatedAt = new Date().toISOString(); + await saveCache(options.cacheFile, cache); + + const filteredShas = sortFilteredShas(cache, filterShas(cache, sortedShas, options)); + const summary = buildSummary(cache, options, filteredShas, fetchedThisRun); + + if (options.output) { + await writeExport(options.output, options.exportFormat, cache, filteredShas, summary); + } + + if (options.json) { + console.log(JSON.stringify(summary, null, 2)); + return; + } + + printSummary(summary); +} + +function parseArgs(argv: string[]): CliOptions { + const options: CliOptions = { + cacheFile: DEFAULT_CACHE_FILE, + end: new Date(), + excludeOwners: [], + exportFormat: "csv", + includePrivate: false, + json: false, + output: null, + query: DEFAULT_QUERY, + refreshSearch: false, + refreshStats: false, + searchField: DEFAULT_SEARCH_FIELD, + start: new Date(DEFAULT_SEARCH_START), + statsConcurrency: DEFAULT_STATS_CONCURRENCY, + statsFetchLimit: DEFAULT_STATS_FETCH_LIMIT, + skipStats: false, + }; + + for (let index = 0; index < argv.length; index += 1) { + const arg = argv[index]; + switch (arg) { + case "--cache-file": + options.cacheFile = requireValue(argv, ++index, arg); + break; + case "--end": + options.end = parseDateArg(requireValue(argv, ++index, arg), arg); + break; + case "--exclude-owner": + options.excludeOwners.push(requireValue(argv, ++index, arg).toLowerCase()); + break; + case "--export-format": { + const value = requireValue(argv, ++index, arg); + if (value !== "csv" && value !== "json") { + throw new Error(`Invalid --export-format value: ${value}`); + } + options.exportFormat = value; + break; + } + case "--include-private": + options.includePrivate = true; + break; + case "--json": + options.json = true; + break; + case "--output": + options.output = requireValue(argv, ++index, arg); + break; + case "--query": + options.query = requireValue(argv, ++index, arg); + break; + case "--refresh-search": + options.refreshSearch = true; + break; + case "--refresh-stats": + options.refreshStats = true; + break; + case "--search-field": { + const value = requireValue(argv, ++index, arg); + if (value !== "author-date" && value !== "committer-date") { + throw new Error(`Invalid --search-field value: ${value}`); + } + options.searchField = value; + break; + } + case "--skip-stats": + options.skipStats = true; + break; + case "--start": + options.start = parseDateArg(requireValue(argv, ++index, arg), arg); + break; + case "--stats-concurrency": + options.statsConcurrency = parsePositiveInt(requireValue(argv, ++index, arg), arg); + break; + case "--stats-fetch-limit": + options.statsFetchLimit = parseNonNegativeInt(requireValue(argv, ++index, arg), arg); + break; + case "--help": + printHelp(); + process.exit(0); + break; + default: + throw new Error(`Unknown argument: ${arg}`); + } + } + + if (Number.isNaN(options.start.getTime()) || Number.isNaN(options.end.getTime())) { + throw new Error("Invalid start or end date"); + } + if (options.start >= options.end) { + throw new Error("--start must be earlier than --end"); + } + + return options; +} + +function requireValue(argv: string[], index: number, flag: string): string { + const value = argv[index]; + if (!value) { + throw new Error(`Missing value for ${flag}`); + } + return value; +} + +function parseDateArg(value: string, flag: string): Date { + const parsed = new Date(value); + if (Number.isNaN(parsed.getTime())) { + throw new Error(`Invalid date for ${flag}: ${value}`); + } + return parsed; +} + +function parsePositiveInt(value: string, flag: string): number { + const parsed = Number.parseInt(value, 10); + if (!Number.isFinite(parsed) || parsed <= 0) { + throw new Error(`Invalid positive integer for ${flag}: ${value}`); + } + return parsed; +} + +function parseNonNegativeInt(value: string, flag: string): number { + const parsed = Number.parseInt(value, 10); + if (!Number.isFinite(parsed) || parsed < 0) { + throw new Error(`Invalid non-negative integer for ${flag}: ${value}`); + } + return parsed; +} + +function printHelp() { + console.log(`Usage: tsx scripts/paperclip-commit-metrics.ts [options] + +Options: + --start ISO date/time lower bound (default: ${DEFAULT_SEARCH_START}) + --end ISO date/time upper bound (default: now) + --query Commit search string (default: ${DEFAULT_QUERY}) + --search-field author-date | committer-date (default: ${DEFAULT_SEARCH_FIELD}) + --include-private Include repos visible to the current token + --exclude-owner Exclude repositories owned by this GitHub owner/org (repeatable) + --cache-file Cache path (default: ${DEFAULT_CACHE_FILE}) + --skip-stats Skip additions/deletions enrichment + --stats-fetch-limit Max uncached commit stats to fetch this run (default: ${DEFAULT_STATS_FETCH_LIMIT}) + --stats-concurrency Parallel commit stat requests (default: ${DEFAULT_STATS_CONCURRENCY}) + --output Write the full filtered result set to a file + --export-format csv | json for --output exports (default: csv) + --refresh-search Ignore cached search windows + --refresh-stats Re-fetch cached commit stats + --json Print JSON summary + --help Show this help +`); +} + +async function resolveGitHubToken(): Promise { + const envToken = process.env.GITHUB_TOKEN ?? process.env.GH_TOKEN; + if (envToken) { + return envToken; + } + + const { stdout } = await execFileAsync("gh", ["auth", "token"]); + const token = stdout.trim(); + if (!token) { + throw new Error("Unable to resolve a GitHub token. Set GITHUB_TOKEN/GH_TOKEN or run `gh auth login`."); + } + return token; +} + +async function loadCache(cacheFile: string, options: CliOptions): Promise { + try { + const raw = await fs.readFile(cacheFile, "utf8"); + const parsed = JSON.parse(raw) as CacheFile; + if (parsed.version !== 1 || parsed.queryKey !== buildQueryKey(options) || parsed.searchField !== options.searchField) { + return createEmptyCache(options); + } + return parsed; + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") { + return createEmptyCache(options); + } + throw error; + } +} + +function createEmptyCache(options: CliOptions): CacheFile { + return { + commits: {}, + queryKey: buildQueryKey(options), + searchField: options.searchField, + stats: {}, + updatedAt: null, + version: 1, + windows: {}, + }; +} + +function buildQueryKey(options: CliOptions): string { + const visibility = options.includePrivate ? "all" : "public"; + return JSON.stringify({ + query: options.query, + searchField: options.searchField, + visibility, + }); +} + +async function saveCache(cacheFile: string, cache: CacheFile): Promise { + await fs.mkdir(path.dirname(cacheFile), { recursive: true }); + await fs.writeFile(cacheFile, JSON.stringify(cache, null, 2), "utf8"); +} + +async function searchWindow( + client: GitHubClient, + cache: CacheFile, + options: CliOptions, + start: Date, + end: Date, +): Promise { + const windowKey = makeWindowKey(start, end); + if (!options.refreshSearch) { + const cached = cache.windows[windowKey]; + if (cached) { + return { shas: new Set(cached.shas), totalCount: cached.totalCount }; + } + } + + const firstPage = await searchPage(client, options, start, end, 1, 100); + if (firstPage.incomplete_results) { + throw new Error(`GitHub returned incomplete search results for window ${windowKey}`); + } + + if (firstPage.total_count > SEARCH_WINDOW_LIMIT) { + const durationMs = end.getTime() - start.getTime(); + if (durationMs <= MIN_WINDOW_MS) { + throw new Error( + `Search window ${windowKey} still has ${firstPage.total_count} results after splitting to ${durationMs}ms.`, + ); + } + + const midpoint = new Date(start.getTime() + Math.floor(durationMs / 2)); + const left = await searchWindow(client, cache, options, start, midpoint); + const right = await searchWindow(client, cache, options, new Date(midpoint.getTime() + 1), end); + const shas = new Set([...left.shas, ...right.shas]); + + cache.windows[windowKey] = { + completedAt: new Date().toISOString(), + key: windowKey, + shas: [...shas], + totalCount: shas.size, + }; + + return { shas, totalCount: shas.size }; + } + + const pageCount = Math.ceil(firstPage.total_count / 100); + const shas = new Set(); + ingestSearchItems(cache, firstPage.items, shas); + + for (let page = 2; page <= pageCount; page += 1) { + const response = await searchPage(client, options, start, end, page, 100); + ingestSearchItems(cache, response.items, shas); + } + + cache.windows[windowKey] = { + completedAt: new Date().toISOString(), + key: windowKey, + shas: [...shas], + totalCount: firstPage.total_count, + }; + + return { shas, totalCount: firstPage.total_count }; +} + +async function searchPage( + client: GitHubClient, + options: CliOptions, + start: Date, + end: Date, + page: number, + perPage: number, +): Promise { + const searchQuery = buildSearchQuery(options, start, end); + const params = new URLSearchParams({ + page: String(page), + per_page: String(perPage), + q: searchQuery, + }); + + return client.getJson(`/search/commits?${params.toString()}`); +} + +function buildSearchQuery(options: CliOptions, start: Date, end: Date): string { + const qualifiers = [`${options.searchField}:${formatQueryDate(start)}..${formatQueryDate(end)}`]; + if (!options.includePrivate) { + qualifiers.push("is:public"); + } + return `${options.query} ${qualifiers.join(" ")}`.trim(); +} + +function filterShas(cache: CacheFile, shas: string[], options: CliOptions): string[] { + if (options.excludeOwners.length === 0) { + return shas; + } + + const excludedOwners = new Set(options.excludeOwners); + return shas.filter((sha) => { + const commit = cache.commits[sha]; + if (!commit) { + return false; + } + return !excludedOwners.has(getRepoOwner(commit.repositoryFullName)); + }); +} + +function sortFilteredShas(cache: CacheFile, shas: string[]): string[] { + return [...shas].sort((leftSha, rightSha) => { + const left = cache.commits[leftSha]; + const right = cache.commits[rightSha]; + const leftTime = left?.committedAt ? Date.parse(left.committedAt) : 0; + const rightTime = right?.committedAt ? Date.parse(right.committedAt) : 0; + if (rightTime !== leftTime) { + return rightTime - leftTime; + } + + const repoCompare = (left?.repositoryFullName ?? "").localeCompare(right?.repositoryFullName ?? ""); + if (repoCompare !== 0) { + return repoCompare; + } + return leftSha.localeCompare(rightSha); + }); +} + +function formatQueryDate(value: Date): string { + return value.toISOString().replace(".000Z", "Z"); +} + +function ingestSearchItems(cache: CacheFile, items: SearchCommitItem[], shas: Set) { + for (const item of items) { + shas.add(item.sha); + cache.commits[item.sha] = { + authorEmail: item.commit.author?.email ?? null, + authorLogin: item.author?.login ?? null, + authorName: item.commit.author?.name ?? null, + committedAt: item.commit.author?.date ?? null, + contributors: extractContributors(item), + htmlUrl: item.html_url, + repositoryFullName: item.repository.full_name, + repositoryUrl: item.repository.html_url, + sha: item.sha, + }; + } +} + +function extractContributors(item: SearchCommitItem): ContributorRecord[] { + const contributors = new Map(); + + const primaryAuthor = normalizeContributor({ + email: item.commit.author?.email ?? null, + login: item.author?.login ?? null, + name: item.commit.author?.name ?? null, + }); + if (primaryAuthor) { + contributors.set(primaryAuthor.key, primaryAuthor); + } + + const coAuthorPattern = /^co-authored-by:\s*(.+?)\s*<([^>]+)>\s*$/gim; + for (const match of item.commit.message.matchAll(coAuthorPattern)) { + const contributor = normalizeContributor({ + email: match[2] ?? null, + login: null, + name: match[1] ?? null, + }); + if (contributor) { + contributors.set(contributor.key, contributor); + } + } + + return [...contributors.values()]; +} + +function normalizeContributor(input: { + email: string | null; + login: string | null; + name: string | null; +}): ContributorRecord | null { + const email = normalizeOptional(input.email); + const login = normalizeOptional(input.login); + const displayName = normalizeOptional(input.name) ?? login ?? email; + + if (!displayName && !email && !login) { + return null; + } + if ((email && email === PAPERCLIP_EMAIL) || (displayName && displayName.toLowerCase() === PAPERCLIP_NAME)) { + return null; + } + + const key = login ? `login:${login}` : email ? `email:${email}` : `name:${displayName!.toLowerCase()}`; + return { + displayName: displayName ?? email ?? login ?? "unknown", + email, + key, + login, + }; +} + +function normalizeOptional(value: string | null | undefined): string | null { + const trimmed = value?.trim(); + return trimmed ? trimmed : null; +} + +function getRepoOwner(repositoryFullName: string): string { + return repositoryFullName.split("/", 1)[0]?.toLowerCase() ?? ""; +} + +async function enrichCommitStats( + client: GitHubClient, + cache: CacheFile, + options: CliOptions, + shas: string[], +): Promise { + const pending = shas.filter((sha) => options.refreshStats || !cache.stats[sha]).slice(0, options.statsFetchLimit); + let nextIndex = 0; + let fetched = 0; + + const workers = Array.from({ length: Math.min(options.statsConcurrency, pending.length) }, async () => { + while (true) { + const currentIndex = nextIndex; + nextIndex += 1; + const sha = pending[currentIndex]; + if (!sha) { + return; + } + const commit = cache.commits[sha]; + if (!commit) { + continue; + } + const stats = await fetchCommitStats(client, commit.repositoryFullName, sha); + cache.stats[sha] = { + ...stats, + fetchedAt: new Date().toISOString(), + }; + fetched += 1; + } + }); + + await Promise.all(workers); + return fetched; +} + +async function fetchCommitStats(client: GitHubClient, repositoryFullName: string, sha: string): Promise { + const response = await client.getJson<{ stats?: CommitStats }>( + `/repos/${repositoryFullName}/commits/${sha}`, + ); + return { + additions: response.stats?.additions ?? 0, + deletions: response.stats?.deletions ?? 0, + total: response.stats?.total ?? 0, + }; +} + +function buildSummary(cache: CacheFile, options: CliOptions, shas: string[], fetchedThisRun: number): Summary { + const repoNames = new Set(); + const contributors = new Map(); + let additions = 0; + let deletions = 0; + let coveredCommits = 0; + + for (const sha of shas) { + const commit = cache.commits[sha]; + if (!commit) { + continue; + } + repoNames.add(commit.repositoryFullName); + for (const contributor of commit.contributors) { + contributors.set(contributor.key, contributor); + } + + const stats = cache.stats[sha]; + if (stats) { + additions += stats.additions; + deletions += stats.deletions; + coveredCommits += 1; + } + } + + const contributorSample = [...contributors.values()] + .sort((left, right) => left.displayName.localeCompare(right.displayName)) + .slice(0, 10); + const repoSample = [...repoNames].sort((left, right) => left.localeCompare(right)).slice(0, 10); + + return { + cacheFile: options.cacheFile, + contributors: { + count: contributors.size, + sample: contributorSample, + }, + detectedQuery: buildSearchQuery(options, options.start, options.end), + lineStats: { + additions, + complete: coveredCommits === shas.length, + coveredCommits, + deletions, + missingCommits: shas.length - coveredCommits, + totalChanges: additions + deletions, + }, + range: { + end: options.end.toISOString(), + searchField: options.searchField, + start: options.start.toISOString(), + }, + filters: { + excludedOwners: [...options.excludeOwners].sort(), + }, + repos: { + count: repoNames.size, + sample: repoSample, + }, + statsFetch: { + fetchedThisRun, + skipped: options.skipStats, + }, + totals: { + commits: shas.length, + }, + }; +} + +function printSummary(summary: Summary) { + console.log("Paperclip commit metrics"); + console.log(`Query: ${summary.detectedQuery}`); + console.log(`Range: ${summary.range.start} -> ${summary.range.end} (${summary.range.searchField})`); + if (summary.filters.excludedOwners.length > 0) { + console.log(`Excluded owners: ${summary.filters.excludedOwners.join(", ")}`); + } + console.log(`Commits: ${summary.totals.commits}`); + console.log(`Distinct repos: ${summary.repos.count}`); + console.log(`Distinct contributors: ${summary.contributors.count}`); + console.log( + `Line stats: +${summary.lineStats.additions} / -${summary.lineStats.deletions} / ${summary.lineStats.totalChanges} total`, + ); + console.log( + `Line stat coverage: ${summary.lineStats.coveredCommits}/${summary.totals.commits}` + + (summary.lineStats.complete ? " (complete)" : " (partial; rerun to hydrate more commits)"), + ); + console.log(`Stats fetched this run: ${summary.statsFetch.fetchedThisRun}${summary.statsFetch.skipped ? " (skipped)" : ""}`); + console.log(`Cache: ${summary.cacheFile}`); + + if (summary.repos.sample.length > 0) { + console.log(`Sample repos: ${summary.repos.sample.join(", ")}`); + } + if (summary.contributors.sample.length > 0) { + console.log( + `Sample contributors: ${summary.contributors.sample + .map((contributor) => contributor.login ?? contributor.displayName) + .join(", ")}`, + ); + } +} + +async function writeExport( + outputPath: string, + format: CliOptions["exportFormat"], + cache: CacheFile, + shas: string[], + summary: Summary, +): Promise { + await fs.mkdir(path.dirname(outputPath), { recursive: true }); + if (format === "json") { + const report = { + summary, + commits: shas.map((sha) => buildExportRow(cache, sha)), + }; + await fs.writeFile(outputPath, JSON.stringify(report, null, 2), "utf8"); + return; + } + + const header = [ + "committedAt", + "repository", + "repositoryUrl", + "sha", + "commitUrl", + "authorLogin", + "authorName", + "authorEmail", + "contributors", + "additions", + "deletions", + "totalChanges", + ]; + const rows = [header.join(",")]; + for (const sha of shas) { + const row = buildExportRow(cache, sha); + rows.push( + [ + row.committedAt, + row.repository, + row.repositoryUrl, + row.sha, + row.commitUrl, + row.authorLogin, + row.authorName, + row.authorEmail, + row.contributors, + String(row.additions), + String(row.deletions), + String(row.totalChanges), + ] + .map(escapeCsv) + .join(","), + ); + } + await fs.writeFile(outputPath, `${rows.join("\n")}\n`, "utf8"); +} + +function buildExportRow(cache: CacheFile, sha: string) { + const commit = cache.commits[sha]; + if (!commit) { + throw new Error(`Missing cached commit for sha ${sha}`); + } + const stats = cache.stats[sha]; + return { + additions: stats?.additions ?? 0, + authorEmail: commit.authorEmail ?? "", + authorLogin: commit.authorLogin ?? "", + authorName: commit.authorName ?? "", + commitUrl: commit.htmlUrl, + committedAt: commit.committedAt ?? "", + contributors: commit.contributors.map((contributor) => contributor.login ?? contributor.displayName).join(" | "), + deletions: stats?.deletions ?? 0, + repository: commit.repositoryFullName, + repositoryUrl: commit.repositoryUrl, + sha: commit.sha, + totalChanges: stats?.total ?? 0, + }; +} + +function escapeCsv(value: string): string { + if (value.includes(",") || value.includes("\"") || value.includes("\n")) { + return `"${value.replaceAll("\"", "\"\"")}"`; + } + return value; +} + +function makeWindowKey(start: Date, end: Date): string { + return `${start.toISOString()}..${end.toISOString()}`; +} + +class GitHubClient { + private readonly apiBase = "https://api.github.com"; + private readonly token: string; + + constructor(token: string) { + this.token = token; + } + + async getJson(pathname: string): Promise { + while (true) { + const response = await fetch(`${this.apiBase}${pathname}`, { + headers: { + Accept: "application/vnd.github+json", + Authorization: `Bearer ${this.token}`, + "User-Agent": "paperclip-commit-metrics", + "X-GitHub-Api-Version": "2022-11-28", + }, + }); + + if (response.ok) { + return (await response.json()) as T; + } + + const remaining = response.headers.get("x-ratelimit-remaining"); + const resetAt = response.headers.get("x-ratelimit-reset"); + if ((response.status === 403 || response.status === 429) && remaining === "0" && resetAt) { + const waitMs = Math.max(Number.parseInt(resetAt, 10) * 1000 - Date.now() + 1_000, 1_000); + console.error(`GitHub rate limit hit for ${pathname}; waiting ${Math.ceil(waitMs / 1000)}s...`); + await sleep(waitMs); + continue; + } + + const body = await response.text(); + throw new Error(`GitHub API request failed (${response.status}) for ${pathname}: ${body}`); + } + } +} + +function sleep(ms: number): Promise { + return new Promise((resolve) => setTimeout(resolve, ms)); +} + +main().catch((error) => { + console.error(error instanceof Error ? error.message : String(error)); + process.exit(1); +}); diff --git a/server/package.json b/server/package.json index 0f7efa44c21..b2d17ad3f12 100644 --- a/server/package.json +++ b/server/package.json @@ -32,15 +32,16 @@ "skills" ], "scripts": { - "dev": "tsx src/index.ts", - "dev:watch": "cross-env PAPERCLIP_MIGRATION_PROMPT=never PAPERCLIP_MIGRATION_AUTO_APPLY=true tsx ./scripts/dev-watch.ts", + "preflight:workspace-links": "tsx ../scripts/ensure-workspace-package-links.ts", + "dev": "pnpm run preflight:workspace-links && tsx src/index.ts", + "dev:watch": "pnpm run preflight:workspace-links && cross-env PAPERCLIP_MIGRATION_PROMPT=never PAPERCLIP_MIGRATION_AUTO_APPLY=true tsx ./scripts/dev-watch.ts", "prepare:ui-dist": "bash ../scripts/prepare-server-ui-dist.sh", - "build": "tsc && mkdir -p dist/onboarding-assets && cp -R src/onboarding-assets/. dist/onboarding-assets/", + "build": "pnpm run preflight:workspace-links && tsc && mkdir -p dist/onboarding-assets && cp -R src/onboarding-assets/. dist/onboarding-assets/", "prepack": "pnpm run prepare:ui-dist", "postpack": "rm -rf ui-dist", "clean": "rm -rf dist", "start": "node dist/index.js", - "typecheck": "pnpm --filter @paperclipai/plugin-sdk build && tsc --noEmit" + "typecheck": "pnpm run preflight:workspace-links && pnpm --filter @paperclipai/plugin-sdk build && tsc --noEmit" }, "dependencies": { "@aws-sdk/client-s3": "^3.888.0", diff --git a/server/src/__tests__/agent-permissions-routes.test.ts b/server/src/__tests__/agent-permissions-routes.test.ts index 08941f77b8d..7bd79f76a17 100644 --- a/server/src/__tests__/agent-permissions-routes.test.ts +++ b/server/src/__tests__/agent-permissions-routes.test.ts @@ -1,6 +1,7 @@ import express from "express"; import request from "supertest"; import { beforeEach, describe, expect, it, vi } from "vitest"; +import { INBOX_MINE_ISSUE_STATUS_FILTER } from "@paperclipai/shared"; import { agentRoutes } from "../routes/agents.js"; import { errorHandler } from "../middleware/index.js"; @@ -272,4 +273,42 @@ describe("agent permission routes", () => { expect(res.body.access.canAssignTasks).toBe(true); expect(res.body.access.taskAssignSource).toBe("agent_creator"); }); + + it("exposes a dedicated agent route for the inbox mine view", async () => { + mockIssueService.list.mockResolvedValue([ + { + id: "issue-1", + identifier: "PAP-910", + title: "Inbox follow-up", + status: "todo", + }, + ]); + + const app = createApp({ + type: "agent", + agentId, + companyId, + runId: "run-1", + source: "agent_key", + }); + + const res = await request(app) + .get("/api/agents/me/inbox/mine") + .query({ userId: "board-user" }); + + expect(res.status).toBe(200); + expect(mockIssueService.list).toHaveBeenCalledWith(companyId, { + touchedByUserId: "board-user", + inboxArchivedByUserId: "board-user", + status: INBOX_MINE_ISSUE_STATUS_FILTER, + }); + expect(res.body).toEqual([ + { + id: "issue-1", + identifier: "PAP-910", + title: "Inbox follow-up", + status: "todo", + }, + ]); + }); }); diff --git a/server/src/__tests__/claude-local-adapter.test.ts b/server/src/__tests__/claude-local-adapter.test.ts index 26fc45318ce..d2cb8665ba5 100644 --- a/server/src/__tests__/claude-local-adapter.test.ts +++ b/server/src/__tests__/claude-local-adapter.test.ts @@ -1,5 +1,7 @@ -import { describe, expect, it } from "vitest"; +import { describe, expect, it, vi } from "vitest"; import { isClaudeMaxTurnsResult } from "@paperclipai/adapter-claude-local/server"; +import { parseClaudeStdoutLine } from "@paperclipai/adapter-claude-local/ui"; +import { printClaudeStreamEvent } from "@paperclipai/adapter-claude-local/cli"; describe("claude_local max-turn detection", () => { it("detects max-turn exhaustion by subtype", () => { @@ -28,3 +30,158 @@ describe("claude_local max-turn detection", () => { ).toBe(false); }); }); + +describe("claude_local ui stdout parser", () => { + it("maps assistant text, thinking, tool calls, and tool results into transcript entries", () => { + const ts = "2026-03-29T00:00:00.000Z"; + + expect( + parseClaudeStdoutLine( + JSON.stringify({ + type: "system", + subtype: "init", + model: "claude-sonnet-4-6", + session_id: "claude-session-1", + }), + ts, + ), + ).toEqual([ + { + kind: "init", + ts, + model: "claude-sonnet-4-6", + sessionId: "claude-session-1", + }, + ]); + + expect( + parseClaudeStdoutLine( + JSON.stringify({ + type: "assistant", + session_id: "claude-session-1", + message: { + content: [ + { type: "text", text: "I will inspect the repo." }, + { type: "thinking", thinking: "Checking the adapter wiring" }, + { type: "tool_use", id: "tool_1", name: "bash", input: { command: "ls -1" } }, + ], + }, + }), + ts, + ), + ).toEqual([ + { kind: "assistant", ts, text: "I will inspect the repo." }, + { kind: "thinking", ts, text: "Checking the adapter wiring" }, + { kind: "tool_call", ts, name: "bash", toolUseId: "tool_1", input: { command: "ls -1" } }, + ]); + + expect( + parseClaudeStdoutLine( + JSON.stringify({ + type: "user", + message: { + content: [ + { + type: "tool_result", + tool_use_id: "tool_1", + content: [{ type: "text", text: "AGENTS.md\nREADME.md" }], + is_error: false, + }, + ], + }, + }), + ts, + ), + ).toEqual([ + { + kind: "tool_result", + ts, + toolUseId: "tool_1", + content: "AGENTS.md\nREADME.md", + isError: false, + }, + ]); + }); +}); + +function stripAnsi(value: string) { + return value.replace(/\x1b\[[0-9;]*m/g, ""); +} + +describe("claude_local cli formatter", () => { + it("prints the user-visible and background transcript events from stream-json output", () => { + const spy = vi.spyOn(console, "log").mockImplementation(() => {}); + + try { + printClaudeStreamEvent( + JSON.stringify({ + type: "system", + subtype: "init", + model: "claude-sonnet-4-6", + session_id: "claude-session-1", + }), + false, + ); + printClaudeStreamEvent( + JSON.stringify({ + type: "assistant", + message: { + content: [ + { type: "text", text: "I will inspect the repo." }, + { type: "thinking", thinking: "Checking the adapter wiring" }, + { type: "tool_use", id: "tool_1", name: "bash", input: { command: "ls -1" } }, + ], + }, + }), + false, + ); + printClaudeStreamEvent( + JSON.stringify({ + type: "user", + message: { + content: [ + { + type: "tool_result", + tool_use_id: "tool_1", + content: [{ type: "text", text: "AGENTS.md\nREADME.md" }], + is_error: false, + }, + ], + }, + }), + false, + ); + printClaudeStreamEvent( + JSON.stringify({ + type: "result", + subtype: "success", + result: "Done", + usage: { input_tokens: 10, output_tokens: 5, cache_read_input_tokens: 2 }, + total_cost_usd: 0.00042, + }), + false, + ); + + const lines = spy.mock.calls + .map((call) => call.map((value) => String(value)).join(" ")) + .map(stripAnsi); + + expect(lines).toEqual( + expect.arrayContaining([ + "Claude initialized (model: claude-sonnet-4-6, session: claude-session-1)", + "assistant: I will inspect the repo.", + "thinking: Checking the adapter wiring", + "tool_call: bash", + '{\n "command": "ls -1"\n}', + "tool_result", + "AGENTS.md\nREADME.md", + "result:", + "Done", + "tokens: in=10 out=5 cached=2 cost=$0.000420", + ]), + ); + } finally { + spy.mockRestore(); + } + }); +}); diff --git a/server/src/__tests__/claude-local-execute.test.ts b/server/src/__tests__/claude-local-execute.test.ts new file mode 100644 index 00000000000..be933bca8f4 --- /dev/null +++ b/server/src/__tests__/claude-local-execute.test.ts @@ -0,0 +1,99 @@ +import { describe, expect, it } from "vitest"; +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { execute } from "@paperclipai/adapter-claude-local/server"; + +async function writeFakeClaudeCommand(commandPath: string): Promise { + const script = `#!/usr/bin/env node +const fs = require("node:fs"); + +const capturePath = process.env.PAPERCLIP_TEST_CAPTURE_PATH; +const payload = { + argv: process.argv.slice(2), + prompt: fs.readFileSync(0, "utf8"), + claudeConfigDir: process.env.CLAUDE_CONFIG_DIR || null, +}; +if (capturePath) { + fs.writeFileSync(capturePath, JSON.stringify(payload), "utf8"); +} +console.log(JSON.stringify({ type: "system", subtype: "init", session_id: "claude-session-1", model: "claude-sonnet" })); +console.log(JSON.stringify({ type: "assistant", session_id: "claude-session-1", message: { content: [{ type: "text", text: "hello" }] } })); +console.log(JSON.stringify({ type: "result", session_id: "claude-session-1", result: "hello", usage: { input_tokens: 1, cache_read_input_tokens: 0, output_tokens: 1 } })); +`; + await fs.writeFile(commandPath, script, "utf8"); + await fs.chmod(commandPath, 0o755); +} + +describe("claude execute", () => { + it("logs HOME, CLAUDE_CONFIG_DIR, and the resolved executable path in invocation metadata", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-claude-execute-meta-")); + const workspace = path.join(root, "workspace"); + const binDir = path.join(root, "bin"); + const commandPath = path.join(binDir, "claude"); + const capturePath = path.join(root, "capture.json"); + const claudeConfigDir = path.join(root, "claude-config"); + await fs.mkdir(workspace, { recursive: true }); + await fs.mkdir(binDir, { recursive: true }); + await fs.mkdir(claudeConfigDir, { recursive: true }); + await writeFakeClaudeCommand(commandPath); + + const previousHome = process.env.HOME; + const previousPath = process.env.PATH; + const previousClaudeConfigDir = process.env.CLAUDE_CONFIG_DIR; + process.env.HOME = root; + process.env.PATH = `${binDir}${path.delimiter}${process.env.PATH ?? ""}`; + process.env.CLAUDE_CONFIG_DIR = claudeConfigDir; + + let loggedCommand: string | null = null; + let loggedEnv: Record = {}; + try { + const result = await execute({ + runId: "run-meta", + agent: { + id: "agent-1", + companyId: "company-1", + name: "Claude Coder", + adapterType: "claude_local", + adapterConfig: {}, + }, + runtime: { + sessionId: null, + sessionParams: null, + sessionDisplayId: null, + taskKey: null, + }, + config: { + command: "claude", + cwd: workspace, + env: { + PAPERCLIP_TEST_CAPTURE_PATH: capturePath, + }, + promptTemplate: "Follow the paperclip heartbeat.", + }, + context: {}, + authToken: "run-jwt-token", + onLog: async () => {}, + onMeta: async (meta) => { + loggedCommand = meta.command; + loggedEnv = meta.env ?? {}; + }, + }); + + expect(result.exitCode).toBe(0); + expect(result.errorMessage).toBeNull(); + expect(loggedCommand).toBe(commandPath); + expect(loggedEnv.HOME).toBe(root); + expect(loggedEnv.CLAUDE_CONFIG_DIR).toBe(claudeConfigDir); + expect(loggedEnv.PAPERCLIP_RESOLVED_COMMAND).toBe(commandPath); + } finally { + if (previousHome === undefined) delete process.env.HOME; + else process.env.HOME = previousHome; + if (previousPath === undefined) delete process.env.PATH; + else process.env.PATH = previousPath; + if (previousClaudeConfigDir === undefined) delete process.env.CLAUDE_CONFIG_DIR; + else process.env.CLAUDE_CONFIG_DIR = previousClaudeConfigDir; + await fs.rm(root, { recursive: true, force: true }); + } + }); +}); diff --git a/server/src/__tests__/codex-local-execute.test.ts b/server/src/__tests__/codex-local-execute.test.ts index b83e3db7bd7..4f584435abf 100644 --- a/server/src/__tests__/codex-local-execute.test.ts +++ b/server/src/__tests__/codex-local-execute.test.ts @@ -195,6 +195,70 @@ describe("codex execute", () => { } }); + it("logs HOME and the resolved executable path in invocation metadata", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-execute-meta-")); + const workspace = path.join(root, "workspace"); + const binDir = path.join(root, "bin"); + const commandPath = path.join(binDir, "codex"); + const capturePath = path.join(root, "capture.json"); + await fs.mkdir(workspace, { recursive: true }); + await fs.mkdir(binDir, { recursive: true }); + await writeFakeCodexCommand(commandPath); + + const previousHome = process.env.HOME; + const previousPath = process.env.PATH; + process.env.HOME = root; + process.env.PATH = `${binDir}${path.delimiter}${process.env.PATH ?? ""}`; + + let loggedCommand: string | null = null; + let loggedEnv: Record = {}; + try { + const result = await execute({ + runId: "run-meta", + agent: { + id: "agent-1", + companyId: "company-1", + name: "Codex Coder", + adapterType: "codex_local", + adapterConfig: {}, + }, + runtime: { + sessionId: null, + sessionParams: null, + sessionDisplayId: null, + taskKey: null, + }, + config: { + command: "codex", + cwd: workspace, + env: { + PAPERCLIP_TEST_CAPTURE_PATH: capturePath, + }, + promptTemplate: "Follow the paperclip heartbeat.", + }, + context: {}, + authToken: "run-jwt-token", + onLog: async () => {}, + onMeta: async (meta) => { + loggedCommand = meta.command; + loggedEnv = meta.env ?? {}; + }, + }); + + expect(result.exitCode).toBe(0); + expect(result.errorMessage).toBeNull(); + expect(loggedCommand).toBe(commandPath); + expect(loggedEnv.HOME).toBe(root); + expect(loggedEnv.PAPERCLIP_RESOLVED_COMMAND).toBe(commandPath); + } finally { + if (previousHome === undefined) delete process.env.HOME; + else process.env.HOME = previousHome; + if (previousPath === undefined) delete process.env.PATH; + else process.env.PATH = previousPath; + await fs.rm(root, { recursive: true, force: true }); + } + }); + it("uses a worktree-isolated CODEX_HOME while preserving shared auth and config", async () => { const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-execute-")); const workspace = path.join(root, "workspace"); diff --git a/server/src/__tests__/execution-workspaces-service.test.ts b/server/src/__tests__/execution-workspaces-service.test.ts new file mode 100644 index 00000000000..d4a50bdccee --- /dev/null +++ b/server/src/__tests__/execution-workspaces-service.test.ts @@ -0,0 +1,325 @@ +import { execFile } from "node:child_process"; +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { randomUUID } from "node:crypto"; +import { promisify } from "node:util"; +import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest"; +import { + companies, + createDb, + executionWorkspaces, + issues, + projectWorkspaces, + projects, +} from "@paperclipai/db"; +import { + getEmbeddedPostgresTestSupport, + startEmbeddedPostgresTestDatabase, +} from "./helpers/embedded-postgres.js"; +import { + executionWorkspaceService, + mergeExecutionWorkspaceConfig, + readExecutionWorkspaceConfig, +} from "../services/execution-workspaces.ts"; + +const execFileAsync = promisify(execFile); + +describe("execution workspace config helpers", () => { + it("reads typed config from persisted metadata", () => { + expect(readExecutionWorkspaceConfig({ + source: "project_primary", + config: { + provisionCommand: "bash ./scripts/provision-worktree.sh", + teardownCommand: "bash ./scripts/teardown-worktree.sh", + cleanupCommand: "pkill -f vite || true", + workspaceRuntime: { + services: [{ name: "web", command: "pnpm dev", port: 3100 }], + }, + }, + })).toEqual({ + provisionCommand: "bash ./scripts/provision-worktree.sh", + teardownCommand: "bash ./scripts/teardown-worktree.sh", + cleanupCommand: "pkill -f vite || true", + desiredState: null, + workspaceRuntime: { + services: [{ name: "web", command: "pnpm dev", port: 3100 }], + }, + }); + }); + + it("merges config patches without dropping unrelated metadata", () => { + expect(mergeExecutionWorkspaceConfig( + { + source: "project_primary", + createdByRuntime: false, + config: { + provisionCommand: "bash ./scripts/provision-worktree.sh", + cleanupCommand: "pkill -f vite || true", + }, + }, + { + teardownCommand: "bash ./scripts/teardown-worktree.sh", + workspaceRuntime: { + services: [{ name: "web", command: "pnpm dev" }], + }, + }, + )).toEqual({ + source: "project_primary", + createdByRuntime: false, + config: { + provisionCommand: "bash ./scripts/provision-worktree.sh", + teardownCommand: "bash ./scripts/teardown-worktree.sh", + cleanupCommand: "pkill -f vite || true", + desiredState: null, + workspaceRuntime: { + services: [{ name: "web", command: "pnpm dev" }], + }, + }, + }); + }); + + it("clears the nested config block when requested", () => { + expect(mergeExecutionWorkspaceConfig( + { + source: "project_primary", + config: { + provisionCommand: "bash ./scripts/provision-worktree.sh", + }, + }, + null, + )).toEqual({ + source: "project_primary", + }); + }); +}); + +const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport(); +const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip; + +if (!embeddedPostgresSupport.supported) { + console.warn( + `Skipping embedded Postgres execution workspace service tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`, + ); +} + +async function runGit(cwd: string, args: string[]) { + await execFileAsync("git", ["-C", cwd, ...args], { cwd }); +} + +async function createTempRepo() { + const repoRoot = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-execution-workspace-")); + await runGit(repoRoot, ["init"]); + await runGit(repoRoot, ["config", "user.name", "Paperclip Test"]); + await runGit(repoRoot, ["config", "user.email", "test@paperclip.local"]); + await fs.writeFile(path.join(repoRoot, "README.md"), "# Test repo\n", "utf8"); + await runGit(repoRoot, ["add", "README.md"]); + await runGit(repoRoot, ["commit", "-m", "Initial commit"]); + await runGit(repoRoot, ["branch", "-M", "main"]); + return repoRoot; +} + +describeEmbeddedPostgres("executionWorkspaceService.getCloseReadiness", () => { + let db!: ReturnType; + let svc!: ReturnType; + let tempDb: Awaited> | null = null; + const tempDirs = new Set(); + + beforeAll(async () => { + tempDb = await startEmbeddedPostgresTestDatabase("paperclip-execution-workspaces-service-"); + db = createDb(tempDb.connectionString); + svc = executionWorkspaceService(db); + }, 20_000); + + afterEach(async () => { + await db.delete(issues); + await db.delete(executionWorkspaces); + await db.delete(projectWorkspaces); + await db.delete(projects); + await db.delete(companies); + + for (const dir of tempDirs) { + await fs.rm(dir, { recursive: true, force: true }); + } + tempDirs.clear(); + }); + + afterAll(async () => { + await tempDb?.cleanup(); + }); + + it("allows archiving shared workspace sessions with warnings even when issues are still open", async () => { + const companyId = randomUUID(); + const projectId = randomUUID(); + const projectWorkspaceId = randomUUID(); + const executionWorkspaceId = randomUUID(); + + await db.insert(companies).values({ + id: companyId, + name: "Paperclip", + issuePrefix: "PAP", + requireBoardApprovalForNewAgents: false, + }); + await db.insert(projects).values({ + id: projectId, + companyId, + name: "Workspaces", + status: "in_progress", + executionWorkspacePolicy: { + enabled: true, + }, + }); + await db.insert(projectWorkspaces).values({ + id: projectWorkspaceId, + companyId, + projectId, + name: "Primary", + sourceType: "local_path", + isPrimary: true, + cwd: "/tmp/paperclip-primary", + }); + await db.insert(executionWorkspaces).values({ + id: executionWorkspaceId, + companyId, + projectId, + projectWorkspaceId, + mode: "shared_workspace", + strategyType: "project_primary", + name: "Shared workspace", + status: "active", + providerType: "local_fs", + cwd: "/tmp/paperclip-primary", + metadata: { + config: { + teardownCommand: "bash ./scripts/teardown.sh", + }, + }, + }); + await db.insert(issues).values({ + id: randomUUID(), + companyId, + projectId, + title: "Still working", + status: "todo", + priority: "medium", + executionWorkspaceId, + }); + + const readiness = await svc.getCloseReadiness(executionWorkspaceId); + + expect(readiness).toMatchObject({ + workspaceId: executionWorkspaceId, + state: "ready_with_warnings", + isSharedWorkspace: true, + isProjectPrimaryWorkspace: true, + isDestructiveCloseAllowed: true, + }); + expect(readiness?.blockingReasons).toEqual([]); + expect(readiness?.warnings).toEqual(expect.arrayContaining([ + "This workspace is still linked to an open issue. Archiving it will detach this shared workspace session from those issues, but keep the underlying project workspace available.", + "This shared workspace session points at project workspace infrastructure. Archiving it only removes the session record.", + ])); + }); + + it("warns about dirty and unmerged git worktrees and reports cleanup actions", async () => { + const repoRoot = await createTempRepo(); + tempDirs.add(repoRoot); + const worktreePath = path.join(path.dirname(repoRoot), `paperclip-worktree-${randomUUID()}`); + tempDirs.add(worktreePath); + + await runGit(repoRoot, ["branch", "paperclip-close-check"]); + await runGit(repoRoot, ["worktree", "add", worktreePath, "paperclip-close-check"]); + await fs.writeFile(path.join(worktreePath, "feature.txt"), "hello\n", "utf8"); + await runGit(worktreePath, ["add", "feature.txt"]); + await runGit(worktreePath, ["commit", "-m", "Feature commit"]); + await fs.writeFile(path.join(worktreePath, "untracked.txt"), "left behind\n", "utf8"); + + const companyId = randomUUID(); + const projectId = randomUUID(); + const projectWorkspaceId = randomUUID(); + const executionWorkspaceId = randomUUID(); + + await db.insert(companies).values({ + id: companyId, + name: "Paperclip", + issuePrefix: "PAP", + requireBoardApprovalForNewAgents: false, + }); + await db.insert(projects).values({ + id: projectId, + companyId, + name: "Workspaces", + status: "in_progress", + executionWorkspacePolicy: { + enabled: true, + workspaceStrategy: { + type: "git_worktree", + teardownCommand: "bash ./scripts/project-teardown.sh", + }, + }, + }); + await db.insert(projectWorkspaces).values({ + id: projectWorkspaceId, + companyId, + projectId, + name: "Primary", + sourceType: "git_repo", + isPrimary: true, + cwd: repoRoot, + cleanupCommand: "printf 'project cleanup\\n'", + }); + await db.insert(executionWorkspaces).values({ + id: executionWorkspaceId, + companyId, + projectId, + projectWorkspaceId, + mode: "isolated_workspace", + strategyType: "git_worktree", + name: "Feature workspace", + status: "active", + providerType: "git_worktree", + cwd: worktreePath, + providerRef: worktreePath, + branchName: "paperclip-close-check", + baseRef: "main", + metadata: { + createdByRuntime: true, + config: { + cleanupCommand: "printf 'workspace cleanup\\n'", + }, + }, + }); + + const readiness = await svc.getCloseReadiness(executionWorkspaceId); + + expect(readiness).toMatchObject({ + workspaceId: executionWorkspaceId, + state: "ready_with_warnings", + isSharedWorkspace: false, + isProjectPrimaryWorkspace: false, + isDestructiveCloseAllowed: true, + git: { + workspacePath: worktreePath, + branchName: "paperclip-close-check", + baseRef: "main", + createdByRuntime: true, + hasDirtyTrackedFiles: false, + hasUntrackedFiles: true, + aheadCount: 1, + behindCount: 0, + isMergedIntoBase: false, + }, + }); + expect(readiness?.warnings).toEqual(expect.arrayContaining([ + "The workspace has 1 untracked file.", + "This workspace is 1 commit ahead of main and is not merged.", + ])); + expect(readiness?.plannedActions.map((action) => action.kind)).toEqual(expect.arrayContaining([ + "archive_record", + "cleanup_command", + "teardown_command", + "git_worktree_remove", + "git_branch_delete", + ])); + }, 20_000); +}); diff --git a/server/src/__tests__/health.test.ts b/server/src/__tests__/health.test.ts index e6656c3daf6..4349d81634e 100644 --- a/server/src/__tests__/health.test.ts +++ b/server/src/__tests__/health.test.ts @@ -1,20 +1,19 @@ -import { describe, expect, it, vi } from "vitest"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import express from "express"; import request from "supertest"; +import type { Db } from "@paperclipai/db"; import { healthRoutes } from "../routes/health.js"; +import * as devServerStatus from "../dev-server-status.js"; import { serverVersion } from "../version.js"; -vi.mock("../dev-server-status.js", async () => { - const actual = await vi.importActual( - "../dev-server-status.js", - ); - return { - ...actual, - readPersistedDevServerStatus: () => null, - }; -}); - describe("GET /health", () => { + beforeEach(() => { + vi.spyOn(devServerStatus, "readPersistedDevServerStatus").mockReturnValue(undefined); + }); + + afterEach(() => { + vi.restoreAllMocks(); + }); it("returns 200 with status ok", async () => { const app = express(); app.use("/health", healthRoutes()); @@ -23,34 +22,34 @@ describe("GET /health", () => { expect(res.body).toEqual({ status: "ok", version: serverVersion }); }); - it("checks database connectivity before reporting healthy", async () => { - const execute = vi.fn().mockResolvedValue([]); + it("returns 200 when the database probe succeeds", async () => { + const db = { + execute: vi.fn().mockResolvedValue([{ "?column?": 1 }]), + } as unknown as Db; const app = express(); - app.use("/health", healthRoutes({ execute } as any)); + app.use("/health", healthRoutes(db)); const res = await request(app).get("/health"); expect(res.status).toBe(200); - expect(execute).toHaveBeenCalledTimes(1); + expect(db.execute).toHaveBeenCalledTimes(1); expect(res.body).toMatchObject({ status: "ok", version: serverVersion }); }); it("returns 503 when the database probe fails", async () => { + const db = { + execute: vi.fn().mockRejectedValue(new Error("connect ECONNREFUSED")), + } as unknown as Db; const app = express(); - app.use( - "/health", - healthRoutes({ - execute: vi.fn().mockRejectedValue(new Error("db offline")), - } as any), - ); + app.use("/health", healthRoutes(db)); const res = await request(app).get("/health"); expect(res.status).toBe(503); expect(res.body).toEqual({ - status: "error", - error: "Database unavailable", + status: "unhealthy", version: serverVersion, + error: "database_unreachable" }); }); }); diff --git a/server/src/__tests__/heartbeat-workspace-session.test.ts b/server/src/__tests__/heartbeat-workspace-session.test.ts index 7fab2b429da..47d37c53309 100644 --- a/server/src/__tests__/heartbeat-workspace-session.test.ts +++ b/server/src/__tests__/heartbeat-workspace-session.test.ts @@ -3,11 +3,14 @@ import type { agents } from "@paperclipai/db"; import { sessionCodec as codexSessionCodec } from "@paperclipai/adapter-codex-local/server"; import { resolveDefaultAgentWorkspaceDir } from "../home-paths.js"; import { + applyPersistedExecutionWorkspaceConfig, + buildRealizedExecutionWorkspaceFromPersisted, buildExplicitResumeSessionOverride, formatRuntimeWorkspaceWarningLog, prioritizeProjectWorkspaceCandidatesForRun, parseSessionCompactionPolicy, resolveRuntimeSessionParamsForWorkspace, + stripWorkspaceRuntimeFromExecutionRunConfig, shouldResetTaskSessionForWake, type ResolvedWorkspaceForRun, } from "../services/heartbeat.ts"; @@ -120,6 +123,109 @@ describe("resolveRuntimeSessionParamsForWorkspace", () => { }); }); +describe("applyPersistedExecutionWorkspaceConfig", () => { + it("does not add workspace runtime when only the project workspace had manual runtime config", () => { + const result = applyPersistedExecutionWorkspaceConfig({ + config: {}, + workspaceConfig: null, + mode: "isolated_workspace", + }); + + expect("workspaceRuntime" in result).toBe(false); + }); + + it("applies explicit persisted execution workspace runtime config when present", () => { + const result = applyPersistedExecutionWorkspaceConfig({ + config: {}, + workspaceConfig: { + provisionCommand: null, + teardownCommand: null, + cleanupCommand: null, + desiredState: null, + workspaceRuntime: { + services: [{ name: "workspace-web" }], + }, + }, + mode: "isolated_workspace", + }); + + expect(result.workspaceRuntime).toEqual({ + services: [{ name: "workspace-web" }], + }); + }); +}); + +describe("buildRealizedExecutionWorkspaceFromPersisted", () => { + it("reuses the persisted execution workspace path instead of deriving a new worktree", () => { + const result = buildRealizedExecutionWorkspaceFromPersisted({ + base: buildResolvedWorkspace({ + cwd: "/tmp/project-primary", + repoRef: "main", + }), + workspace: { + id: "execution-workspace-1", + companyId: "company-1", + projectId: "project-1", + projectWorkspaceId: "workspace-1", + sourceIssueId: "issue-1", + mode: "isolated_workspace", + strategyType: "git_worktree", + name: "PAP-880-thumbs-capture-for-evals-feature", + status: "active", + cwd: "/tmp/reused-worktree", + repoUrl: "https://example.com/paperclip.git", + baseRef: "main", + branchName: "PAP-880-thumbs-capture-for-evals-feature", + providerType: "git_worktree", + providerRef: "/tmp/reused-worktree", + derivedFromExecutionWorkspaceId: null, + lastUsedAt: new Date(), + openedAt: new Date(), + closedAt: null, + cleanupEligibleAt: null, + cleanupReason: null, + config: null, + metadata: null, + createdAt: new Date(), + updatedAt: new Date(), + }, + }); + + expect(result.created).toBe(false); + expect(result.strategy).toBe("git_worktree"); + expect(result.cwd).toBe("/tmp/reused-worktree"); + expect(result.worktreePath).toBe("/tmp/reused-worktree"); + expect(result.branchName).toBe("PAP-880-thumbs-capture-for-evals-feature"); + expect(result.source).toBe("task_session"); + }); +}); + +describe("stripWorkspaceRuntimeFromExecutionRunConfig", () => { + it("removes workspace runtime before heartbeat execution", () => { + const input = { + cwd: "/tmp/project", + workspaceStrategy: { + type: "git_worktree", + }, + workspaceRuntime: { + services: [{ name: "web" }], + }, + }; + + const result = stripWorkspaceRuntimeFromExecutionRunConfig(input); + + expect(result).toEqual({ + cwd: "/tmp/project", + workspaceStrategy: { + type: "git_worktree", + }, + }); + expect(input.workspaceRuntime).toEqual({ + services: [{ name: "web" }], + }); + }); +}); + describe("shouldResetTaskSessionForWake", () => { it("resets session context on assignment wake", () => { expect(shouldResetTaskSessionForWake({ wakeReason: "issue_assigned" })).toBe(true); diff --git a/server/src/__tests__/issue-comment-reopen-routes.test.ts b/server/src/__tests__/issue-comment-reopen-routes.test.ts index 42c4cb0db28..21bb44aa3bb 100644 --- a/server/src/__tests__/issue-comment-reopen-routes.test.ts +++ b/server/src/__tests__/issue-comment-reopen-routes.test.ts @@ -19,6 +19,9 @@ const mockAccessService = vi.hoisted(() => ({ const mockHeartbeatService = vi.hoisted(() => ({ wakeup: vi.fn(async () => undefined), reportRunActivity: vi.fn(async () => undefined), + getRun: vi.fn(async () => null), + getActiveRunForAgent: vi.fn(async () => null), + cancelRun: vi.fn(async () => null), })); const mockAgentService = vi.hoisted(() => ({ @@ -143,4 +146,46 @@ describe("issue comment reopen routes", () => { }), ); }); + + it("interrupts an active run before a combined comment update", async () => { + const issue = { + ...makeIssue("todo"), + executionRunId: "run-1", + }; + mockIssueService.getById.mockResolvedValue(issue); + mockIssueService.update.mockImplementation(async (_id: string, patch: Record) => ({ + ...issue, + ...patch, + })); + mockHeartbeatService.getRun.mockResolvedValue({ + id: "run-1", + companyId: "company-1", + agentId: "22222222-2222-4222-8222-222222222222", + status: "running", + }); + mockHeartbeatService.cancelRun.mockResolvedValue({ + id: "run-1", + companyId: "company-1", + agentId: "22222222-2222-4222-8222-222222222222", + status: "cancelled", + }); + + const res = await request(createApp()) + .patch("/api/issues/11111111-1111-4111-8111-111111111111") + .send({ comment: "hello", interrupt: true, assigneeAgentId: "33333333-3333-4333-8333-333333333333" }); + + expect(res.status).toBe(200); + expect(mockHeartbeatService.getRun).toHaveBeenCalledWith("run-1"); + expect(mockHeartbeatService.cancelRun).toHaveBeenCalledWith("run-1"); + expect(mockLogActivity).toHaveBeenCalledWith( + expect.anything(), + expect.objectContaining({ + action: "heartbeat.cancelled", + details: expect.objectContaining({ + source: "issue_comment_interrupt", + issueId: "11111111-1111-4111-8111-111111111111", + }), + }), + ); + }); }); diff --git a/server/src/__tests__/issues-service.test.ts b/server/src/__tests__/issues-service.test.ts index c5aef4b3d18..9543a6140c9 100644 --- a/server/src/__tests__/issues-service.test.ts +++ b/server/src/__tests__/issues-service.test.ts @@ -5,14 +5,19 @@ import { agents, companies, createDb, + executionWorkspaces, + instanceSettings, issueComments, issueInboxArchives, issues, + projectWorkspaces, + projects, } from "@paperclipai/db"; import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase, } from "./helpers/embedded-postgres.js"; +import { instanceSettingsService } from "../services/instance-settings.ts"; import { issueService } from "../services/issues.ts"; const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport(); @@ -40,7 +45,11 @@ describeEmbeddedPostgres("issueService.list participantAgentId", () => { await db.delete(issueInboxArchives); await db.delete(activityLog); await db.delete(issues); + await db.delete(executionWorkspaces); + await db.delete(projectWorkspaces); + await db.delete(projects); await db.delete(agents); + await db.delete(instanceSettings); await db.delete(companies); }); @@ -219,6 +228,86 @@ describeEmbeddedPostgres("issueService.list participantAgentId", () => { expect(result.map((issue) => issue.id)).toEqual([matchedIssueId]); }); + it("filters issues by execution workspace id", async () => { + const companyId = randomUUID(); + const projectId = randomUUID(); + const targetWorkspaceId = randomUUID(); + const otherWorkspaceId = randomUUID(); + const linkedIssueId = randomUUID(); + const otherLinkedIssueId = randomUUID(); + const unlinkedIssueId = randomUUID(); + + await db.insert(companies).values({ + id: companyId, + name: "Paperclip", + issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`, + requireBoardApprovalForNewAgents: false, + }); + + await db.insert(projects).values({ + id: projectId, + companyId, + name: "Workspace project", + status: "in_progress", + }); + + await db.insert(executionWorkspaces).values([ + { + id: targetWorkspaceId, + companyId, + projectId, + mode: "shared_workspace", + strategyType: "project_primary", + name: "Target workspace", + status: "active", + providerType: "local_fs", + }, + { + id: otherWorkspaceId, + companyId, + projectId, + mode: "shared_workspace", + strategyType: "project_primary", + name: "Other workspace", + status: "active", + providerType: "local_fs", + }, + ]); + + await db.insert(issues).values([ + { + id: linkedIssueId, + companyId, + projectId, + title: "Linked issue", + status: "todo", + priority: "medium", + executionWorkspaceId: targetWorkspaceId, + }, + { + id: otherLinkedIssueId, + companyId, + projectId, + title: "Other linked issue", + status: "todo", + priority: "medium", + executionWorkspaceId: otherWorkspaceId, + }, + { + id: unlinkedIssueId, + companyId, + projectId, + title: "Unlinked issue", + status: "todo", + priority: "medium", + }, + ]); + + const result = await svc.list(companyId, { executionWorkspaceId: targetWorkspaceId }); + + expect(result.map((issue) => issue.id)).toEqual([linkedIssueId]); + }); + it("hides archived inbox issues until new external activity arrives", async () => { const companyId = randomUUID(); const userId = "user-1"; @@ -314,3 +403,278 @@ describeEmbeddedPostgres("issueService.list participantAgentId", () => { ])); }); }); + +describeEmbeddedPostgres("issueService.create workspace inheritance", () => { + let db!: ReturnType; + let svc!: ReturnType; + let tempDb: Awaited> | null = null; + + beforeAll(async () => { + tempDb = await startEmbeddedPostgresTestDatabase("paperclip-issues-create-"); + db = createDb(tempDb.connectionString); + svc = issueService(db); + }, 20_000); + + afterEach(async () => { + await db.delete(issueComments); + await db.delete(issueInboxArchives); + await db.delete(activityLog); + await db.delete(issues); + await db.delete(executionWorkspaces); + await db.delete(projectWorkspaces); + await db.delete(projects); + await db.delete(agents); + await db.delete(instanceSettings); + await db.delete(companies); + }); + + afterAll(async () => { + await tempDb?.cleanup(); + }); + + it("inherits the parent issue workspace linkage when child workspace fields are omitted", async () => { + const companyId = randomUUID(); + const projectId = randomUUID(); + const parentIssueId = randomUUID(); + const projectWorkspaceId = randomUUID(); + const executionWorkspaceId = randomUUID(); + + await db.insert(companies).values({ + id: companyId, + name: "Paperclip", + issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`, + requireBoardApprovalForNewAgents: false, + }); + await instanceSettingsService(db).updateExperimental({ enableIsolatedWorkspaces: true }); + + await db.insert(projects).values({ + id: projectId, + companyId, + name: "Workspace project", + status: "in_progress", + }); + + await db.insert(projectWorkspaces).values({ + id: projectWorkspaceId, + companyId, + projectId, + name: "Primary workspace", + isPrimary: true, + sharedWorkspaceKey: "workspace-key", + }); + + await db.insert(executionWorkspaces).values({ + id: executionWorkspaceId, + companyId, + projectId, + projectWorkspaceId, + mode: "isolated_workspace", + strategyType: "git_worktree", + name: "Issue worktree", + status: "active", + providerType: "git_worktree", + providerRef: `/tmp/${executionWorkspaceId}`, + }); + + await db.insert(issues).values({ + id: parentIssueId, + companyId, + projectId, + projectWorkspaceId, + title: "Parent issue", + status: "in_progress", + priority: "medium", + executionWorkspaceId, + executionWorkspacePreference: "reuse_existing", + executionWorkspaceSettings: { + mode: "isolated_workspace", + workspaceRuntime: { profile: "agent" }, + }, + }); + + const child = await svc.create(companyId, { + parentId: parentIssueId, + projectId, + title: "Child issue", + }); + + expect(child.parentId).toBe(parentIssueId); + expect(child.projectWorkspaceId).toBe(projectWorkspaceId); + expect(child.executionWorkspaceId).toBe(executionWorkspaceId); + expect(child.executionWorkspacePreference).toBe("reuse_existing"); + expect(child.executionWorkspaceSettings).toEqual({ + mode: "isolated_workspace", + workspaceRuntime: { profile: "agent" }, + }); + }); + + it("keeps explicit workspace fields instead of inheriting the parent linkage", async () => { + const companyId = randomUUID(); + const projectId = randomUUID(); + const parentIssueId = randomUUID(); + const parentProjectWorkspaceId = randomUUID(); + const parentExecutionWorkspaceId = randomUUID(); + const explicitProjectWorkspaceId = randomUUID(); + const explicitExecutionWorkspaceId = randomUUID(); + + await db.insert(companies).values({ + id: companyId, + name: "Paperclip", + issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`, + requireBoardApprovalForNewAgents: false, + }); + await instanceSettingsService(db).updateExperimental({ enableIsolatedWorkspaces: true }); + + await db.insert(projects).values({ + id: projectId, + companyId, + name: "Workspace project", + status: "in_progress", + }); + + await db.insert(projectWorkspaces).values([ + { + id: parentProjectWorkspaceId, + companyId, + projectId, + name: "Parent workspace", + }, + { + id: explicitProjectWorkspaceId, + companyId, + projectId, + name: "Explicit workspace", + }, + ]); + + await db.insert(executionWorkspaces).values([ + { + id: parentExecutionWorkspaceId, + companyId, + projectId, + projectWorkspaceId: parentProjectWorkspaceId, + mode: "isolated_workspace", + strategyType: "git_worktree", + name: "Parent worktree", + status: "active", + providerType: "git_worktree", + }, + { + id: explicitExecutionWorkspaceId, + companyId, + projectId, + projectWorkspaceId: explicitProjectWorkspaceId, + mode: "shared_workspace", + strategyType: "project_primary", + name: "Explicit shared workspace", + status: "active", + providerType: "local_fs", + }, + ]); + + await db.insert(issues).values({ + id: parentIssueId, + companyId, + projectId, + projectWorkspaceId: parentProjectWorkspaceId, + title: "Parent issue", + status: "in_progress", + priority: "medium", + executionWorkspaceId: parentExecutionWorkspaceId, + executionWorkspacePreference: "reuse_existing", + executionWorkspaceSettings: { + mode: "isolated_workspace", + }, + }); + + const child = await svc.create(companyId, { + parentId: parentIssueId, + projectId, + title: "Child issue", + projectWorkspaceId: explicitProjectWorkspaceId, + executionWorkspaceId: explicitExecutionWorkspaceId, + executionWorkspacePreference: "reuse_existing", + executionWorkspaceSettings: { + mode: "shared_workspace", + }, + }); + + expect(child.projectWorkspaceId).toBe(explicitProjectWorkspaceId); + expect(child.executionWorkspaceId).toBe(explicitExecutionWorkspaceId); + expect(child.executionWorkspacePreference).toBe("reuse_existing"); + expect(child.executionWorkspaceSettings).toEqual({ + mode: "shared_workspace", + }); + }); + + it("inherits workspace linkage from an explicit source issue without creating a parent-child relationship", async () => { + const companyId = randomUUID(); + const projectId = randomUUID(); + const sourceIssueId = randomUUID(); + const projectWorkspaceId = randomUUID(); + const executionWorkspaceId = randomUUID(); + + await db.insert(companies).values({ + id: companyId, + name: "Paperclip", + issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`, + requireBoardApprovalForNewAgents: false, + }); + await instanceSettingsService(db).updateExperimental({ enableIsolatedWorkspaces: true }); + + await db.insert(projects).values({ + id: projectId, + companyId, + name: "Workspace project", + status: "in_progress", + }); + + await db.insert(projectWorkspaces).values({ + id: projectWorkspaceId, + companyId, + projectId, + name: "Primary workspace", + }); + + await db.insert(executionWorkspaces).values({ + id: executionWorkspaceId, + companyId, + projectId, + projectWorkspaceId, + mode: "operator_branch", + strategyType: "git_worktree", + name: "Operator branch", + status: "active", + providerType: "git_worktree", + }); + + await db.insert(issues).values({ + id: sourceIssueId, + companyId, + projectId, + projectWorkspaceId, + title: "Source issue", + status: "todo", + priority: "medium", + executionWorkspaceId, + executionWorkspacePreference: "reuse_existing", + executionWorkspaceSettings: { + mode: "operator_branch", + }, + }); + + const followUp = await svc.create(companyId, { + projectId, + title: "Follow-up issue", + inheritExecutionWorkspaceFromIssueId: sourceIssueId, + }); + + expect(followUp.parentId).toBeNull(); + expect(followUp.projectWorkspaceId).toBe(projectWorkspaceId); + expect(followUp.executionWorkspaceId).toBe(executionWorkspaceId); + expect(followUp.executionWorkspacePreference).toBe("reuse_existing"); + expect(followUp.executionWorkspaceSettings).toEqual({ + mode: "operator_branch", + }); + }); +}); diff --git a/server/src/__tests__/openclaw-invite-prompt-route.test.ts b/server/src/__tests__/openclaw-invite-prompt-route.test.ts index 189126f92cd..990587d3e06 100644 --- a/server/src/__tests__/openclaw-invite-prompt-route.test.ts +++ b/server/src/__tests__/openclaw-invite-prompt-route.test.ts @@ -1,6 +1,7 @@ import express from "express"; import request from "supertest"; import { beforeEach, describe, expect, it, vi } from "vitest"; +import { companies, invites } from "@paperclipai/db"; import { accessRoutes } from "../routes/access.js"; import { errorHandler } from "../middleware/index.js"; @@ -51,19 +52,35 @@ function createDbStub() { inviteType: "company_join", allowedJoinTypes: "agent", defaultsPayload: null, - expiresAt: new Date("2026-03-07T00:10:00.000Z"), + expiresAt: new Date("2099-03-07T00:10:00.000Z"), invitedByUserId: null, tokenHash: "hash", revokedAt: null, acceptedAt: null, - createdAt: new Date("2026-03-07T00:00:00.000Z"), - updatedAt: new Date("2026-03-07T00:00:00.000Z"), + createdAt: new Date("2099-03-07T00:00:00.000Z"), + updatedAt: new Date("2099-03-07T00:00:00.000Z"), }; const returning = vi.fn().mockResolvedValue([createdInvite]); const values = vi.fn().mockReturnValue({ returning }); const insert = vi.fn().mockReturnValue({ values }); + const select = vi.fn(() => ({ + from(table: unknown) { + return { + where: vi.fn().mockImplementation(() => { + if (table === invites) { + return Promise.resolve([createdInvite]); + } + if (table === companies) { + return Promise.resolve([{ name: "Acme AI" }]); + } + return Promise.resolve([]); + }), + }; + }, + })); return { insert, + select, }; } @@ -143,9 +160,30 @@ describe("POST /companies/:companyId/openclaw/invite-prompt", () => { expect(res.status).toBe(201); expect(res.body.allowedJoinTypes).toBe("agent"); expect(typeof res.body.token).toBe("string"); + expect(res.body.companyName).toBe("Acme AI"); expect(res.body.onboardingTextPath).toContain("/api/invites/"); }); + it("includes companyName in invite summary responses", async () => { + const db = createDbStub(); + const app = createApp( + { + type: "board", + userId: "user-1", + companyIds: ["company-1"], + source: "session", + isInstanceAdmin: false, + }, + db, + ); + + const res = await request(app).get("/api/invites/pcp_invite_test"); + + expect(res.status).toBe(200); + expect(res.body.companyId).toBe("company-1"); + expect(res.body.companyName).toBe("Acme AI"); + }); + it("allows board callers with invite permission", async () => { const db = createDbStub(); mockAccessService.canUser.mockResolvedValue(true); diff --git a/server/src/__tests__/workspace-runtime.test.ts b/server/src/__tests__/workspace-runtime.test.ts index 6a55a72b71e..573013bea66 100644 --- a/server/src/__tests__/workspace-runtime.test.ts +++ b/server/src/__tests__/workspace-runtime.test.ts @@ -1,25 +1,51 @@ import { execFile } from "node:child_process"; +import { randomUUID } from "node:crypto"; import fs from "node:fs/promises"; import os from "node:os"; import path from "node:path"; import { fileURLToPath } from "node:url"; import { promisify } from "node:util"; -import { afterEach, describe, expect, it } from "vitest"; +import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest"; +import { + agents, + companies, + createDb, + executionWorkspaces, + heartbeatRuns, + projects, + workspaceRuntimeServices, +} from "@paperclipai/db"; +import { eq } from "drizzle-orm"; import { cleanupExecutionWorkspaceArtifacts, ensureRuntimeServicesForRun, normalizeAdapterManagedRuntimeServices, + reconcilePersistedRuntimeServicesOnStartup, realizeExecutionWorkspace, releaseRuntimeServicesForRun, + resetRuntimeServicesForTests, + sanitizeRuntimeServiceBaseEnv, stopRuntimeServicesForExecutionWorkspace, type RealizedExecutionWorkspace, } from "../services/workspace-runtime.ts"; import { resolvePaperclipConfigPath } from "../paths.ts"; import type { WorkspaceOperation } from "@paperclipai/shared"; import type { WorkspaceOperationRecorder } from "../services/workspace-operations.ts"; +import { + getEmbeddedPostgresTestSupport, + startEmbeddedPostgresTestDatabase, +} from "./helpers/embedded-postgres.js"; const execFileAsync = promisify(execFile); const leasedRunIds = new Set(); +const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport(); +const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip; + +if (!embeddedPostgresSupport.supported) { + console.warn( + `Skipping embedded Postgres workspace-runtime tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`, + ); +} async function runGit(cwd: string, args: string[]) { await execFileAsync("git", args, { cwd }); @@ -128,6 +154,28 @@ afterEach(async () => { delete process.env.PAPERCLIP_INSTANCE_ID; delete process.env.PAPERCLIP_WORKTREES_DIR; delete process.env.DATABASE_URL; + await resetRuntimeServicesForTests(); +}); + +describe("sanitizeRuntimeServiceBaseEnv", () => { + it("removes inherited Paperclip and pnpm auth flags before spawning runtime services", () => { + const sanitized = sanitizeRuntimeServiceBaseEnv({ + PATH: process.env.PATH, + DATABASE_URL: "postgres://example.test/paperclip", + PAPERCLIP_HOME: "/tmp/paperclip-home", + PAPERCLIP_INSTANCE_ID: "runtime-instance", + npm_config_tailscale_auth: "true", + npm_config_authenticated_private: "true", + HOST: "0.0.0.0", + }); + + expect(sanitized.PAPERCLIP_HOME).toBeUndefined(); + expect(sanitized.PAPERCLIP_INSTANCE_ID).toBeUndefined(); + expect(sanitized.DATABASE_URL).toBeUndefined(); + expect(sanitized.npm_config_tailscale_auth).toBeUndefined(); + expect(sanitized.npm_config_authenticated_private).toBeUndefined(); + expect(sanitized.HOST).toBe("0.0.0.0"); + }); }); describe("realizeExecutionWorkspace", () => { @@ -199,6 +247,43 @@ describe("realizeExecutionWorkspace", () => { expect(second.branchName).toBe(first.branchName); }); + it("slugifies unsafe issue titles for branch names and worktree folders", async () => { + const repoRoot = await createTempRepo(); + + const realized = await realizeExecutionWorkspace({ + base: { + baseCwd: repoRoot, + source: "project_primary", + projectId: "project-1", + workspaceId: "workspace-1", + repoUrl: null, + repoRef: "HEAD", + }, + config: { + workspaceStrategy: { + type: "git_worktree", + branchTemplate: "{{issue.identifier}}-{{slug}}", + }, + }, + issue: { + id: "issue-unsafe", + identifier: "PAP-991", + title: "there should be a setting for the allowance of thumbs up / thumbs down data; `rm -rf`", + }, + agent: { + id: "agent-1", + name: "Codex Coder", + companyId: "company-1", + }, + }); + + expect(realized.branchName).toBe( + "PAP-991-there-should-be-a-setting-for-the-allowance-of-thumbs-up-thumbs-down-data-rm-rf", + ); + expect(realized.branchName?.includes("/")).toBe(false); + expect(path.basename(realized.cwd)).toBe(realized.branchName); + }); + it("runs a configured provision command inside the derived worktree", async () => { const repoRoot = await createTempRepo(); await fs.mkdir(path.join(repoRoot, "scripts"), { recursive: true }); @@ -834,6 +919,101 @@ describe("ensureRuntimeServicesForRun", () => { expect(third[0]?.id).not.toBe(first[0]?.id); }); + it("does not reuse project-scoped shared services across different workspace launch contexts", async () => { + const primaryWorkspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-runtime-primary-")); + const worktreeWorkspaceRoot = path.join(primaryWorkspaceRoot, ".paperclip", "worktrees", "PAP-874-chat-speed-issues"); + await fs.mkdir(worktreeWorkspaceRoot, { recursive: true }); + + const primaryWorkspace = buildWorkspace(primaryWorkspaceRoot); + const executionWorkspace: RealizedExecutionWorkspace = { + ...buildWorkspace(worktreeWorkspaceRoot), + source: "task_session", + strategy: "git_worktree", + cwd: worktreeWorkspaceRoot, + branchName: "PAP-874-chat-speed-issues", + worktreePath: worktreeWorkspaceRoot, + }; + const serviceCommand = + "node -e \"require('node:http').createServer((req,res)=>res.end(process.env.PAPERCLIP_HOME)).listen(Number(process.env.PORT), '127.0.0.1')\""; + const config = { + workspaceRuntime: { + services: [ + { + name: "paperclip-dev", + command: serviceCommand, + cwd: ".", + env: { + PAPERCLIP_HOME: "{{workspace.cwd}}/.paperclip/runtime-services", + }, + port: { type: "auto" }, + readiness: { + type: "http", + urlTemplate: "http://127.0.0.1:{{port}}", + timeoutSec: 10, + intervalMs: 100, + }, + expose: { + type: "url", + urlTemplate: "http://127.0.0.1:{{port}}", + }, + lifecycle: "shared", + reuseScope: "project_workspace", + stopPolicy: { + type: "on_run_finish", + }, + }, + ], + }, + }; + + const primaryRunId = "run-project-workspace"; + const executionRunId = "run-execution-workspace"; + leasedRunIds.add(primaryRunId); + leasedRunIds.add(executionRunId); + + const primaryServices = await ensureRuntimeServicesForRun({ + runId: primaryRunId, + agent: { + id: "agent-1", + name: "Codex Coder", + companyId: "company-1", + }, + issue: null, + workspace: primaryWorkspace, + config, + adapterEnv: {}, + }); + + const executionServices = await ensureRuntimeServicesForRun({ + runId: executionRunId, + agent: { + id: "agent-1", + name: "Codex Coder", + companyId: "company-1", + }, + issue: null, + workspace: executionWorkspace, + executionWorkspaceId: "execution-workspace-1", + config, + adapterEnv: {}, + }); + + expect(primaryServices).toHaveLength(1); + expect(executionServices).toHaveLength(1); + expect(primaryServices[0]?.reused).toBe(false); + expect(executionServices[0]?.reused).toBe(false); + expect(executionServices[0]?.id).not.toBe(primaryServices[0]?.id); + expect(executionServices[0]?.executionWorkspaceId).toBe("execution-workspace-1"); + expect(executionServices[0]?.cwd).toBe(worktreeWorkspaceRoot); + expect(executionServices[0]?.url).not.toBe(primaryServices[0]?.url); + + const primaryResponse = await fetch(primaryServices[0]!.url!); + expect(await primaryResponse.text()).toBe(path.join(primaryWorkspaceRoot, ".paperclip", "runtime-services")); + + const executionResponse = await fetch(executionServices[0]!.url!); + expect(await executionResponse.text()).toBe(path.join(worktreeWorkspaceRoot, ".paperclip", "runtime-services")); + }); + it("does not leak parent Paperclip instance env into runtime service commands", async () => { const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-runtime-env-")); const workspace = buildWorkspace(workspaceRoot); @@ -1028,6 +1208,258 @@ describe("ensureRuntimeServicesForRun", () => { }); }); +describeEmbeddedPostgres("workspace runtime startup reconciliation", () => { + let db!: ReturnType; + let tempDb: Awaited> | null = null; + + beforeAll(async () => { + tempDb = await startEmbeddedPostgresTestDatabase("paperclip-workspace-runtime-"); + db = createDb(tempDb.connectionString); + }, 20_000); + + afterAll(async () => { + await tempDb?.cleanup(); + }); + + afterEach(async () => { + await db.delete(workspaceRuntimeServices); + await db.delete(executionWorkspaces); + await db.delete(projects); + await db.delete(heartbeatRuns); + await db.delete(agents); + await db.delete(companies); + }); + + it("adopts a live auto-port shared service after runtime state is reset", async () => { + const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-runtime-reconcile-")); + const paperclipHome = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-runtime-home-")); + process.env.PAPERCLIP_HOME = paperclipHome; + process.env.PAPERCLIP_INSTANCE_ID = `runtime-reconcile-${randomUUID()}`; + + const companyId = randomUUID(); + const agentId = randomUUID(); + const runId = randomUUID(); + const executionWorkspaceId = randomUUID(); + + await db.insert(companies).values({ + id: companyId, + name: "Paperclip", + issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`, + requireBoardApprovalForNewAgents: false, + }); + await db.insert(agents).values({ + id: agentId, + companyId, + name: "Codex Coder", + role: "engineer", + status: "active", + adapterType: "codex_local", + adapterConfig: {}, + runtimeConfig: {}, + permissions: {}, + }); + await db.insert(heartbeatRuns).values({ + id: runId, + companyId, + agentId, + invocationSource: "manual", + status: "running", + startedAt: new Date(), + updatedAt: new Date(), + }); + + const workspace = { + ...buildWorkspace(workspaceRoot), + projectId: null, + workspaceId: null, + }; + leasedRunIds.add(runId); + + const services = await ensureRuntimeServicesForRun({ + db, + runId, + agent: { + id: agentId, + name: "Codex Coder", + companyId, + }, + issue: null, + workspace, + config: { + workspaceRuntime: { + services: [ + { + name: "web", + command: + "node -e \"require('node:http').createServer((req,res)=>res.end('ok')).listen(Number(process.env.PORT), '127.0.0.1')\"", + port: { type: "auto" }, + readiness: { + type: "http", + urlTemplate: "http://127.0.0.1:{{port}}", + timeoutSec: 10, + intervalMs: 100, + }, + lifecycle: "shared", + reuseScope: "agent", + stopPolicy: { + type: "manual", + }, + }, + ], + }, + }, + adapterEnv: {}, + }); + + expect(services).toHaveLength(1); + const service = services[0]; + expect(service?.url).toMatch(/^http:\/\/127\.0\.0\.1:\d+$/); + await expect(fetch(service!.url!)).resolves.toMatchObject({ ok: true }); + + await resetRuntimeServicesForTests(); + + const result = await reconcilePersistedRuntimeServicesOnStartup(db); + expect(result).toMatchObject({ reconciled: 1, adopted: 1, stopped: 0 }); + + const persisted = await db + .select() + .from(workspaceRuntimeServices) + .where(eq(workspaceRuntimeServices.id, service!.id)) + .then((rows) => rows[0] ?? null); + expect(persisted?.status).toBe("running"); + expect(persisted?.providerRef).toMatch(/^\d+$/); + + await stopRuntimeServicesForExecutionWorkspace({ + db, + executionWorkspaceId, + workspaceCwd: workspace.cwd, + }); + + await expect(fetch(service!.url!)).rejects.toThrow(); + }); + + it("persists controlled execution workspace stops as stopped", async () => { + const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-runtime-stop-persisted-")); + const companyId = randomUUID(); + const agentId = randomUUID(); + const projectId = randomUUID(); + const runId = randomUUID(); + const executionWorkspaceId = randomUUID(); + + await db.insert(companies).values({ + id: companyId, + name: "Paperclip", + issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`, + requireBoardApprovalForNewAgents: false, + }); + await db.insert(agents).values({ + id: agentId, + companyId, + name: "Codex Coder", + role: "engineer", + status: "active", + adapterType: "codex_local", + adapterConfig: {}, + runtimeConfig: {}, + permissions: {}, + }); + await db.insert(projects).values({ + id: projectId, + companyId, + name: "Runtime stop test", + status: "active", + }); + await db.insert(executionWorkspaces).values({ + id: executionWorkspaceId, + companyId, + projectId, + mode: "isolated_workspace", + strategyType: "git_worktree", + name: "Execution workspace stop test", + status: "active", + cwd: workspaceRoot, + providerType: "local_fs", + providerRef: workspaceRoot, + }); + await db.insert(heartbeatRuns).values({ + id: runId, + companyId, + agentId, + invocationSource: "manual", + status: "running", + startedAt: new Date(), + updatedAt: new Date(), + }); + + const workspace = { + ...buildWorkspace(workspaceRoot), + projectId: null, + workspaceId: null, + }; + leasedRunIds.add(runId); + + const services = await ensureRuntimeServicesForRun({ + db, + runId, + agent: { + id: agentId, + name: "Codex Coder", + companyId, + }, + issue: null, + workspace, + executionWorkspaceId, + config: { + workspaceRuntime: { + services: [ + { + name: "web", + command: + "node -e \"require('node:http').createServer((req,res)=>res.end('ok')).listen(Number(process.env.PORT), '127.0.0.1')\"", + port: { type: "auto" }, + readiness: { + type: "http", + urlTemplate: "http://127.0.0.1:{{port}}", + timeoutSec: 10, + intervalMs: 100, + }, + lifecycle: "shared", + reuseScope: "execution_workspace", + stopPolicy: { + type: "manual", + }, + }, + ], + }, + }, + adapterEnv: {}, + }); + + expect(services[0]?.url).toBeTruthy(); + + await stopRuntimeServicesForExecutionWorkspace({ + db, + executionWorkspaceId, + workspaceCwd: workspace.cwd, + }); + await releaseRuntimeServicesForRun(runId); + leasedRunIds.delete(runId); + await new Promise((resolve) => setTimeout(resolve, 250)); + + await expect(fetch(services[0]!.url!)).rejects.toThrow(); + + const persisted = await db + .select() + .from(workspaceRuntimeServices) + .where(eq(workspaceRuntimeServices.id, services[0]!.id)) + .then((rows) => rows[0] ?? null); + + expect(persisted?.status).toBe("stopped"); + expect(persisted?.healthStatus).toBe("unknown"); + expect(persisted?.stoppedAt).toBeTruthy(); + }); +}); + describe("normalizeAdapterManagedRuntimeServices", () => { it("fills workspace defaults and derives stable ids for adapter-managed services", () => { const workspace = buildWorkspace("/tmp/project"); diff --git a/server/src/adapters/process/execute.ts b/server/src/adapters/process/execute.ts index 1d0cf9f6741..ff2bf82e85d 100644 --- a/server/src/adapters/process/execute.ts +++ b/server/src/adapters/process/execute.ts @@ -5,7 +5,9 @@ import { asStringArray, parseObject, buildPaperclipEnv, - redactEnvForLogs, + buildInvocationEnvForLogs, + ensurePathInEnv, + resolveCommandForLogs, runChildProcess, } from "../utils.js"; @@ -21,6 +23,13 @@ export async function execute(ctx: AdapterExecutionContext): Promise; + includeRuntimeKeys?: string[]; + resolvedCommand?: string | null; + resolvedCommandEnvKey?: string; +}; + +export const runningProcesses: Map = + serverUtils.runningProcesses; +export const MAX_CAPTURE_BYTES = serverUtils.MAX_CAPTURE_BYTES; +export const MAX_EXCERPT_BYTES = serverUtils.MAX_EXCERPT_BYTES; +export const parseObject = serverUtils.parseObject; +export const asString = serverUtils.asString; +export const asNumber = serverUtils.asNumber; +export const asBoolean = serverUtils.asBoolean; +export const asStringArray = serverUtils.asStringArray; +export const parseJson = serverUtils.parseJson; +export const appendWithCap = serverUtils.appendWithCap; +export const resolvePathValue = serverUtils.resolvePathValue; +export const renderTemplate = serverUtils.renderTemplate; +export const redactEnvForLogs = serverUtils.redactEnvForLogs; +export const buildPaperclipEnv = serverUtils.buildPaperclipEnv; +export const defaultPathForPlatform = serverUtils.defaultPathForPlatform; +export const ensurePathInEnv = serverUtils.ensurePathInEnv; +export const ensureAbsoluteDirectory = serverUtils.ensureAbsoluteDirectory; +export const ensureCommandResolvable = serverUtils.ensureCommandResolvable; +export const resolveCommandForLogs = serverUtils.resolveCommandForLogs; + +export function buildInvocationEnvForLogs( + env: Record, + options: BuildInvocationEnvForLogsOptions = {}, +): Record { + const maybeBuildInvocationEnvForLogs = ( + serverUtils as typeof serverUtils & { + buildInvocationEnvForLogs?: ( + env: Record, + options?: BuildInvocationEnvForLogsOptions, + ) => Record; + } + ).buildInvocationEnvForLogs; + + if (typeof maybeBuildInvocationEnvForLogs === "function") { + return maybeBuildInvocationEnvForLogs(env, options); + } + + const merged: Record = { ...env }; + const runtimeEnv = options.runtimeEnv ?? {}; + + for (const key of options.includeRuntimeKeys ?? []) { + if (key in merged) continue; + const value = runtimeEnv[key]; + if (typeof value !== "string" || value.length === 0) continue; + merged[key] = value; + } + + const resolvedCommand = options.resolvedCommand?.trim(); + if (resolvedCommand) { + merged[options.resolvedCommandEnvKey ?? "PAPERCLIP_RESOLVED_COMMAND"] = resolvedCommand; + } + + return redactEnvForLogs(merged); +} // Re-export runChildProcess with the server's pino logger wired in. -import { runChildProcess as _runChildProcess } from "@paperclipai/adapter-utils/server-utils"; import type { RunProcessResult } from "@paperclipai/adapter-utils/server-utils"; +const _runChildProcess = serverUtils.runChildProcess; export async function runChildProcess( runId: string, diff --git a/server/src/onboarding-assets/ceo/HEARTBEAT.md b/server/src/onboarding-assets/ceo/HEARTBEAT.md index 161348a25ad..bc6273c73c8 100644 --- a/server/src/onboarding-assets/ceo/HEARTBEAT.md +++ b/server/src/onboarding-assets/ceo/HEARTBEAT.md @@ -37,7 +37,7 @@ If `PAPERCLIP_APPROVAL_ID` is set: ## 6. Delegation -- Create subtasks with `POST /api/companies/{companyId}/issues`. Always set `parentId` and `goalId`. +- Create subtasks with `POST /api/companies/{companyId}/issues`. Always set `parentId` and `goalId`. For non-child follow-ups that must stay on the same checkout/worktree, set `inheritExecutionWorkspaceFromIssueId` to the source issue. - Use `paperclip-create-agent` skill when hiring new agents. - Assign work to the right agent for the job. diff --git a/server/src/routes/access.ts b/server/src/routes/access.ts index 1ac76aa8f1b..958a0a0ffc7 100644 --- a/server/src/routes/access.ts +++ b/server/src/routes/access.ts @@ -872,7 +872,7 @@ function toInviteSummaryResponse( req: Request, token: string, invite: typeof invites.$inferSelect, - companyName: string | null = null, + companyName: string | null = null ) { const baseUrl = requestBaseUrl(req); const onboardingPath = `/api/invites/${token}/onboarding`; @@ -900,18 +900,6 @@ function toInviteSummaryResponse( }; } -async function resolveInviteCompanyName( - db: Db, - companyId: string | null, -): Promise { - if (!companyId) return null; - return db - .select({ name: companies.name }) - .from(companies) - .where(eq(companies.id, companyId)) - .then((rows) => rows[0]?.name ?? null); -} - function buildOnboardingDiscoveryDiagnostics(input: { apiBaseUrl: string; deploymentMode: DeploymentMode; @@ -1023,6 +1011,7 @@ function buildInviteOnboardingManifest( token: string, invite: typeof invites.$inferSelect, opts: { + companyName?: string | null; deploymentMode: DeploymentMode; deploymentExposure: DeploymentExposure; bindHost: string; @@ -1055,7 +1044,12 @@ function buildInviteOnboardingManifest( }); return { - invite: toInviteSummaryResponse(req, token, invite, companyName), + invite: toInviteSummaryResponse( + req, + token, + invite, + opts.companyName ?? null + ), onboarding: { instructions: "Join as an OpenClaw Gateway agent, save your one-time claim secret, wait for board approval, then claim your API key. Save the claim response token to ~/.openclaw/workspace/paperclip-claimed-api-key.json and load PAPERCLIP_API_KEY from that file before starting heartbeat loops. You MUST submit adapterType='openclaw_gateway', set agentDefaultsPayload.url to your ws:// or wss:// OpenClaw gateway endpoint, and include agentDefaultsPayload.headers.x-openclaw-token (or legacy x-openclaw-auth).", @@ -1115,6 +1109,7 @@ export function buildInviteOnboardingTextDocument( token: string, invite: typeof invites.$inferSelect, opts: { + companyName?: string | null; deploymentMode: DeploymentMode; deploymentExposure: DeploymentExposure; bindHost: string; @@ -1164,6 +1159,10 @@ export function buildInviteOnboardingTextDocument( - expiresAt: ${invite.expiresAt.toISOString()} `); + if (manifest.invite.companyName) { + lines.push(`- companyName: ${manifest.invite.companyName}`); + } + if (onboarding.inviteMessage) { appendBlock(` ## Message from inviter @@ -1908,6 +1907,16 @@ export function accessRoutes( return { token, created, normalizedAgentMessage }; } + async function getInviteCompanyName(companyId: string | null) { + if (!companyId) return null; + const company = await db + .select({ name: companies.name }) + .from(companies) + .where(eq(companies.id, companyId)) + .then((rows) => rows[0] ?? null); + return company?.name ?? null; + } + router.get("/skills/available", (_req, res) => { res.json({ skills: listAvailableSkills() }); }); @@ -1976,16 +1985,18 @@ export function accessRoutes( } }); + const companyName = await getInviteCompanyName(created.companyId); const inviteSummary = toInviteSummaryResponse( req, token, created, - await resolveInviteCompanyName(db, created.companyId), + companyName ); res.status(201).json({ ...created, token, inviteUrl: `/invite/${token}`, + companyName, onboardingTextPath: inviteSummary.onboardingTextPath, onboardingTextUrl: inviteSummary.onboardingTextUrl, inviteMessage: inviteSummary.inviteMessage @@ -2026,16 +2037,18 @@ export function accessRoutes( } }); + const companyName = await getInviteCompanyName(created.companyId); const inviteSummary = toInviteSummaryResponse( req, token, created, - await resolveInviteCompanyName(db, created.companyId), + companyName ); res.status(201).json({ ...created, token, inviteUrl: `/invite/${token}`, + companyName, onboardingTextPath: inviteSummary.onboardingTextPath, onboardingTextUrl: inviteSummary.onboardingTextUrl, inviteMessage: inviteSummary.inviteMessage @@ -2060,14 +2073,8 @@ export function accessRoutes( throw notFound("Invite not found"); } - res.json( - toInviteSummaryResponse( - req, - token, - invite, - await resolveInviteCompanyName(db, invite.companyId), - ), - ); + const companyName = await getInviteCompanyName(invite.companyId); + res.json(toInviteSummaryResponse(req, token, invite, companyName)); }); router.get("/invites/:token/onboarding", async (req, res) => { @@ -2082,15 +2089,11 @@ export function accessRoutes( throw notFound("Invite not found"); } - res.json( - buildInviteOnboardingManifest( - req, - token, - invite, - opts, - await resolveInviteCompanyName(db, invite.companyId), - ), - ); + const companyName = await getInviteCompanyName(invite.companyId); + res.json(buildInviteOnboardingManifest(req, token, invite, { + ...opts, + companyName + })); }); router.get("/invites/:token/onboarding.txt", async (req, res) => { @@ -2105,9 +2108,15 @@ export function accessRoutes( throw notFound("Invite not found"); } + const companyName = await getInviteCompanyName(invite.companyId); res .type("text/plain; charset=utf-8") - .send(buildInviteOnboardingTextDocument(req, token, invite, opts)); + .send( + buildInviteOnboardingTextDocument(req, token, invite, { + ...opts, + companyName + }) + ); }); router.get("/invites/:token/test-resolution", async (req, res) => { @@ -2557,11 +2566,15 @@ export function accessRoutes( const response = toJoinRequestResponse(created); if (claimSecret) { + const companyName = await getInviteCompanyName(invite.companyId); const onboardingManifest = buildInviteOnboardingManifest( req, token, invite, - opts + { + ...opts, + companyName + } ); res.status(202).json({ ...response, diff --git a/server/src/routes/agents.ts b/server/src/routes/agents.ts index b496457855d..2ad85e63133 100644 --- a/server/src/routes/agents.ts +++ b/server/src/routes/agents.ts @@ -6,6 +6,7 @@ import { agents as agentsTable, companies, heartbeatRuns } from "@paperclipai/db import { and, desc, eq, inArray, not, sql } from "drizzle-orm"; import { agentSkillSyncSchema, + agentMineInboxQuerySchema, createAgentKeySchema, createAgentHireSchema, createAgentSchema, @@ -1006,6 +1007,23 @@ export function agentRoutes(db: Db) { ); }); + router.get("/agents/me/inbox/mine", async (req, res) => { + if (req.actor.type !== "agent" || !req.actor.agentId || !req.actor.companyId) { + res.status(401).json({ error: "Agent authentication required" }); + return; + } + + const query = agentMineInboxQuerySchema.parse(req.query); + const issuesSvc = issueService(db); + const rows = await issuesSvc.list(req.actor.companyId, { + touchedByUserId: query.userId, + inboxArchivedByUserId: query.userId, + status: query.status, + }); + + res.json(rows); + }); + router.get("/agents/:id", async (req, res) => { const id = req.params.id as string; const agent = await svc.getById(id); diff --git a/server/src/routes/execution-workspaces.ts b/server/src/routes/execution-workspaces.ts index a7276704631..4fa204255a0 100644 --- a/server/src/routes/execution-workspaces.ts +++ b/server/src/routes/execution-workspaces.ts @@ -5,15 +5,16 @@ import { issues, projects, projectWorkspaces } from "@paperclipai/db"; import { updateExecutionWorkspaceSchema } from "@paperclipai/shared"; import { validate } from "../middleware/validate.js"; import { executionWorkspaceService, logActivity, workspaceOperationService } from "../services/index.js"; +import { mergeExecutionWorkspaceConfig, readExecutionWorkspaceConfig } from "../services/execution-workspaces.js"; import { parseProjectExecutionWorkspacePolicy } from "../services/execution-workspace-policy.js"; +import { readProjectWorkspaceRuntimeConfig } from "../services/project-workspace-runtime-config.js"; import { cleanupExecutionWorkspaceArtifacts, + startRuntimeServicesForWorkspaceControl, stopRuntimeServicesForExecutionWorkspace, } from "../services/workspace-runtime.js"; import { assertCompanyAccess, getActorInfo } from "./authz.js"; -const TERMINAL_ISSUE_STATUSES = new Set(["done", "cancelled"]); - export function executionWorkspaceRoutes(db: Db) { const router = Router(); const svc = executionWorkspaceService(db); @@ -43,6 +44,202 @@ export function executionWorkspaceRoutes(db: Db) { res.json(workspace); }); + router.get("/execution-workspaces/:id/close-readiness", async (req, res) => { + const id = req.params.id as string; + const workspace = await svc.getById(id); + if (!workspace) { + res.status(404).json({ error: "Execution workspace not found" }); + return; + } + assertCompanyAccess(req, workspace.companyId); + const readiness = await svc.getCloseReadiness(id); + if (!readiness) { + res.status(404).json({ error: "Execution workspace not found" }); + return; + } + res.json(readiness); + }); + + router.get("/execution-workspaces/:id/workspace-operations", async (req, res) => { + const id = req.params.id as string; + const workspace = await svc.getById(id); + if (!workspace) { + res.status(404).json({ error: "Execution workspace not found" }); + return; + } + assertCompanyAccess(req, workspace.companyId); + const operations = await workspaceOperationsSvc.listForExecutionWorkspace(id); + res.json(operations); + }); + + router.post("/execution-workspaces/:id/runtime-services/:action", async (req, res) => { + const id = req.params.id as string; + const action = String(req.params.action ?? "").trim().toLowerCase(); + if (action !== "start" && action !== "stop" && action !== "restart") { + res.status(404).json({ error: "Runtime service action not found" }); + return; + } + + const existing = await svc.getById(id); + if (!existing) { + res.status(404).json({ error: "Execution workspace not found" }); + return; + } + assertCompanyAccess(req, existing.companyId); + + const workspaceCwd = existing.cwd; + if (!workspaceCwd) { + res.status(422).json({ error: "Execution workspace needs a local path before Paperclip can manage local runtime services" }); + return; + } + + const projectWorkspace = existing.projectWorkspaceId + ? await db + .select({ + id: projectWorkspaces.id, + cwd: projectWorkspaces.cwd, + repoUrl: projectWorkspaces.repoUrl, + repoRef: projectWorkspaces.repoRef, + defaultRef: projectWorkspaces.defaultRef, + metadata: projectWorkspaces.metadata, + }) + .from(projectWorkspaces) + .where( + and( + eq(projectWorkspaces.id, existing.projectWorkspaceId), + eq(projectWorkspaces.companyId, existing.companyId), + ), + ) + .then((rows) => rows[0] ?? null) + : null; + const projectWorkspaceRuntime = readProjectWorkspaceRuntimeConfig( + (projectWorkspace?.metadata as Record | null) ?? null, + )?.workspaceRuntime ?? null; + const effectiveRuntimeConfig = existing.config?.workspaceRuntime ?? projectWorkspaceRuntime ?? null; + + if ((action === "start" || action === "restart") && !effectiveRuntimeConfig) { + res.status(422).json({ error: "Execution workspace has no runtime service configuration or inherited project workspace default" }); + return; + } + + const actor = getActorInfo(req); + const recorder = workspaceOperationsSvc.createRecorder({ + companyId: existing.companyId, + executionWorkspaceId: existing.id, + }); + let runtimeServiceCount = existing.runtimeServices?.length ?? 0; + const stdout: string[] = []; + const stderr: string[] = []; + + const operation = await recorder.recordOperation({ + phase: action === "stop" ? "workspace_teardown" : "workspace_provision", + command: `workspace runtime ${action}`, + cwd: existing.cwd, + metadata: { + action, + executionWorkspaceId: existing.id, + }, + run: async () => { + const onLog = async (stream: "stdout" | "stderr", chunk: string) => { + if (stream === "stdout") stdout.push(chunk); + else stderr.push(chunk); + }; + + if (action === "stop" || action === "restart") { + await stopRuntimeServicesForExecutionWorkspace({ + db, + executionWorkspaceId: existing.id, + workspaceCwd, + }); + } + + if (action === "start" || action === "restart") { + const startedServices = await startRuntimeServicesForWorkspaceControl({ + db, + actor: { + id: actor.agentId ?? null, + name: actor.actorType === "user" ? "Board" : "Agent", + companyId: existing.companyId, + }, + issue: existing.sourceIssueId + ? { + id: existing.sourceIssueId, + identifier: null, + title: existing.name, + } + : null, + workspace: { + baseCwd: workspaceCwd, + source: existing.mode === "shared_workspace" ? "project_primary" : "task_session", + projectId: existing.projectId, + workspaceId: existing.projectWorkspaceId, + repoUrl: existing.repoUrl, + repoRef: existing.baseRef, + strategy: existing.strategyType === "git_worktree" ? "git_worktree" : "project_primary", + cwd: workspaceCwd, + branchName: existing.branchName, + worktreePath: existing.strategyType === "git_worktree" ? workspaceCwd : null, + warnings: [], + created: false, + }, + executionWorkspaceId: existing.id, + config: { workspaceRuntime: effectiveRuntimeConfig }, + adapterEnv: {}, + onLog, + }); + runtimeServiceCount = startedServices.length; + } else { + runtimeServiceCount = 0; + } + + const metadata = mergeExecutionWorkspaceConfig(existing.metadata as Record | null, { + desiredState: action === "stop" ? "stopped" : "running", + }); + await svc.update(existing.id, { metadata }); + + return { + status: "succeeded", + stdout: stdout.join(""), + stderr: stderr.join(""), + system: + action === "stop" + ? "Stopped execution workspace runtime services.\n" + : action === "restart" + ? "Restarted execution workspace runtime services.\n" + : "Started execution workspace runtime services.\n", + metadata: { + runtimeServiceCount, + }, + }; + }, + }); + + const workspace = await svc.getById(id); + if (!workspace) { + res.status(404).json({ error: "Execution workspace not found" }); + return; + } + + await logActivity(db, { + companyId: existing.companyId, + actorType: actor.actorType, + actorId: actor.actorId, + agentId: actor.agentId, + runId: actor.runId, + action: `execution_workspace.runtime_${action}`, + entityType: "execution_workspace", + entityId: existing.id, + details: { + runtimeServiceCount, + }, + }); + + res.json({ + workspace, + operation, + }); + }); + router.patch("/execution-workspaces/:id", validate(updateExecutionWorkspaceSchema), async (req, res) => { const id = req.params.id as string; const existing = await svc.getById(id); @@ -52,25 +249,43 @@ export function executionWorkspaceRoutes(db: Db) { } assertCompanyAccess(req, existing.companyId); const patch: Record = { - ...req.body, - ...(req.body.cleanupEligibleAt ? { cleanupEligibleAt: new Date(req.body.cleanupEligibleAt) } : {}), + ...(req.body.name === undefined ? {} : { name: req.body.name }), + ...(req.body.cwd === undefined ? {} : { cwd: req.body.cwd }), + ...(req.body.repoUrl === undefined ? {} : { repoUrl: req.body.repoUrl }), + ...(req.body.baseRef === undefined ? {} : { baseRef: req.body.baseRef }), + ...(req.body.branchName === undefined ? {} : { branchName: req.body.branchName }), + ...(req.body.providerRef === undefined ? {} : { providerRef: req.body.providerRef }), + ...(req.body.status === undefined ? {} : { status: req.body.status }), + ...(req.body.cleanupReason === undefined ? {} : { cleanupReason: req.body.cleanupReason }), + ...(req.body.cleanupEligibleAt !== undefined + ? { cleanupEligibleAt: req.body.cleanupEligibleAt ? new Date(req.body.cleanupEligibleAt) : null } + : {}), }; + if (req.body.metadata !== undefined || req.body.config !== undefined) { + const requestedMetadata = req.body.metadata === undefined + ? (existing.metadata as Record | null) + : (req.body.metadata as Record | null); + patch.metadata = req.body.config === undefined + ? requestedMetadata + : mergeExecutionWorkspaceConfig(requestedMetadata, req.body.config ?? null); + } let workspace = existing; let cleanupWarnings: string[] = []; + const configForCleanup = readExecutionWorkspaceConfig( + ((patch.metadata as Record | null | undefined) ?? (existing.metadata as Record | null)) ?? null, + ); if (req.body.status === "archived" && existing.status !== "archived") { - const linkedIssues = await db - .select({ - id: issues.id, - status: issues.status, - }) - .from(issues) - .where(and(eq(issues.companyId, existing.companyId), eq(issues.executionWorkspaceId, existing.id))); - const activeLinkedIssues = linkedIssues.filter((issue) => !TERMINAL_ISSUE_STATUSES.has(issue.status)); + const readiness = await svc.getCloseReadiness(existing.id); + if (!readiness) { + res.status(404).json({ error: "Execution workspace not found" }); + return; + } - if (activeLinkedIssues.length > 0) { + if (readiness.state === "blocked") { res.status(409).json({ - error: `Cannot archive execution workspace while ${activeLinkedIssues.length} linked issue(s) are still open`, + error: readiness.blockingReasons[0] ?? "Execution workspace cannot be closed right now", + closeReadiness: readiness, }); return; } @@ -88,6 +303,21 @@ export function executionWorkspaceRoutes(db: Db) { } workspace = archivedWorkspace; + if (existing.mode === "shared_workspace") { + await db + .update(issues) + .set({ + executionWorkspaceId: null, + updatedAt: new Date(), + }) + .where( + and( + eq(issues.companyId, existing.companyId), + eq(issues.executionWorkspaceId, existing.id), + ), + ); + } + try { await stopRuntimeServicesForExecutionWorkspace({ db, @@ -101,7 +331,7 @@ export function executionWorkspaceRoutes(db: Db) { cleanupCommand: projectWorkspaces.cleanupCommand, }) .from(projectWorkspaces) - .where( + .where( and( eq(projectWorkspaces.id, existing.projectWorkspaceId), eq(projectWorkspaces.companyId, existing.companyId), @@ -121,7 +351,8 @@ export function executionWorkspaceRoutes(db: Db) { const cleanupResult = await cleanupExecutionWorkspaceArtifacts({ workspace: existing, projectWorkspace, - teardownCommand: projectPolicy?.workspaceStrategy?.teardownCommand ?? null, + teardownCommand: configForCleanup?.teardownCommand ?? projectPolicy?.workspaceStrategy?.teardownCommand ?? null, + cleanupCommand: configForCleanup?.cleanupCommand ?? null, recorder: workspaceOperationsSvc.createRecorder({ companyId: existing.companyId, executionWorkspaceId: existing.id, diff --git a/server/src/routes/health.ts b/server/src/routes/health.ts index 2cf416696a9..b965761a0d9 100644 --- a/server/src/routes/health.ts +++ b/server/src/routes/health.ts @@ -31,13 +31,13 @@ export function healthRoutes( } try { - await db.execute(sql`select 1`); + await db.execute(sql`SELECT 1`); } catch (error) { logger.warn({ err: error }, "Health check database probe failed"); res.status(503).json({ - status: "error", - error: "Database unavailable", + status: "unhealthy", version: serverVersion, + error: "database_unreachable" }); return; } diff --git a/server/src/routes/issues.ts b/server/src/routes/issues.ts index 11ec162c965..688547aeb2e 100644 --- a/server/src/routes/issues.ts +++ b/server/src/routes/issues.ts @@ -1,5 +1,6 @@ import { Router, type Request, type Response } from "express"; import multer from "multer"; +import { z } from "zod"; import type { Db } from "@paperclipai/db"; import { addIssueCommentSchema, @@ -38,6 +39,9 @@ import { isAllowedContentType, MAX_ATTACHMENT_BYTES } from "../attachment-types. import { queueIssueAssignmentWakeup } from "../services/issue-assignment-wakeup.js"; const MAX_ISSUE_COMMENT_LIMIT = 500; +const updateIssueRouteSchema = updateIssueSchema.extend({ + interrupt: z.boolean().optional(), +}); export function issueRoutes(db: Db, storage: StorageService) { const router = Router(); @@ -161,6 +165,30 @@ export function issueRoutes(db: Db, storage: StorageService) { return true; } + async function resolveActiveIssueRun(issue: { + id: string; + assigneeAgentId: string | null; + executionRunId?: string | null; + }) { + let runToInterrupt = issue.executionRunId ? await heartbeat.getRun(issue.executionRunId) : null; + + if ((!runToInterrupt || runToInterrupt.status !== "running") && issue.assigneeAgentId) { + const activeRun = await heartbeat.getActiveRunForAgent(issue.assigneeAgentId); + const activeIssueId = + activeRun && + activeRun.contextSnapshot && + typeof activeRun.contextSnapshot === "object" && + typeof (activeRun.contextSnapshot as Record).issueId === "string" + ? ((activeRun.contextSnapshot as Record).issueId as string) + : null; + if (activeRun && activeRun.status === "running" && activeIssueId === issue.id) { + runToInterrupt = activeRun; + } + } + + return runToInterrupt?.status === "running" ? runToInterrupt : null; + } + async function normalizeIssueIdentifier(rawId: string): Promise { if (/^[A-Z]+-\d+$/i.test(rawId)) { const issue = await svc.getByIdentifier(rawId); @@ -275,6 +303,7 @@ export function issueRoutes(db: Db, storage: StorageService) { inboxArchivedByUserId, unreadForUserId, projectId: req.query.projectId as string | undefined, + executionWorkspaceId: req.query.executionWorkspaceId as string | undefined, parentId: req.query.parentId as string | undefined, labelId: req.query.labelId as string | undefined, originKind: req.query.originKind as string | undefined, @@ -713,6 +742,38 @@ export function issueRoutes(db: Db, storage: StorageService) { res.json(readState); }); + router.delete("/issues/:id/read", async (req, res) => { + const id = req.params.id as string; + const issue = await svc.getById(id); + if (!issue) { + res.status(404).json({ error: "Issue not found" }); + return; + } + assertCompanyAccess(req, issue.companyId); + if (req.actor.type !== "board") { + res.status(403).json({ error: "Board authentication required" }); + return; + } + if (!req.actor.userId) { + res.status(403).json({ error: "Board user context required" }); + return; + } + const removed = await svc.markUnread(issue.companyId, issue.id, req.actor.userId); + const actor = getActorInfo(req); + await logActivity(db, { + companyId: issue.companyId, + actorType: actor.actorType, + actorId: actor.actorId, + agentId: actor.agentId, + runId: actor.runId, + action: "issue.read_unmarked", + entityType: "issue", + entityId: issue.id, + details: { userId: req.actor.userId }, + }); + res.json({ id: issue.id, removed }); + }); + router.post("/issues/:id/inbox-archive", async (req, res) => { const id = req.params.id as string; const issue = await svc.getById(id); @@ -887,7 +948,7 @@ export function issueRoutes(db: Db, storage: StorageService) { res.status(201).json(issue); }); - router.patch("/issues/:id", validate(updateIssueSchema), async (req, res) => { + router.patch("/issues/:id", validate(updateIssueRouteSchema), async (req, res) => { const id = req.params.id as string; const existing = await svc.getById(id); if (!existing) { @@ -917,7 +978,45 @@ export function issueRoutes(db: Db, storage: StorageService) { const actor = getActorInfo(req); const isClosed = existing.status === "done" || existing.status === "cancelled"; - const { comment: commentBody, reopen: reopenRequested, hiddenAt: hiddenAtRaw, ...updateFields } = req.body; + const { + comment: commentBody, + reopen: reopenRequested, + interrupt: interruptRequested, + hiddenAt: hiddenAtRaw, + ...updateFields + } = req.body; + let interruptedRunId: string | null = null; + + if (interruptRequested) { + if (!commentBody) { + res.status(400).json({ error: "Interrupt is only supported when posting a comment" }); + return; + } + if (req.actor.type !== "board") { + res.status(403).json({ error: "Only board users can interrupt active runs from issue comments" }); + return; + } + + const runToInterrupt = await resolveActiveIssueRun(existing); + if (runToInterrupt) { + const cancelled = await heartbeat.cancelRun(runToInterrupt.id); + if (cancelled) { + interruptedRunId = cancelled.id; + await logActivity(db, { + companyId: cancelled.companyId, + actorType: actor.actorType, + actorId: actor.actorId, + agentId: actor.agentId, + runId: actor.runId, + action: "heartbeat.cancelled", + entityType: "heartbeat_run", + entityId: cancelled.id, + details: { agentId: cancelled.agentId, source: "issue_comment_interrupt", issueId: existing.id }, + }); + } + } + } + if (hiddenAtRaw !== undefined) { updateFields.hiddenAt = hiddenAtRaw ? new Date(hiddenAtRaw) : null; } @@ -992,6 +1091,7 @@ export function issueRoutes(db: Db, storage: StorageService) { identifier: issue.identifier, ...(commentBody ? { source: "comment" } : {}), ...(reopened ? { reopened: true, reopenedFrom: reopenFromStatus } : {}), + ...(interruptedRunId ? { interruptedRunId } : {}), _previous: hasFieldChanges ? previous : undefined, }, }); @@ -1018,6 +1118,7 @@ export function issueRoutes(db: Db, storage: StorageService) { identifier: issue.identifier, issueTitle: issue.title, ...(reopened ? { reopened: true, reopenedFrom: reopenFromStatus, source: "comment" } : {}), + ...(interruptedRunId ? { interruptedRunId } : {}), ...(hasFieldChanges ? { updated: true } : {}), }, }); @@ -1039,10 +1140,18 @@ export function issueRoutes(db: Db, storage: StorageService) { source: "assignment", triggerDetail: "system", reason: "issue_assigned", - payload: { issueId: issue.id, mutation: "update" }, + payload: { + issueId: issue.id, + mutation: "update", + ...(interruptedRunId ? { interruptedRunId } : {}), + }, requestedByActorType: actor.actorType, requestedByActorId: actor.actorId, - contextSnapshot: { issueId: issue.id, source: "issue.update" }, + contextSnapshot: { + issueId: issue.id, + source: "issue.update", + ...(interruptedRunId ? { interruptedRunId } : {}), + }, }); } @@ -1051,10 +1160,18 @@ export function issueRoutes(db: Db, storage: StorageService) { source: "automation", triggerDetail: "system", reason: "issue_status_changed", - payload: { issueId: issue.id, mutation: "update" }, + payload: { + issueId: issue.id, + mutation: "update", + ...(interruptedRunId ? { interruptedRunId } : {}), + }, requestedByActorType: actor.actorType, requestedByActorId: actor.actorId, - contextSnapshot: { issueId: issue.id, source: "issue.status_change" }, + contextSnapshot: { + issueId: issue.id, + source: "issue.status_change", + ...(interruptedRunId ? { interruptedRunId } : {}), + }, }); } @@ -1347,28 +1464,8 @@ export function issueRoutes(db: Db, storage: StorageService) { return; } - let runToInterrupt = currentIssue.executionRunId - ? await heartbeat.getRun(currentIssue.executionRunId) - : null; - - if ( - (!runToInterrupt || runToInterrupt.status !== "running") && - currentIssue.assigneeAgentId - ) { - const activeRun = await heartbeat.getActiveRunForAgent(currentIssue.assigneeAgentId); - const activeIssueId = - activeRun && - activeRun.contextSnapshot && - typeof activeRun.contextSnapshot === "object" && - typeof (activeRun.contextSnapshot as Record).issueId === "string" - ? ((activeRun.contextSnapshot as Record).issueId as string) - : null; - if (activeRun && activeRun.status === "running" && activeIssueId === currentIssue.id) { - runToInterrupt = activeRun; - } - } - - if (runToInterrupt && runToInterrupt.status === "running") { + const runToInterrupt = await resolveActiveIssueRun(currentIssue); + if (runToInterrupt) { const cancelled = await heartbeat.cancelRun(runToInterrupt.id); if (cancelled) { interruptedRunId = cancelled.id; diff --git a/server/src/routes/projects.ts b/server/src/routes/projects.ts index 51555ff5c15..b200b35472f 100644 --- a/server/src/routes/projects.ts +++ b/server/src/routes/projects.ts @@ -8,13 +8,15 @@ import { updateProjectWorkspaceSchema, } from "@paperclipai/shared"; import { validate } from "../middleware/validate.js"; -import { projectService, logActivity } from "../services/index.js"; +import { projectService, logActivity, workspaceOperationService } from "../services/index.js"; import { conflict } from "../errors.js"; import { assertCompanyAccess, getActorInfo } from "./authz.js"; +import { startRuntimeServicesForWorkspaceControl, stopRuntimeServicesForProjectWorkspace } from "../services/workspace-runtime.js"; export function projectRoutes(db: Db) { const router = Router(); const svc = projectService(db); + const workspaceOperations = workspaceOperationService(db); async function resolveCompanyIdForProjectReference(req: Request) { const companyIdQuery = req.query.companyId; @@ -229,6 +231,145 @@ export function projectRoutes(db: Db) { }, ); + router.post("/projects/:id/workspaces/:workspaceId/runtime-services/:action", async (req, res) => { + const id = req.params.id as string; + const workspaceId = req.params.workspaceId as string; + const action = String(req.params.action ?? "").trim().toLowerCase(); + if (action !== "start" && action !== "stop" && action !== "restart") { + res.status(404).json({ error: "Runtime service action not found" }); + return; + } + + const project = await svc.getById(id); + if (!project) { + res.status(404).json({ error: "Project not found" }); + return; + } + assertCompanyAccess(req, project.companyId); + + const workspace = project.workspaces.find((entry) => entry.id === workspaceId) ?? null; + if (!workspace) { + res.status(404).json({ error: "Project workspace not found" }); + return; + } + + const workspaceCwd = workspace.cwd; + if (!workspaceCwd) { + res.status(422).json({ error: "Project workspace needs a local path before Paperclip can manage local runtime services" }); + return; + } + + const runtimeConfig = workspace.runtimeConfig?.workspaceRuntime ?? null; + if ((action === "start" || action === "restart") && !runtimeConfig) { + res.status(422).json({ error: "Project workspace has no runtime service configuration" }); + return; + } + + const actor = getActorInfo(req); + const recorder = workspaceOperations.createRecorder({ companyId: project.companyId }); + let runtimeServiceCount = workspace.runtimeServices?.length ?? 0; + const stdout: string[] = []; + const stderr: string[] = []; + + const operation = await recorder.recordOperation({ + phase: action === "stop" ? "workspace_teardown" : "workspace_provision", + command: `workspace runtime ${action}`, + cwd: workspace.cwd, + metadata: { + action, + projectId: project.id, + projectWorkspaceId: workspace.id, + }, + run: async () => { + const onLog = async (stream: "stdout" | "stderr", chunk: string) => { + if (stream === "stdout") stdout.push(chunk); + else stderr.push(chunk); + }; + + if (action === "stop" || action === "restart") { + await stopRuntimeServicesForProjectWorkspace({ + db, + projectWorkspaceId: workspace.id, + }); + } + + if (action === "start" || action === "restart") { + const startedServices = await startRuntimeServicesForWorkspaceControl({ + db, + actor: { + id: actor.agentId ?? null, + name: actor.actorType === "user" ? "Board" : "Agent", + companyId: project.companyId, + }, + issue: null, + workspace: { + baseCwd: workspaceCwd, + source: "project_primary", + projectId: project.id, + workspaceId: workspace.id, + repoUrl: workspace.repoUrl, + repoRef: workspace.repoRef, + strategy: "project_primary", + cwd: workspaceCwd, + branchName: workspace.defaultRef ?? workspace.repoRef ?? null, + worktreePath: null, + warnings: [], + created: false, + }, + config: { workspaceRuntime: runtimeConfig }, + adapterEnv: {}, + onLog, + }); + runtimeServiceCount = startedServices.length; + } else { + runtimeServiceCount = 0; + } + + await svc.updateWorkspace(project.id, workspace.id, { + runtimeConfig: { + desiredState: action === "stop" ? "stopped" : "running", + }, + }); + + return { + status: "succeeded", + stdout: stdout.join(""), + stderr: stderr.join(""), + system: + action === "stop" + ? "Stopped project workspace runtime services.\n" + : action === "restart" + ? "Restarted project workspace runtime services.\n" + : "Started project workspace runtime services.\n", + metadata: { + runtimeServiceCount, + }, + }; + }, + }); + + const updatedWorkspace = (await svc.listWorkspaces(project.id)).find((entry) => entry.id === workspace.id) ?? workspace; + + await logActivity(db, { + companyId: project.companyId, + actorType: actor.actorType, + actorId: actor.actorId, + agentId: actor.agentId, + action: `project.workspace_runtime_${action}`, + entityType: "project", + entityId: project.id, + details: { + projectWorkspaceId: workspace.id, + runtimeServiceCount, + }, + }); + + res.json({ + workspace: updatedWorkspace, + operation, + }); + }); + router.delete("/projects/:id/workspaces/:workspaceId", async (req, res) => { const id = req.params.id as string; const workspaceId = req.params.workspaceId as string; diff --git a/server/src/services/execution-workspaces.ts b/server/src/services/execution-workspaces.ts index ea4dd163d6c..a1d3b41d69a 100644 --- a/server/src/services/execution-workspaces.ts +++ b/server/src/services/execution-workspaces.ts @@ -1,11 +1,292 @@ +import { execFile } from "node:child_process"; +import fs from "node:fs/promises"; +import path from "node:path"; +import { promisify } from "node:util"; import { and, desc, eq, inArray } from "drizzle-orm"; import type { Db } from "@paperclipai/db"; -import { executionWorkspaces } from "@paperclipai/db"; -import type { ExecutionWorkspace } from "@paperclipai/shared"; +import { executionWorkspaces, issues, projects, projectWorkspaces, workspaceRuntimeServices } from "@paperclipai/db"; +import type { + ExecutionWorkspace, + ExecutionWorkspaceCloseAction, + ExecutionWorkspaceCloseGitReadiness, + ExecutionWorkspaceCloseReadiness, + ExecutionWorkspaceConfig, + WorkspaceRuntimeService, +} from "@paperclipai/shared"; +import { parseProjectExecutionWorkspacePolicy } from "./execution-workspace-policy.js"; type ExecutionWorkspaceRow = typeof executionWorkspaces.$inferSelect; +type WorkspaceRuntimeServiceRow = typeof workspaceRuntimeServices.$inferSelect; +const execFileAsync = promisify(execFile); +const TERMINAL_ISSUE_STATUSES = new Set(["done", "cancelled"]); -function toExecutionWorkspace(row: ExecutionWorkspaceRow): ExecutionWorkspace { +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function readNullableString(value: unknown): string | null { + if (typeof value !== "string") return null; + const trimmed = value.trim(); + return trimmed.length > 0 ? trimmed : null; +} + +function cloneRecord(value: unknown): Record | null { + if (!isRecord(value)) return null; + return { ...value }; +} + +async function pathExists(value: string | null | undefined) { + if (!value) return false; + try { + await fs.access(value); + return true; + } catch { + return false; + } +} + +async function runGit(args: string[], cwd: string) { + return await execFileAsync("git", ["-C", cwd, ...args], { cwd }); +} + +async function inspectGitCloseReadiness(workspace: ExecutionWorkspace): Promise<{ + git: ExecutionWorkspaceCloseGitReadiness | null; + warnings: string[]; +}> { + const warnings: string[] = []; + const workspacePath = readNullableString(workspace.providerRef) ?? readNullableString(workspace.cwd); + const createdByRuntime = workspace.metadata?.createdByRuntime === true; + const expectsGitInspection = + workspace.providerType === "git_worktree" || + Boolean(workspace.repoUrl || workspace.baseRef || workspace.branchName || workspacePath); + + if (!expectsGitInspection) { + return { git: null, warnings }; + } + + if (!workspacePath) { + warnings.push("Workspace has no local path, so Paperclip cannot inspect git status before close."); + return { git: null, warnings }; + } + + if (!(await pathExists(workspacePath))) { + warnings.push(`Workspace path "${workspacePath}" does not exist, so Paperclip cannot inspect git status before close.`); + return { + git: { + repoRoot: null, + workspacePath, + branchName: workspace.branchName, + baseRef: workspace.baseRef, + hasDirtyTrackedFiles: false, + hasUntrackedFiles: false, + dirtyEntryCount: 0, + untrackedEntryCount: 0, + aheadCount: null, + behindCount: null, + isMergedIntoBase: null, + createdByRuntime, + }, + warnings, + }; + } + + let repoRoot: string | null = null; + try { + repoRoot = (await runGit(["rev-parse", "--show-toplevel"], workspacePath)).stdout.trim() || null; + } catch (error) { + warnings.push( + `Could not inspect git status for "${workspacePath}": ${error instanceof Error ? error.message : String(error)}`, + ); + } + + let branchName = workspace.branchName; + if (repoRoot && !branchName) { + try { + branchName = (await runGit(["rev-parse", "--abbrev-ref", "HEAD"], workspacePath)).stdout.trim() || null; + } catch { + branchName = workspace.branchName; + } + } + + let dirtyEntryCount = 0; + let untrackedEntryCount = 0; + if (repoRoot) { + try { + const statusOutput = (await runGit(["status", "--porcelain=v1", "--untracked-files=all"], workspacePath)).stdout; + for (const line of statusOutput.split(/\r?\n/)) { + if (!line) continue; + if (line.startsWith("??")) { + untrackedEntryCount += 1; + continue; + } + dirtyEntryCount += 1; + } + } catch (error) { + warnings.push( + `Could not read git working tree status for "${workspacePath}": ${error instanceof Error ? error.message : String(error)}`, + ); + } + } + + let aheadCount: number | null = null; + let behindCount: number | null = null; + let isMergedIntoBase: boolean | null = null; + const baseRef = workspace.baseRef; + + if (repoRoot && baseRef) { + try { + const counts = (await runGit(["rev-list", "--left-right", "--count", `${baseRef}...HEAD`], workspacePath)).stdout.trim(); + const [behindRaw, aheadRaw] = counts.split(/\s+/); + behindCount = behindRaw ? Number.parseInt(behindRaw, 10) : 0; + aheadCount = aheadRaw ? Number.parseInt(aheadRaw, 10) : 0; + } catch (error) { + warnings.push( + `Could not compare this workspace against ${baseRef}: ${error instanceof Error ? error.message : String(error)}`, + ); + } + + try { + await runGit(["merge-base", "--is-ancestor", "HEAD", baseRef], workspacePath); + isMergedIntoBase = true; + } catch (error) { + const code = typeof error === "object" && error && "code" in error ? (error as { code?: unknown }).code : null; + if (code === 1) isMergedIntoBase = false; + else { + warnings.push( + `Could not determine whether this workspace is merged into ${baseRef}: ${error instanceof Error ? error.message : String(error)}`, + ); + } + } + } + + return { + git: { + repoRoot, + workspacePath, + branchName, + baseRef, + hasDirtyTrackedFiles: dirtyEntryCount > 0, + hasUntrackedFiles: untrackedEntryCount > 0, + dirtyEntryCount, + untrackedEntryCount, + aheadCount, + behindCount, + isMergedIntoBase, + createdByRuntime, + }, + warnings, + }; +} + +export function readExecutionWorkspaceConfig(metadata: Record | null | undefined): ExecutionWorkspaceConfig | null { + const raw = isRecord(metadata?.config) ? metadata.config : null; + if (!raw) return null; + + const config: ExecutionWorkspaceConfig = { + provisionCommand: readNullableString(raw.provisionCommand), + teardownCommand: readNullableString(raw.teardownCommand), + cleanupCommand: readNullableString(raw.cleanupCommand), + workspaceRuntime: cloneRecord(raw.workspaceRuntime), + desiredState: raw.desiredState === "running" || raw.desiredState === "stopped" ? raw.desiredState : null, + }; + + const hasConfig = Object.values(config).some((value) => { + if (value === null) return false; + if (typeof value === "object") return Object.keys(value).length > 0; + return true; + }); + + return hasConfig ? config : null; +} + +export function mergeExecutionWorkspaceConfig( + metadata: Record | null | undefined, + patch: Partial | null, +): Record | null { + const nextMetadata = isRecord(metadata) ? { ...metadata } : {}; + const current = readExecutionWorkspaceConfig(metadata) ?? { + provisionCommand: null, + teardownCommand: null, + cleanupCommand: null, + workspaceRuntime: null, + desiredState: null, + }; + + if (patch === null) { + delete nextMetadata.config; + return Object.keys(nextMetadata).length > 0 ? nextMetadata : null; + } + + const nextConfig: ExecutionWorkspaceConfig = { + provisionCommand: patch.provisionCommand !== undefined ? readNullableString(patch.provisionCommand) : current.provisionCommand, + teardownCommand: patch.teardownCommand !== undefined ? readNullableString(patch.teardownCommand) : current.teardownCommand, + cleanupCommand: patch.cleanupCommand !== undefined ? readNullableString(patch.cleanupCommand) : current.cleanupCommand, + workspaceRuntime: patch.workspaceRuntime !== undefined ? cloneRecord(patch.workspaceRuntime) : current.workspaceRuntime, + desiredState: + patch.desiredState !== undefined + ? patch.desiredState === "running" || patch.desiredState === "stopped" + ? patch.desiredState + : null + : current.desiredState, + }; + + const hasConfig = Object.values(nextConfig).some((value) => { + if (value === null) return false; + if (typeof value === "object") return Object.keys(value).length > 0; + return true; + }); + + if (hasConfig) { + nextMetadata.config = { + provisionCommand: nextConfig.provisionCommand, + teardownCommand: nextConfig.teardownCommand, + cleanupCommand: nextConfig.cleanupCommand, + workspaceRuntime: nextConfig.workspaceRuntime, + desiredState: nextConfig.desiredState, + }; + } else { + delete nextMetadata.config; + } + + return Object.keys(nextMetadata).length > 0 ? nextMetadata : null; +} + +function toRuntimeService(row: WorkspaceRuntimeServiceRow): WorkspaceRuntimeService { + return { + id: row.id, + companyId: row.companyId, + projectId: row.projectId ?? null, + projectWorkspaceId: row.projectWorkspaceId ?? null, + executionWorkspaceId: row.executionWorkspaceId ?? null, + issueId: row.issueId ?? null, + scopeType: row.scopeType as WorkspaceRuntimeService["scopeType"], + scopeId: row.scopeId ?? null, + serviceName: row.serviceName, + status: row.status as WorkspaceRuntimeService["status"], + lifecycle: row.lifecycle as WorkspaceRuntimeService["lifecycle"], + reuseKey: row.reuseKey ?? null, + command: row.command ?? null, + cwd: row.cwd ?? null, + port: row.port ?? null, + url: row.url ?? null, + provider: row.provider as WorkspaceRuntimeService["provider"], + providerRef: row.providerRef ?? null, + ownerAgentId: row.ownerAgentId ?? null, + startedByRunId: row.startedByRunId ?? null, + lastUsedAt: row.lastUsedAt, + startedAt: row.startedAt, + stoppedAt: row.stoppedAt ?? null, + stopPolicy: (row.stopPolicy as Record | null) ?? null, + healthStatus: row.healthStatus as WorkspaceRuntimeService["healthStatus"], + createdAt: row.createdAt, + updatedAt: row.updatedAt, + }; +} + +function toExecutionWorkspace( + row: ExecutionWorkspaceRow, + runtimeServices: WorkspaceRuntimeService[] = [], +): ExecutionWorkspace { return { id: row.id, companyId: row.companyId, @@ -28,7 +309,9 @@ function toExecutionWorkspace(row: ExecutionWorkspaceRow): ExecutionWorkspace { closedAt: row.closedAt ?? null, cleanupEligibleAt: row.cleanupEligibleAt ?? null, cleanupReason: row.cleanupReason ?? null, + config: readExecutionWorkspaceConfig((row.metadata as Record | null) ?? null), metadata: (row.metadata as Record | null) ?? null, + runtimeServices, createdAt: row.createdAt, updatedAt: row.updatedAt, }; @@ -63,7 +346,7 @@ export function executionWorkspaceService(db: Db) { .from(executionWorkspaces) .where(and(...conditions)) .orderBy(desc(executionWorkspaces.lastUsedAt), desc(executionWorkspaces.createdAt)); - return rows.map(toExecutionWorkspace); + return rows.map((row) => toExecutionWorkspace(row)); }, getById: async (id: string) => { @@ -72,7 +355,268 @@ export function executionWorkspaceService(db: Db) { .from(executionWorkspaces) .where(eq(executionWorkspaces.id, id)) .then((rows) => rows[0] ?? null); - return row ? toExecutionWorkspace(row) : null; + if (!row) return null; + const runtimeServiceRows = await db + .select() + .from(workspaceRuntimeServices) + .where(eq(workspaceRuntimeServices.executionWorkspaceId, row.id)) + .orderBy(desc(workspaceRuntimeServices.updatedAt), desc(workspaceRuntimeServices.createdAt)); + return toExecutionWorkspace(row, runtimeServiceRows.map(toRuntimeService)); + }, + + getCloseReadiness: async (id: string): Promise => { + const workspace = await db + .select() + .from(executionWorkspaces) + .where(eq(executionWorkspaces.id, id)) + .then((rows) => rows[0] ?? null); + if (!workspace) return null; + + const runtimeServiceRows = await db + .select() + .from(workspaceRuntimeServices) + .where(eq(workspaceRuntimeServices.executionWorkspaceId, workspace.id)) + .orderBy(desc(workspaceRuntimeServices.updatedAt), desc(workspaceRuntimeServices.createdAt)); + const runtimeServices = runtimeServiceRows.map(toRuntimeService); + + const linkedIssues = await db + .select({ + id: issues.id, + identifier: issues.identifier, + title: issues.title, + status: issues.status, + }) + .from(issues) + .where(and(eq(issues.companyId, workspace.companyId), eq(issues.executionWorkspaceId, workspace.id))); + + const projectWorkspace = workspace.projectWorkspaceId + ? await db + .select({ + id: projectWorkspaces.id, + cwd: projectWorkspaces.cwd, + cleanupCommand: projectWorkspaces.cleanupCommand, + isPrimary: projectWorkspaces.isPrimary, + }) + .from(projectWorkspaces) + .where( + and( + eq(projectWorkspaces.companyId, workspace.companyId), + eq(projectWorkspaces.id, workspace.projectWorkspaceId), + ), + ) + .then((rows) => rows[0] ?? null) + : null; + + const primaryProjectWorkspace = workspace.projectId + ? await db + .select({ + id: projectWorkspaces.id, + }) + .from(projectWorkspaces) + .where( + and( + eq(projectWorkspaces.companyId, workspace.companyId), + eq(projectWorkspaces.projectId, workspace.projectId), + eq(projectWorkspaces.isPrimary, true), + ), + ) + .then((rows) => rows[0] ?? null) + : null; + + const projectPolicy = workspace.projectId + ? await db + .select({ + executionWorkspacePolicy: projects.executionWorkspacePolicy, + }) + .from(projects) + .where(and(eq(projects.id, workspace.projectId), eq(projects.companyId, workspace.companyId))) + .then((rows) => parseProjectExecutionWorkspacePolicy(rows[0]?.executionWorkspacePolicy)) + : null; + + const executionWorkspace = toExecutionWorkspace(workspace, runtimeServices); + const config = readExecutionWorkspaceConfig((workspace.metadata as Record | null) ?? null); + const { git, warnings: gitWarnings } = await inspectGitCloseReadiness(executionWorkspace); + const warnings = [...gitWarnings]; + const blockingReasons: string[] = []; + const isSharedWorkspace = executionWorkspace.mode === "shared_workspace"; + const workspacePath = readNullableString(executionWorkspace.providerRef) ?? readNullableString(executionWorkspace.cwd); + const resolvedWorkspacePath = workspacePath ? path.resolve(workspacePath) : null; + const resolvedPrimaryWorkspacePath = projectWorkspace?.cwd ? path.resolve(projectWorkspace.cwd) : null; + const isProjectPrimaryWorkspace = + workspace.projectWorkspaceId != null + && workspace.projectWorkspaceId === primaryProjectWorkspace?.id + && resolvedWorkspacePath != null + && resolvedPrimaryWorkspacePath != null + && resolvedWorkspacePath === resolvedPrimaryWorkspacePath; + + const linkedIssueSummaries = linkedIssues.map((issue) => ({ + ...issue, + isTerminal: TERMINAL_ISSUE_STATUSES.has(issue.status), + })); + + const blockingIssues = linkedIssueSummaries.filter((issue) => !issue.isTerminal); + if (blockingIssues.length > 0) { + const linkedIssueMessage = + blockingIssues.length === 1 + ? "This workspace is still linked to an open issue." + : `This workspace is still linked to ${blockingIssues.length} open issues.`; + if (isSharedWorkspace) { + warnings.push(`${linkedIssueMessage} Archiving it will detach this shared workspace session from those issues, but keep the underlying project workspace available.`); + } else { + blockingReasons.push(linkedIssueMessage); + } + } + + if (isSharedWorkspace) { + warnings.push("This shared workspace session points at project workspace infrastructure. Archiving it only removes the session record."); + } + + if (runtimeServices.some((service) => service.status !== "stopped")) { + warnings.push( + runtimeServices.length === 1 + ? "Closing this workspace will stop 1 attached runtime service." + : `Closing this workspace will stop ${runtimeServices.length} attached runtime services.`, + ); + } + + if (git?.hasDirtyTrackedFiles) { + warnings.push( + git.dirtyEntryCount === 1 + ? "The workspace has 1 modified tracked file." + : `The workspace has ${git.dirtyEntryCount} modified tracked files.`, + ); + } + if (git?.hasUntrackedFiles) { + warnings.push( + git.untrackedEntryCount === 1 + ? "The workspace has 1 untracked file." + : `The workspace has ${git.untrackedEntryCount} untracked files.`, + ); + } + if (git?.aheadCount && git.aheadCount > 0 && git.isMergedIntoBase === false) { + warnings.push( + git.aheadCount === 1 + ? `This workspace is 1 commit ahead of ${git.baseRef ?? "the base ref"} and is not merged.` + : `This workspace is ${git.aheadCount} commits ahead of ${git.baseRef ?? "the base ref"} and is not merged.`, + ); + } + if (git?.behindCount && git.behindCount > 0) { + warnings.push( + git.behindCount === 1 + ? `This workspace is 1 commit behind ${git.baseRef ?? "the base ref"}.` + : `This workspace is ${git.behindCount} commits behind ${git.baseRef ?? "the base ref"}.`, + ); + } + + const plannedActions: ExecutionWorkspaceCloseAction[] = [ + { + kind: "archive_record", + label: "Archive workspace record", + description: "Keep the execution workspace history and issue linkage, but remove it from active workspace lists.", + command: null, + }, + ]; + + if (runtimeServices.some((service) => service.status !== "stopped")) { + plannedActions.push({ + kind: "stop_runtime_services", + label: runtimeServices.length === 1 ? "Stop attached runtime service" : "Stop attached runtime services", + description: + runtimeServices.length === 1 + ? `${runtimeServices[0]?.serviceName ?? "A runtime service"} will be stopped before cleanup.` + : `${runtimeServices.length} runtime services will be stopped before cleanup.`, + command: null, + }); + } + + const configuredCleanupCommands = [ + { + kind: "cleanup_command" as const, + label: "Run workspace cleanup command", + description: "Workspace-specific cleanup runs before teardown.", + command: config?.cleanupCommand ?? null, + }, + { + kind: "cleanup_command" as const, + label: "Run project workspace cleanup command", + description: "Project workspace cleanup runs before execution workspace teardown.", + command: projectWorkspace?.cleanupCommand ?? null, + }, + ]; + for (const action of configuredCleanupCommands) { + if (!action.command) continue; + plannedActions.push(action); + } + + const teardownCommand = config?.teardownCommand ?? projectPolicy?.workspaceStrategy?.teardownCommand ?? null; + if (teardownCommand) { + plannedActions.push({ + kind: "teardown_command", + label: "Run teardown command", + description: "Teardown runs after cleanup commands during workspace close.", + command: teardownCommand, + }); + } + + if (executionWorkspace.providerType === "git_worktree" && workspacePath) { + plannedActions.push({ + kind: "git_worktree_remove", + label: "Remove git worktree", + description: `Paperclip will run git worktree cleanup for ${workspacePath}.`, + command: `git worktree remove --force ${workspacePath}`, + }); + } + + if (git?.createdByRuntime && executionWorkspace.branchName) { + plannedActions.push({ + kind: "git_branch_delete", + label: "Delete runtime-created branch", + description: "Paperclip will try to delete the runtime-created branch after removing the worktree.", + command: `git branch -d ${executionWorkspace.branchName}`, + }); + } + + if (executionWorkspace.providerType === "local_fs" && git?.createdByRuntime && workspacePath) { + const resolvedWorkspacePath = path.resolve(workspacePath); + const resolvedProjectWorkspacePath = projectWorkspace?.cwd ? path.resolve(projectWorkspace.cwd) : null; + const containsProjectWorkspace = resolvedProjectWorkspacePath + ? ( + resolvedWorkspacePath === resolvedProjectWorkspacePath || + resolvedProjectWorkspacePath.startsWith(`${resolvedWorkspacePath}${path.sep}`) + ) + : false; + if (containsProjectWorkspace) { + warnings.push(`Paperclip will archive this workspace but keep "${workspacePath}" because it contains the project workspace.`); + } else { + plannedActions.push({ + kind: "remove_local_directory", + label: "Remove runtime-created directory", + description: `Paperclip will remove the runtime-created directory at ${workspacePath}.`, + command: `rm -rf ${workspacePath}`, + }); + } + } + + const state = + blockingReasons.length > 0 + ? "blocked" + : warnings.length > 0 + ? "ready_with_warnings" + : "ready"; + + return { + workspaceId: workspace.id, + state, + blockingReasons, + warnings, + linkedIssues: linkedIssueSummaries, + plannedActions, + isDestructiveCloseAllowed: blockingReasons.length === 0, + isSharedWorkspace, + isProjectPrimaryWorkspace, + git, + runtimeServices, + }; }, create: async (data: typeof executionWorkspaces.$inferInsert) => { diff --git a/server/src/services/heartbeat.ts b/server/src/services/heartbeat.ts index c909b9b77c1..4ab87852274 100644 --- a/server/src/services/heartbeat.ts +++ b/server/src/services/heartbeat.ts @@ -4,7 +4,7 @@ import { execFile as execFileCallback } from "node:child_process"; import { promisify } from "node:util"; import { and, asc, desc, eq, gt, inArray, sql } from "drizzle-orm"; import type { Db } from "@paperclipai/db"; -import type { BillingType } from "@paperclipai/shared"; +import type { BillingType, ExecutionWorkspace, ExecutionWorkspaceConfig } from "@paperclipai/shared"; import { agents, agentRuntimeState, @@ -37,10 +37,12 @@ import { persistAdapterManagedRuntimeServices, realizeExecutionWorkspace, releaseRuntimeServicesForRun, + type ExecutionWorkspaceInput, + type RealizedExecutionWorkspace, sanitizeRuntimeServiceBaseEnv, } from "./workspace-runtime.js"; import { issueService } from "./issues.js"; -import { executionWorkspaceService } from "./execution-workspaces.js"; +import { executionWorkspaceService, mergeExecutionWorkspaceConfig } from "./execution-workspaces.js"; import { workspaceOperationService } from "./workspace-operations.js"; import { buildExecutionWorkspaceAdapterConfig, @@ -76,6 +78,87 @@ const SESSIONED_LOCAL_ADAPTERS = new Set([ "pi_local", ]); +export function applyPersistedExecutionWorkspaceConfig(input: { + config: Record; + workspaceConfig: ExecutionWorkspaceConfig | null; + mode: ReturnType; +}) { + const nextConfig = { ...input.config }; + + if (input.mode !== "agent_default") { + if (input.workspaceConfig?.workspaceRuntime === null) { + delete nextConfig.workspaceRuntime; + } else if (input.workspaceConfig?.workspaceRuntime) { + nextConfig.workspaceRuntime = { ...input.workspaceConfig.workspaceRuntime }; + } + } + + if (input.workspaceConfig && input.mode === "isolated_workspace") { + const nextStrategy = parseObject(nextConfig.workspaceStrategy); + if (input.workspaceConfig.provisionCommand === null) delete nextStrategy.provisionCommand; + else nextStrategy.provisionCommand = input.workspaceConfig.provisionCommand; + if (input.workspaceConfig.teardownCommand === null) delete nextStrategy.teardownCommand; + else nextStrategy.teardownCommand = input.workspaceConfig.teardownCommand; + nextConfig.workspaceStrategy = nextStrategy; + } + + return nextConfig; +} + +export function stripWorkspaceRuntimeFromExecutionRunConfig(config: Record) { + const nextConfig = { ...config }; + delete nextConfig.workspaceRuntime; + return nextConfig; +} + +export function buildRealizedExecutionWorkspaceFromPersisted(input: { + base: ExecutionWorkspaceInput; + workspace: ExecutionWorkspace; +}): RealizedExecutionWorkspace { + const cwd = readNonEmptyString(input.workspace.cwd) ?? readNonEmptyString(input.workspace.providerRef); + if (!cwd) { + throw new Error(`Execution workspace ${input.workspace.id} has no local path to reuse.`); + } + + const strategy = input.workspace.strategyType === "git_worktree" ? "git_worktree" : "project_primary"; + return { + baseCwd: input.base.baseCwd, + source: input.workspace.mode === "shared_workspace" ? "project_primary" : "task_session", + projectId: input.workspace.projectId ?? input.base.projectId, + workspaceId: input.workspace.projectWorkspaceId ?? input.base.workspaceId, + repoUrl: input.workspace.repoUrl ?? input.base.repoUrl, + repoRef: input.workspace.baseRef ?? input.base.repoRef, + strategy, + cwd, + branchName: input.workspace.branchName ?? null, + worktreePath: strategy === "git_worktree" ? (readNonEmptyString(input.workspace.providerRef) ?? cwd) : null, + warnings: [], + created: false, + }; +} + +function buildExecutionWorkspaceConfigSnapshot(config: Record): Partial | null { + const strategy = parseObject(config.workspaceStrategy); + const snapshot: Partial = {}; + + if ("workspaceStrategy" in config) { + snapshot.provisionCommand = typeof strategy.provisionCommand === "string" ? strategy.provisionCommand : null; + snapshot.teardownCommand = typeof strategy.teardownCommand === "string" ? strategy.teardownCommand : null; + } + + if ("workspaceRuntime" in config) { + const workspaceRuntime = parseObject(config.workspaceRuntime); + snapshot.workspaceRuntime = Object.keys(workspaceRuntime).length > 0 ? workspaceRuntime : null; + } + + const hasSnapshot = Object.values(snapshot).some((value) => { + if (value === null) return false; + if (typeof value === "object") return Object.keys(value).length > 0; + return true; + }); + return hasSnapshot ? snapshot : null; +} + function deriveRepoNameFromRepoUrl(repoUrl: string | null): string | null { const trimmed = repoUrl?.trim() ?? ""; if (!trimmed) return null; @@ -2030,7 +2113,7 @@ export function heartbeatService(db: Db) { (explicitResumeSessionDisplayId ? { sessionId: explicitResumeSessionDisplayId } : null) ?? normalizeSessionParams(sessionCodec.deserialize(taskSessionForRun?.sessionParamsJson ?? null)); const config = parseObject(agent.adapterConfig); - const executionWorkspaceMode = resolveExecutionWorkspaceMode({ + const requestedExecutionWorkspaceMode = resolveExecutionWorkspaceMode({ projectPolicy: projectExecutionWorkspacePolicy, issueSettings: issueExecutionWorkspaceSettings, legacyUseProjectWorkspace: issueAssigneeOverrides?.useProjectWorkspace ?? null, @@ -2039,27 +2122,8 @@ export function heartbeatService(db: Db) { agent, context, previousSessionParams, - { useProjectWorkspace: executionWorkspaceMode !== "agent_default" }, + { useProjectWorkspace: requestedExecutionWorkspaceMode !== "agent_default" }, ); - const workspaceManagedConfig = buildExecutionWorkspaceAdapterConfig({ - agentConfig: config, - projectPolicy: projectExecutionWorkspacePolicy, - issueSettings: issueExecutionWorkspaceSettings, - mode: executionWorkspaceMode, - legacyUseProjectWorkspace: issueAssigneeOverrides?.useProjectWorkspace ?? null, - }); - const mergedConfig = issueAssigneeOverrides?.adapterConfig - ? { ...workspaceManagedConfig, ...issueAssigneeOverrides.adapterConfig } - : workspaceManagedConfig; - const { config: resolvedConfig, secretKeys } = await secretsSvc.resolveAdapterConfigForRuntime( - agent.companyId, - mergedConfig, - ); - const runtimeSkillEntries = await companySkills.listRuntimeSkillEntries(agent.companyId); - const runtimeConfig = { - ...resolvedConfig, - paperclipRuntimeSkills: runtimeSkillEntries, - }; const issueRef = issueContext ? { id: issueContext.id, @@ -2073,36 +2137,89 @@ export function heartbeatService(db: Db) { : null; const existingExecutionWorkspace = issueRef?.executionWorkspaceId ? await executionWorkspacesSvc.getById(issueRef.executionWorkspaceId) : null; + const shouldReuseExisting = + issueRef?.executionWorkspacePreference === "reuse_existing" && + existingExecutionWorkspace && + existingExecutionWorkspace.status !== "archived"; + const persistedExecutionWorkspaceMode = shouldReuseExisting && existingExecutionWorkspace + ? issueExecutionWorkspaceModeForPersistedWorkspace(existingExecutionWorkspace.mode) + : null; + const effectiveExecutionWorkspaceMode: ReturnType = + persistedExecutionWorkspaceMode === "isolated_workspace" || + persistedExecutionWorkspaceMode === "operator_branch" || + persistedExecutionWorkspaceMode === "agent_default" + ? persistedExecutionWorkspaceMode + : requestedExecutionWorkspaceMode; + const workspaceManagedConfig = shouldReuseExisting + ? { ...config } + : buildExecutionWorkspaceAdapterConfig({ + agentConfig: config, + projectPolicy: projectExecutionWorkspacePolicy, + issueSettings: issueExecutionWorkspaceSettings, + mode: requestedExecutionWorkspaceMode, + legacyUseProjectWorkspace: issueAssigneeOverrides?.useProjectWorkspace ?? null, + }); + const persistedWorkspaceManagedConfig = applyPersistedExecutionWorkspaceConfig({ + config: workspaceManagedConfig, + workspaceConfig: existingExecutionWorkspace?.config ?? null, + mode: effectiveExecutionWorkspaceMode, + }); + const mergedConfig = issueAssigneeOverrides?.adapterConfig + ? { ...persistedWorkspaceManagedConfig, ...issueAssigneeOverrides.adapterConfig } + : persistedWorkspaceManagedConfig; + const configSnapshot = buildExecutionWorkspaceConfigSnapshot(mergedConfig); + const executionRunConfig = stripWorkspaceRuntimeFromExecutionRunConfig(mergedConfig); + const { config: resolvedConfig, secretKeys } = await secretsSvc.resolveAdapterConfigForRuntime( + agent.companyId, + executionRunConfig, + ); + const runtimeSkillEntries = await companySkills.listRuntimeSkillEntries(agent.companyId); + const runtimeConfig = { + ...resolvedConfig, + paperclipRuntimeSkills: runtimeSkillEntries, + }; const workspaceOperationRecorder = workspaceOperationsSvc.createRecorder({ companyId: agent.companyId, heartbeatRunId: run.id, executionWorkspaceId: existingExecutionWorkspace?.id ?? null, }); - const executionWorkspace = await realizeExecutionWorkspace({ - base: { - baseCwd: resolvedWorkspace.cwd, - source: resolvedWorkspace.source, - projectId: resolvedWorkspace.projectId, - workspaceId: resolvedWorkspace.workspaceId, - repoUrl: resolvedWorkspace.repoUrl, - repoRef: resolvedWorkspace.repoRef, - }, - config: runtimeConfig, - issue: issueRef, - agent: { - id: agent.id, - name: agent.name, - companyId: agent.companyId, - }, - recorder: workspaceOperationRecorder, - }); + const executionWorkspaceBase = { + baseCwd: resolvedWorkspace.cwd, + source: resolvedWorkspace.source, + projectId: resolvedWorkspace.projectId, + workspaceId: resolvedWorkspace.workspaceId, + repoUrl: resolvedWorkspace.repoUrl, + repoRef: resolvedWorkspace.repoRef, + } satisfies ExecutionWorkspaceInput; + const executionWorkspace = shouldReuseExisting && existingExecutionWorkspace + ? buildRealizedExecutionWorkspaceFromPersisted({ + base: executionWorkspaceBase, + workspace: existingExecutionWorkspace, + }) + : await realizeExecutionWorkspace({ + base: executionWorkspaceBase, + config: runtimeConfig, + issue: issueRef, + agent: { + id: agent.id, + name: agent.name, + companyId: agent.companyId, + }, + recorder: workspaceOperationRecorder, + }); const resolvedProjectId = executionWorkspace.projectId ?? issueRef?.projectId ?? executionProjectId ?? null; const resolvedProjectWorkspaceId = issueRef?.projectWorkspaceId ?? resolvedWorkspace.workspaceId ?? null; - const shouldReuseExisting = - issueRef?.executionWorkspacePreference === "reuse_existing" && - existingExecutionWorkspace && - existingExecutionWorkspace.status !== "archived"; let persistedExecutionWorkspace = null; + const nextExecutionWorkspaceMetadataBase = { + ...(existingExecutionWorkspace?.metadata ?? {}), + source: executionWorkspace.source, + createdByRuntime: executionWorkspace.created, + } as Record; + const nextExecutionWorkspaceMetadata = shouldReuseExisting + ? nextExecutionWorkspaceMetadataBase + : configSnapshot + ? mergeExecutionWorkspaceConfig(nextExecutionWorkspaceMetadataBase, configSnapshot) + : nextExecutionWorkspaceMetadataBase; try { persistedExecutionWorkspace = shouldReuseExisting && existingExecutionWorkspace ? await executionWorkspacesSvc.update(existingExecutionWorkspace.id, { @@ -2114,11 +2231,7 @@ export function heartbeatService(db: Db) { providerRef: executionWorkspace.worktreePath, status: "active", lastUsedAt: new Date(), - metadata: { - ...(existingExecutionWorkspace.metadata ?? {}), - source: executionWorkspace.source, - createdByRuntime: executionWorkspace.created, - }, + metadata: nextExecutionWorkspaceMetadata, }) : resolvedProjectId ? await executionWorkspacesSvc.create({ @@ -2127,11 +2240,11 @@ export function heartbeatService(db: Db) { projectWorkspaceId: resolvedProjectWorkspaceId, sourceIssueId: issueRef?.id ?? null, mode: - executionWorkspaceMode === "isolated_workspace" + requestedExecutionWorkspaceMode === "isolated_workspace" ? "isolated_workspace" - : executionWorkspaceMode === "operator_branch" + : requestedExecutionWorkspaceMode === "operator_branch" ? "operator_branch" - : executionWorkspaceMode === "agent_default" + : requestedExecutionWorkspaceMode === "agent_default" ? "adapter_managed" : "shared_workspace", strategyType: executionWorkspace.strategy === "git_worktree" ? "git_worktree" : "project_primary", @@ -2145,10 +2258,7 @@ export function heartbeatService(db: Db) { providerRef: executionWorkspace.worktreePath, lastUsedAt: new Date(), openedAt: new Date(), - metadata: { - source: executionWorkspace.source, - createdByRuntime: executionWorkspace.created, - }, + metadata: nextExecutionWorkspaceMetadata, }) : null; } catch (error) { @@ -2175,7 +2285,8 @@ export function heartbeatService(db: Db) { cwd: resolvedWorkspace.cwd, cleanupCommand: null, }, - teardownCommand: projectExecutionWorkspacePolicy?.workspaceStrategy?.teardownCommand ?? null, + cleanupCommand: configSnapshot?.cleanupCommand ?? null, + teardownCommand: configSnapshot?.teardownCommand ?? projectExecutionWorkspacePolicy?.workspaceStrategy?.teardownCommand ?? null, recorder: workspaceOperationRecorder, }); } catch (cleanupError) { @@ -2208,8 +2319,8 @@ export function heartbeatService(db: Db) { const nextIssueWorkspaceMode = issueExecutionWorkspaceModeForPersistedWorkspace(persistedExecutionWorkspace.mode); const shouldSwitchIssueToExistingWorkspace = issueRef?.executionWorkspacePreference === "reuse_existing" || - executionWorkspaceMode === "isolated_workspace" || - executionWorkspaceMode === "operator_branch"; + requestedExecutionWorkspaceMode === "isolated_workspace" || + requestedExecutionWorkspaceMode === "operator_branch"; const nextIssuePatch: Record = {}; if (issueRef?.executionWorkspaceId !== persistedExecutionWorkspace.id) { nextIssuePatch.executionWorkspaceId = persistedExecutionWorkspace.id; @@ -2262,7 +2373,7 @@ export function heartbeatService(db: Db) { context.paperclipWorkspace = { cwd: executionWorkspace.cwd, source: executionWorkspace.source, - mode: executionWorkspaceMode, + mode: effectiveExecutionWorkspaceMode, strategy: executionWorkspace.strategy, projectId: executionWorkspace.projectId, workspaceId: executionWorkspace.workspaceId, diff --git a/server/src/services/index.ts b/server/src/services/index.ts index fccd6c7ff7d..241355b6e17 100644 --- a/server/src/services/index.ts +++ b/server/src/services/index.ts @@ -28,5 +28,5 @@ export { workProductService } from "./work-products.js"; export { logActivity, type LogActivityInput } from "./activity-log.js"; export { notifyHireApproved, type NotifyHireApprovedInput } from "./hire-hook.js"; export { publishLiveEvent, subscribeCompanyLiveEvents } from "./live-events.js"; -export { reconcilePersistedRuntimeServicesOnStartup } from "./workspace-runtime.js"; +export { reconcilePersistedRuntimeServicesOnStartup, restartDesiredRuntimeServicesOnStartup } from "./workspace-runtime.js"; export { createStorageServiceFromConfig, getStorageService } from "../storage/index.js"; diff --git a/server/src/services/issues.ts b/server/src/services/issues.ts index 086f4658e31..b6bdb0666cd 100644 --- a/server/src/services/issues.ts +++ b/server/src/services/issues.ts @@ -26,6 +26,7 @@ import { conflict, notFound, unprocessable } from "../errors.js"; import { defaultIssueExecutionWorkspaceSettingsForProject, gateProjectExecutionWorkspacePolicy, + issueExecutionWorkspaceModeForPersistedWorkspace, parseProjectExecutionWorkspacePolicy, } from "./execution-workspace-policy.js"; import { instanceSettingsService } from "./instance-settings.js"; @@ -70,6 +71,7 @@ export interface IssueFilters { inboxArchivedByUserId?: string; unreadForUserId?: string; projectId?: string; + executionWorkspaceId?: string; parentId?: string; labelId?: string; originKind?: string; @@ -104,6 +106,11 @@ type IssueUserContextInput = { updatedAt: Date | string; }; type ProjectGoalReader = Pick; +type DbReader = Pick; +type IssueCreateInput = Omit & { + labelIds?: string[]; + inheritExecutionWorkspaceFromIssueId?: string | null; +}; function sameRunLock(checkoutRunId: string | null, actorRunId: string | null) { if (actorRunId) return checkoutRunId === actorRunId; @@ -130,6 +137,28 @@ async function getProjectDefaultGoalId( return row?.goalId ?? null; } +async function getWorkspaceInheritanceIssue( + db: DbReader, + companyId: string, + issueId: string, +) { + const issue = await db + .select({ + id: issues.id, + projectId: issues.projectId, + projectWorkspaceId: issues.projectWorkspaceId, + executionWorkspaceId: issues.executionWorkspaceId, + executionWorkspaceSettings: issues.executionWorkspaceSettings, + }) + .from(issues) + .where(and(eq(issues.id, issueId), eq(issues.companyId, companyId))) + .then((rows) => rows[0] ?? null); + if (!issue) { + throw notFound("Workspace inheritance issue not found"); + } + return issue; +} + function touchedByUserCondition(companyId: string, userId: string) { return sql` ( @@ -486,8 +515,13 @@ export function issueService(db: Db) { } } - async function assertValidProjectWorkspace(companyId: string, projectId: string | null | undefined, projectWorkspaceId: string) { - const workspace = await db + async function assertValidProjectWorkspace( + companyId: string, + projectId: string | null | undefined, + projectWorkspaceId: string, + dbOrTx: DbReader = db, + ) { + const workspace = await dbOrTx .select({ id: projectWorkspaces.id, companyId: projectWorkspaces.companyId, @@ -503,8 +537,13 @@ export function issueService(db: Db) { } } - async function assertValidExecutionWorkspace(companyId: string, projectId: string | null | undefined, executionWorkspaceId: string) { - const workspace = await db + async function assertValidExecutionWorkspace( + companyId: string, + projectId: string | null | undefined, + executionWorkspaceId: string, + dbOrTx: DbReader = db, + ) { + const workspace = await dbOrTx .select({ id: executionWorkspaces.id, companyId: executionWorkspaces.companyId, @@ -647,6 +686,9 @@ export function issueService(db: Db) { conditions.push(unreadForUserCondition(companyId, unreadForUserId)); } if (filters?.projectId) conditions.push(eq(issues.projectId, filters.projectId)); + if (filters?.executionWorkspaceId) { + conditions.push(eq(issues.executionWorkspaceId, filters.executionWorkspaceId)); + } if (filters?.parentId) conditions.push(eq(issues.parentId, filters.parentId)); if (filters?.originKind) conditions.push(eq(issues.originKind, filters.originKind)); if (filters?.originId) conditions.push(eq(issues.originId, filters.originId)); @@ -791,6 +833,20 @@ export function issueService(db: Db) { return row; }, + markUnread: async (companyId: string, issueId: string, userId: string) => { + const deleted = await db + .delete(issueReadStates) + .where( + and( + eq(issueReadStates.companyId, companyId), + eq(issueReadStates.issueId, issueId), + eq(issueReadStates.userId, userId), + ), + ) + .returning(); + return deleted.length > 0; + }, + archiveInbox: async (companyId: string, issueId: string, userId: string, archivedAt: Date = new Date()) => { const now = new Date(); const [row] = await db @@ -851,9 +907,9 @@ export function issueService(db: Db) { create: async ( companyId: string, - data: Omit & { labelIds?: string[] }, + data: IssueCreateInput, ) => { - const { labelIds: inputLabelIds, ...issueData } = data; + const { labelIds: inputLabelIds, inheritExecutionWorkspaceFromIssueId, ...issueData } = data; const isolatedWorkspacesEnabled = (await instanceSettings.getExperimental()).enableIsolatedWorkspaces; if (!isolatedWorkspacesEnabled) { delete issueData.executionWorkspaceId; @@ -869,21 +925,55 @@ export function issueService(db: Db) { if (data.assigneeUserId) { await assertAssignableUser(companyId, data.assigneeUserId); } - if (data.projectWorkspaceId) { - await assertValidProjectWorkspace(companyId, data.projectId, data.projectWorkspaceId); - } - if (data.executionWorkspaceId) { - await assertValidExecutionWorkspace(companyId, data.projectId, data.executionWorkspaceId); - } if (data.status === "in_progress" && !data.assigneeAgentId && !data.assigneeUserId) { throw unprocessable("in_progress issues require an assignee"); } return db.transaction(async (tx) => { const defaultCompanyGoal = await getDefaultCompanyGoal(tx, companyId); const projectGoalId = await getProjectDefaultGoalId(tx, companyId, issueData.projectId); + let projectWorkspaceId = issueData.projectWorkspaceId ?? null; + let executionWorkspaceId = issueData.executionWorkspaceId ?? null; + let executionWorkspacePreference = issueData.executionWorkspacePreference ?? null; let executionWorkspaceSettings = (issueData.executionWorkspaceSettings as Record | null | undefined) ?? null; - if (executionWorkspaceSettings == null && issueData.projectId) { + const workspaceInheritanceIssueId = inheritExecutionWorkspaceFromIssueId ?? issueData.parentId ?? null; + const hasExplicitExecutionWorkspaceOverride = + issueData.executionWorkspaceId !== undefined || + issueData.executionWorkspacePreference !== undefined || + issueData.executionWorkspaceSettings !== undefined; + if (workspaceInheritanceIssueId) { + const workspaceSource = await getWorkspaceInheritanceIssue(tx, companyId, workspaceInheritanceIssueId); + if (projectWorkspaceId == null && workspaceSource.projectWorkspaceId) { + projectWorkspaceId = workspaceSource.projectWorkspaceId; + } + if ( + isolatedWorkspacesEnabled && + !hasExplicitExecutionWorkspaceOverride && + workspaceSource.executionWorkspaceId + ) { + const sourceWorkspace = await tx + .select({ + id: executionWorkspaces.id, + mode: executionWorkspaces.mode, + }) + .from(executionWorkspaces) + .where(eq(executionWorkspaces.id, workspaceSource.executionWorkspaceId)) + .then((rows) => rows[0] ?? null); + if (sourceWorkspace) { + executionWorkspaceId = sourceWorkspace.id; + executionWorkspacePreference = "reuse_existing"; + executionWorkspaceSettings = { + ...((workspaceSource.executionWorkspaceSettings as Record | null | undefined) ?? {}), + mode: issueExecutionWorkspaceModeForPersistedWorkspace(sourceWorkspace.mode), + }; + } + } + } + if ( + executionWorkspaceSettings == null && + executionWorkspaceId == null && + issueData.projectId + ) { const project = await tx .select({ executionWorkspacePolicy: projects.executionWorkspacePolicy }) .from(projects) @@ -897,7 +987,6 @@ export function issueService(db: Db) { ), ) as Record | null; } - let projectWorkspaceId = issueData.projectWorkspaceId ?? null; if (!projectWorkspaceId && issueData.projectId) { const project = await tx .select({ @@ -917,6 +1006,12 @@ export function issueService(db: Db) { .then((rows) => rows[0]?.id ?? null); } } + if (projectWorkspaceId) { + await assertValidProjectWorkspace(companyId, issueData.projectId, projectWorkspaceId, tx); + } + if (executionWorkspaceId) { + await assertValidExecutionWorkspace(companyId, issueData.projectId, executionWorkspaceId, tx); + } const [company] = await tx .update(companies) .set({ issueCounter: sql`${companies.issueCounter} + 1` }) @@ -936,6 +1031,8 @@ export function issueService(db: Db) { defaultGoalId: defaultCompanyGoal?.id ?? null, }), ...(projectWorkspaceId ? { projectWorkspaceId } : {}), + ...(executionWorkspaceId ? { executionWorkspaceId } : {}), + ...(executionWorkspacePreference ? { executionWorkspacePreference } : {}), ...(executionWorkspaceSettings ? { executionWorkspaceSettings } : {}), companyId, issueNumber, diff --git a/server/src/services/local-service-supervisor.ts b/server/src/services/local-service-supervisor.ts new file mode 100644 index 00000000000..68dbbdc89df --- /dev/null +++ b/server/src/services/local-service-supervisor.ts @@ -0,0 +1,302 @@ +import { execFile } from "node:child_process"; +import { createHash } from "node:crypto"; +import fs from "node:fs/promises"; +import path from "node:path"; +import { setTimeout as delay } from "node:timers/promises"; +import { promisify } from "node:util"; +import { resolvePaperclipInstanceRoot } from "../home-paths.js"; + +const execFileAsync = promisify(execFile); + +export interface LocalServiceRegistryRecord { + version: 1; + serviceKey: string; + profileKind: string; + serviceName: string; + command: string; + cwd: string; + envFingerprint: string; + port: number | null; + url: string | null; + pid: number; + processGroupId: number | null; + provider: "local_process"; + runtimeServiceId: string | null; + reuseKey: string | null; + startedAt: string; + lastSeenAt: string; + metadata: Record | null; +} + +export interface LocalServiceIdentityInput { + profileKind: string; + serviceName: string; + cwd: string; + command: string; + envFingerprint: string; + port: number | null; + scope: Record | null; +} + +function stableStringify(value: unknown): string { + if (Array.isArray(value)) { + return `[${value.map((entry) => stableStringify(entry)).join(",")}]`; + } + if (value && typeof value === "object") { + const rec = value as Record; + return `{${Object.keys(rec).sort().map((key) => `${JSON.stringify(key)}:${stableStringify(rec[key])}`).join(",")}}`; + } + return JSON.stringify(value); +} + +function sanitizeServiceKeySegment(value: string, fallback: string): string { + const normalized = value + .trim() + .toLowerCase() + .replace(/[^a-z0-9._-]+/g, "-") + .replace(/-+/g, "-") + .replace(/^-+|-+$/g, ""); + return normalized || fallback; +} + +function getRuntimeServicesDir() { + return path.resolve(resolvePaperclipInstanceRoot(), "runtime-services"); +} + +function getRuntimeServiceRegistryPath(serviceKey: string) { + return path.resolve(getRuntimeServicesDir(), `${serviceKey}.json`); +} + +function normalizeRegistryRecord(raw: unknown): LocalServiceRegistryRecord | null { + if (!raw || typeof raw !== "object") return null; + const rec = raw as Record; + if ( + rec.version !== 1 || + typeof rec.serviceKey !== "string" || + typeof rec.profileKind !== "string" || + typeof rec.serviceName !== "string" || + typeof rec.command !== "string" || + typeof rec.cwd !== "string" || + typeof rec.envFingerprint !== "string" || + typeof rec.pid !== "number" + ) { + return null; + } + + return { + version: 1, + serviceKey: rec.serviceKey, + profileKind: rec.profileKind, + serviceName: rec.serviceName, + command: rec.command, + cwd: rec.cwd, + envFingerprint: rec.envFingerprint, + port: typeof rec.port === "number" ? rec.port : null, + url: typeof rec.url === "string" ? rec.url : null, + pid: rec.pid, + processGroupId: typeof rec.processGroupId === "number" ? rec.processGroupId : null, + provider: "local_process", + runtimeServiceId: typeof rec.runtimeServiceId === "string" ? rec.runtimeServiceId : null, + reuseKey: typeof rec.reuseKey === "string" ? rec.reuseKey : null, + startedAt: typeof rec.startedAt === "string" ? rec.startedAt : new Date().toISOString(), + lastSeenAt: typeof rec.lastSeenAt === "string" ? rec.lastSeenAt : new Date().toISOString(), + metadata: + rec.metadata && typeof rec.metadata === "object" && !Array.isArray(rec.metadata) + ? (rec.metadata as Record) + : null, + }; +} + +async function safeReadRegistryRecord(filePath: string) { + try { + const raw = JSON.parse(await fs.readFile(filePath, "utf8")) as unknown; + return normalizeRegistryRecord(raw); + } catch { + return null; + } +} + +export function createLocalServiceKey(input: LocalServiceIdentityInput) { + const digest = createHash("sha256") + .update( + stableStringify({ + profileKind: input.profileKind, + serviceName: input.serviceName, + cwd: path.resolve(input.cwd), + command: input.command, + envFingerprint: input.envFingerprint, + port: input.port, + scope: input.scope ?? null, + }), + ) + .digest("hex") + .slice(0, 24); + + return `${sanitizeServiceKeySegment(input.profileKind, "service")}-${sanitizeServiceKeySegment(input.serviceName, "service")}-${digest}`; +} + +export async function writeLocalServiceRegistryRecord(record: LocalServiceRegistryRecord) { + await fs.mkdir(getRuntimeServicesDir(), { recursive: true }); + await fs.writeFile( + getRuntimeServiceRegistryPath(record.serviceKey), + `${JSON.stringify(record, null, 2)}\n`, + "utf8", + ); +} + +export async function removeLocalServiceRegistryRecord(serviceKey: string) { + await fs.rm(getRuntimeServiceRegistryPath(serviceKey), { force: true }); +} + +export async function readLocalServiceRegistryRecord(serviceKey: string) { + return await safeReadRegistryRecord(getRuntimeServiceRegistryPath(serviceKey)); +} + +export async function listLocalServiceRegistryRecords(filter?: { + profileKind?: string; + metadata?: Record; +}) { + try { + const entries = await fs.readdir(getRuntimeServicesDir(), { withFileTypes: true }); + const records = await Promise.all( + entries + .filter((entry) => entry.isFile() && entry.name.endsWith(".json")) + .map((entry) => safeReadRegistryRecord(path.resolve(getRuntimeServicesDir(), entry.name))), + ); + + return records + .filter((record): record is LocalServiceRegistryRecord => record !== null) + .filter((record) => { + if (filter?.profileKind && record.profileKind !== filter.profileKind) return false; + if (!filter?.metadata) return true; + return Object.entries(filter.metadata).every(([key, value]) => record.metadata?.[key] === value); + }) + .sort((left, right) => left.serviceKey.localeCompare(right.serviceKey)); + } catch { + return []; + } +} + +export async function findLocalServiceRegistryRecordByRuntimeServiceId(input: { + runtimeServiceId: string; + profileKind?: string; +}) { + const records = await listLocalServiceRegistryRecords( + input.profileKind ? { profileKind: input.profileKind } : undefined, + ); + return records.find((record) => record.runtimeServiceId === input.runtimeServiceId) ?? null; +} + +export function isPidAlive(pid: number) { + if (!Number.isInteger(pid) || pid <= 0) return false; + try { + process.kill(pid, 0); + return true; + } catch { + return false; + } +} + +async function isLikelyMatchingCommand(record: LocalServiceRegistryRecord) { + if (process.platform === "win32") return true; + try { + const { stdout } = await execFileAsync("ps", ["-o", "command=", "-p", String(record.pid)]); + const commandLine = stdout.trim(); + if (!commandLine) return false; + return commandLine.includes(record.command) || commandLine.includes(record.serviceName); + } catch { + return true; + } +} + +export async function findAdoptableLocalService(input: { + serviceKey: string; + command?: string | null; + cwd?: string | null; + envFingerprint?: string | null; + port?: number | null; +}) { + const record = await readLocalServiceRegistryRecord(input.serviceKey); + if (!record) return null; + + if (!isPidAlive(record.pid)) { + await removeLocalServiceRegistryRecord(input.serviceKey); + return null; + } + if (!(await isLikelyMatchingCommand(record))) { + await removeLocalServiceRegistryRecord(input.serviceKey); + return null; + } + if (input.command && record.command !== input.command) return null; + if (input.cwd && path.resolve(record.cwd) !== path.resolve(input.cwd)) return null; + if (input.envFingerprint && record.envFingerprint !== input.envFingerprint) return null; + if (input.port !== undefined && input.port !== null && record.port !== input.port) return null; + return record; +} + +export async function touchLocalServiceRegistryRecord( + serviceKey: string, + patch?: Partial>, +) { + const existing = await readLocalServiceRegistryRecord(serviceKey); + if (!existing) return null; + const next: LocalServiceRegistryRecord = { + ...existing, + ...patch, + version: 1, + serviceKey, + lastSeenAt: patch?.lastSeenAt ?? new Date().toISOString(), + }; + await writeLocalServiceRegistryRecord(next); + return next; +} + +export async function terminateLocalService( + record: Pick, + opts?: { signal?: NodeJS.Signals; forceAfterMs?: number }, +) { + const signal = opts?.signal ?? "SIGTERM"; + const targetProcessGroup = process.platform !== "win32" && record.processGroupId && record.processGroupId > 0; + try { + if (targetProcessGroup) { + process.kill(-record.processGroupId!, signal); + } else { + process.kill(record.pid, signal); + } + } catch { + return; + } + + const deadline = Date.now() + (opts?.forceAfterMs ?? 2_000); + while (Date.now() < deadline) { + if (!isPidAlive(record.pid)) { + return; + } + await delay(100); + } + + if (!isPidAlive(record.pid)) return; + try { + if (targetProcessGroup) { + process.kill(-record.processGroupId!, "SIGKILL"); + } else { + process.kill(record.pid, "SIGKILL"); + } + } catch { + // Ignore cleanup races. + } +} + +export async function readLocalServicePortOwner(port: number) { + if (!Number.isInteger(port) || port <= 0 || process.platform === "win32") return null; + try { + const { stdout } = await execFileAsync("lsof", ["-nPiTCP", `:${port}`, "-sTCP:LISTEN", "-t"]); + const firstPid = stdout + .split("\n") + .map((line) => Number.parseInt(line.trim(), 10)) + .find((value) => Number.isInteger(value) && value > 0); + return firstPid ?? null; + } catch { + return null; + } +} diff --git a/server/src/services/project-workspace-runtime-config.ts b/server/src/services/project-workspace-runtime-config.ts new file mode 100644 index 00000000000..8252fecd31d --- /dev/null +++ b/server/src/services/project-workspace-runtime-config.ts @@ -0,0 +1,59 @@ +import type { ProjectWorkspaceRuntimeConfig } from "@paperclipai/shared"; + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function cloneRecord(value: unknown): Record | null { + return isRecord(value) ? { ...value } : null; +} + +function readDesiredState(value: unknown): ProjectWorkspaceRuntimeConfig["desiredState"] { + return value === "running" || value === "stopped" ? value : null; +} + +export function readProjectWorkspaceRuntimeConfig( + metadata: Record | null | undefined, +): ProjectWorkspaceRuntimeConfig | null { + const raw = isRecord(metadata?.runtimeConfig) ? metadata.runtimeConfig : null; + if (!raw) return null; + + const config: ProjectWorkspaceRuntimeConfig = { + workspaceRuntime: cloneRecord(raw.workspaceRuntime), + desiredState: readDesiredState(raw.desiredState), + }; + + const hasConfig = config.workspaceRuntime !== null || config.desiredState !== null; + return hasConfig ? config : null; +} + +export function mergeProjectWorkspaceRuntimeConfig( + metadata: Record | null | undefined, + patch: Partial | null, +): Record | null { + const nextMetadata = isRecord(metadata) ? { ...metadata } : {}; + const current = readProjectWorkspaceRuntimeConfig(metadata) ?? { + workspaceRuntime: null, + desiredState: null, + }; + + if (patch === null) { + delete nextMetadata.runtimeConfig; + return Object.keys(nextMetadata).length > 0 ? nextMetadata : null; + } + + const nextConfig: ProjectWorkspaceRuntimeConfig = { + workspaceRuntime: + patch.workspaceRuntime !== undefined ? cloneRecord(patch.workspaceRuntime) : current.workspaceRuntime, + desiredState: + patch.desiredState !== undefined ? readDesiredState(patch.desiredState) : current.desiredState, + }; + + if (nextConfig.workspaceRuntime === null && nextConfig.desiredState === null) { + delete nextMetadata.runtimeConfig; + } else { + nextMetadata.runtimeConfig = nextConfig; + } + + return Object.keys(nextMetadata).length > 0 ? nextMetadata : null; +} diff --git a/server/src/services/projects.ts b/server/src/services/projects.ts index 4f7d1eb2afc..7c7c4bb0a09 100644 --- a/server/src/services/projects.ts +++ b/server/src/services/projects.ts @@ -9,11 +9,13 @@ import { type ProjectCodebase, type ProjectExecutionWorkspacePolicy, type ProjectGoalRef, + type ProjectWorkspaceRuntimeConfig, type ProjectWorkspace, type WorkspaceRuntimeService, } from "@paperclipai/shared"; import { listWorkspaceRuntimeServicesForProjectWorkspaces } from "./workspace-runtime.js"; import { parseProjectExecutionWorkspacePolicy } from "./execution-workspace-policy.js"; +import { mergeProjectWorkspaceRuntimeConfig, readProjectWorkspaceRuntimeConfig } from "./project-workspace-runtime-config.js"; import { resolveManagedProjectWorkspaceDir } from "../home-paths.js"; type ProjectRow = typeof projects.$inferSelect; @@ -34,6 +36,7 @@ type CreateWorkspaceInput = { remoteWorkspaceRef?: string | null; sharedWorkspaceKey?: string | null; metadata?: Record | null; + runtimeConfig?: Partial | null; isPrimary?: boolean; }; type UpdateWorkspaceInput = Partial; @@ -149,6 +152,7 @@ function toWorkspace( remoteWorkspaceRef: row.remoteWorkspaceRef ?? null, sharedWorkspaceKey: row.sharedWorkspaceKey ?? null, metadata: (row.metadata as Record | null) ?? null, + runtimeConfig: readProjectWorkspaceRuntimeConfig((row.metadata as Record | null) ?? null), isPrimary: row.isPrimary, runtimeServices, createdAt: row.createdAt, @@ -611,7 +615,13 @@ export function projectService(db: Db) { remoteProvider: readNonEmptyString(data.remoteProvider), remoteWorkspaceRef, sharedWorkspaceKey: readNonEmptyString(data.sharedWorkspaceKey), - metadata: (data.metadata as Record | null | undefined) ?? null, + metadata: + data.runtimeConfig !== undefined + ? mergeProjectWorkspaceRuntimeConfig( + (data.metadata as Record | null | undefined) ?? null, + data.runtimeConfig ?? null, + ) + : (data.metadata as Record | null | undefined) ?? null, isPrimary: shouldBePrimary, }) .returning() @@ -681,7 +691,17 @@ export function projectService(db: Db) { if (data.remoteProvider !== undefined) patch.remoteProvider = readNonEmptyString(data.remoteProvider); if (data.remoteWorkspaceRef !== undefined) patch.remoteWorkspaceRef = nextRemoteWorkspaceRef; if (data.sharedWorkspaceKey !== undefined) patch.sharedWorkspaceKey = readNonEmptyString(data.sharedWorkspaceKey); - if (data.metadata !== undefined) patch.metadata = data.metadata; + if (data.metadata !== undefined || data.runtimeConfig !== undefined) { + patch.metadata = + data.runtimeConfig !== undefined + ? mergeProjectWorkspaceRuntimeConfig( + data.metadata !== undefined + ? (data.metadata as Record | null | undefined) + : ((existing.metadata as Record | null | undefined) ?? null), + data.runtimeConfig ?? null, + ) + : data.metadata; + } const updated = await db.transaction(async (tx) => { if (data.isPrimary === true) { diff --git a/server/src/services/workspace-runtime.ts b/server/src/services/workspace-runtime.ts index 7cb780cea0c..bafadba5f62 100644 --- a/server/src/services/workspace-runtime.ts +++ b/server/src/services/workspace-runtime.ts @@ -6,11 +6,23 @@ import path from "node:path"; import { setTimeout as delay } from "node:timers/promises"; import type { AdapterRuntimeServiceReport } from "@paperclipai/adapter-utils"; import type { Db } from "@paperclipai/db"; -import { workspaceRuntimeServices } from "@paperclipai/db"; +import { executionWorkspaces, projectWorkspaces, workspaceRuntimeServices } from "@paperclipai/db"; import { and, desc, eq, inArray } from "drizzle-orm"; import { asNumber, asString, parseObject, renderTemplate } from "../adapters/utils.js"; import { resolveHomeAwarePath } from "../home-paths.js"; +import { + createLocalServiceKey, + findLocalServiceRegistryRecordByRuntimeServiceId, + findAdoptableLocalService, + readLocalServicePortOwner, + removeLocalServiceRegistryRecord, + terminateLocalService, + touchLocalServiceRegistryRecord, + writeLocalServiceRegistryRecord, +} from "./local-service-supervisor.js"; import type { WorkspaceOperationRecorder } from "./workspace-operations.js"; +import { readExecutionWorkspaceConfig } from "./execution-workspaces.js"; +import { readProjectWorkspaceRuntimeConfig } from "./project-workspace-runtime-config.js"; export interface ExecutionWorkspaceInput { baseCwd: string; @@ -28,7 +40,7 @@ export interface ExecutionWorkspaceIssueRef { } export interface ExecutionWorkspaceAgentRef { - id: string; + id: string | null; name: string; companyId: string; } @@ -77,12 +89,24 @@ interface RuntimeServiceRecord extends RuntimeServiceRef { leaseRunIds: Set; idleTimer: ReturnType | null; envFingerprint: string; + serviceKey: string; + profileKind: string; + processGroupId: number | null; } const runtimeServicesById = new Map(); const runtimeServicesByReuseKey = new Map(); const runtimeServiceLeasesByRun = new Map(); +export async function resetRuntimeServicesForTests() { + for (const record of runtimeServicesById.values()) { + clearIdleTimer(record); + } + runtimeServicesById.clear(); + runtimeServicesByReuseKey.clear(); + runtimeServiceLeasesByRun.clear(); +} + function stableStringify(value: unknown): string { if (Array.isArray(value)) { return `[${value.map((entry) => stableStringify(entry)).join(",")}]`; @@ -102,6 +126,8 @@ export function sanitizeRuntimeServiceBaseEnv(baseEnv: NodeJS.ProcessEnv): NodeJ } } delete env.DATABASE_URL; + delete env.npm_config_tailscale_auth; + delete env.npm_config_authenticated_private; return env; } @@ -168,9 +194,9 @@ function toRuntimeServiceRef(record: RuntimeServiceRecord, overrides?: Partial 0 ? normalized : fallback; } @@ -189,7 +215,7 @@ function renderWorkspaceTemplate(template: string, input: { title: input.issue?.title ?? "", }, agent: { - id: input.agent.id, + id: input.agent.id ?? "", name: input.agent.name, }, project: { @@ -205,9 +231,9 @@ function renderWorkspaceTemplate(template: string, input: { function sanitizeBranchName(value: string): string { return value .trim() - .replace(/[^A-Za-z0-9._/-]+/g, "-") + .replace(/[^A-Za-z0-9_-]+/g, "-") .replace(/-+/g, "-") - .replace(/^[-/.]+|[-/.]+$/g, "") + .replace(/^[-_]+|[-_]+$/g, "") .slice(0, 120) || "paperclip-work"; } @@ -312,7 +338,7 @@ function buildWorkspaceCommandEnv(input: { env.PAPERCLIP_WORKSPACE_CREATED = input.created ? "true" : "false"; env.PAPERCLIP_PROJECT_ID = input.base.projectId ?? ""; env.PAPERCLIP_PROJECT_WORKSPACE_ID = input.base.workspaceId ?? ""; - env.PAPERCLIP_AGENT_ID = input.agent.id; + env.PAPERCLIP_AGENT_ID = input.agent.id ?? ""; env.PAPERCLIP_AGENT_NAME = input.agent.name; env.PAPERCLIP_COMPANY_ID = input.agent.companyId; env.PAPERCLIP_ISSUE_ID = input.issue?.id ?? ""; @@ -702,6 +728,7 @@ export async function cleanupExecutionWorkspaceArtifacts(input: { cwd: string | null; cleanupCommand: string | null; } | null; + cleanupCommand?: string | null; teardownCommand?: string | null; recorder?: WorkspaceOperationRecorder | null; }) { @@ -713,6 +740,7 @@ export async function cleanupExecutionWorkspaceArtifacts(input: { }); const createdByRuntime = input.workspace.metadata?.createdByRuntime === true; const cleanupCommands = [ + input.cleanupCommand ?? null, input.projectWorkspace?.cleanupCommand ?? null, input.teardownCommand ?? null, ] @@ -879,13 +907,95 @@ function buildTemplateData(input: { title: input.issue?.title ?? "", }, agent: { - id: input.agent.id, + id: input.agent.id ?? "", name: input.agent.name, }, port: input.port ?? "", }; } +function renderRuntimeServiceEnv(input: { + envConfig: Record; + templateData: ReturnType; +}) { + const rendered: Record = {}; + for (const [key, value] of Object.entries(input.envConfig)) { + if (typeof value !== "string") continue; + rendered[key] = renderTemplate(value, input.templateData); + } + return rendered; +} + +function resolveRuntimeServiceReuseIdentity(input: { + service: Record; + workspace: RealizedExecutionWorkspace; + agent: ExecutionWorkspaceAgentRef; + issue: ExecutionWorkspaceIssueRef | null; + adapterEnv: Record; + scopeType: RuntimeServiceRef["scopeType"]; + scopeId: string | null; +}): { + serviceName: string; + lifecycle: RuntimeServiceRef["lifecycle"]; + command: string; + serviceCwd: string; + envConfig: Record; + envFingerprint: string; + explicitPort: number; + identityPort: number | null; + reuseKey: string | null; +} { + const serviceName = asString(input.service.name, "service"); + const lifecycle = asString(input.service.lifecycle, "shared") === "ephemeral" ? "ephemeral" : "shared"; + const command = asString(input.service.command, ""); + const serviceCwdTemplate = asString(input.service.cwd, "."); + const portConfig = parseObject(input.service.port); + const envConfig = parseObject(input.service.env); + const explicitPort = asNumber(portConfig.value, asNumber(input.service.port, 0)); + const identityPort = explicitPort > 0 ? explicitPort : null; + const templateData = buildTemplateData({ + workspace: input.workspace, + agent: input.agent, + issue: input.issue, + adapterEnv: input.adapterEnv, + port: identityPort, + }); + const serviceCwd = resolveConfiguredPath(renderTemplate(serviceCwdTemplate, templateData), input.workspace.cwd); + const renderedEnv = renderRuntimeServiceEnv({ + envConfig, + templateData, + }); + const envFingerprint = createHash("sha256").update(stableStringify(renderedEnv)).digest("hex"); + const reuseKey = + lifecycle === "shared" + ? createHash("sha256") + .update( + stableStringify({ + scopeType: input.scopeType, + scopeId: input.scopeId, + serviceName, + command, + cwd: serviceCwd, + port: identityPort, + env: renderedEnv, + }), + ) + .digest("hex") + : null; + + return { + serviceName, + lifecycle, + command, + serviceCwd, + envConfig, + envFingerprint, + explicitPort, + identityPort, + reuseKey, + }; +} + function resolveServiceScopeId(input: { service: Record; workspace: RealizedExecutionWorkspace; @@ -1067,7 +1177,7 @@ export function normalizeAdapterManagedRuntimeServices(input: { url: report.url ?? null, provider: "adapter_managed", providerRef: report.providerRef ?? null, - ownerAgentId: report.ownerAgentId ?? input.agent.id, + ownerAgentId: report.ownerAgentId ?? input.agent.id ?? null, startedByRunId: input.runId, lastUsedAt: nowIso, startedAt: nowIso, @@ -1093,14 +1203,31 @@ async function startLocalRuntimeService(input: { scopeType: "project_workspace" | "execution_workspace" | "run" | "agent"; scopeId: string | null; }): Promise { - const serviceName = asString(input.service.name, "service"); - const lifecycle = asString(input.service.lifecycle, "shared") === "ephemeral" ? "ephemeral" : "shared"; - const command = asString(input.service.command, ""); + const identity = resolveRuntimeServiceReuseIdentity({ + service: input.service, + workspace: input.workspace, + agent: input.agent, + issue: input.issue, + adapterEnv: input.adapterEnv, + scopeType: input.scopeType, + scopeId: input.scopeId, + }); + const serviceName = identity.serviceName; + const lifecycle = identity.lifecycle; + const command = identity.command; if (!command) throw new Error(`Runtime service "${serviceName}" is missing command`); - const serviceCwdTemplate = asString(input.service.cwd, "."); const portConfig = parseObject(input.service.port); - const port = asString(portConfig.type, "") === "auto" ? await allocatePort() : null; - const envConfig = parseObject(input.service.env); + const envConfig = identity.envConfig; + const envFingerprint = identity.envFingerprint; + const serviceIdentityFingerprint = input.reuseKey ?? envFingerprint; + const explicitPort = identity.explicitPort; + const identityPort = identity.identityPort; + const port = + asString(portConfig.type, "") === "auto" + ? await allocatePort() + : explicitPort > 0 + ? explicitPort + : null; const templateData = buildTemplateData({ workspace: input.workspace, agent: input.agent, @@ -1108,20 +1235,95 @@ async function startLocalRuntimeService(input: { adapterEnv: input.adapterEnv, port, }); - const serviceCwd = resolveConfiguredPath(renderTemplate(serviceCwdTemplate, templateData), input.workspace.cwd); + const serviceCwd = + port === identityPort + ? identity.serviceCwd + : resolveConfiguredPath(renderTemplate(asString(input.service.cwd, "."), templateData), input.workspace.cwd); const env: Record = { ...sanitizeRuntimeServiceBaseEnv(process.env), ...input.adapterEnv, } as Record; - for (const [key, value] of Object.entries(envConfig)) { - if (typeof value === "string") { - env[key] = renderTemplate(value, templateData); - } + for (const [key, value] of Object.entries(renderRuntimeServiceEnv({ envConfig, templateData }))) { + env[key] = value; } if (port) { const portEnvKey = asString(portConfig.envKey, "PORT"); env[portEnvKey] = String(port); } + const expose = parseObject(input.service.expose); + const readiness = parseObject(input.service.readiness); + const urlTemplate = + asString(expose.urlTemplate, "") || + asString(readiness.urlTemplate, ""); + const url = urlTemplate ? renderTemplate(urlTemplate, templateData) : null; + const stopPolicy = parseObject(input.service.stopPolicy); + const serviceKey = createLocalServiceKey({ + profileKind: "workspace-runtime", + serviceName, + cwd: serviceCwd, + command, + envFingerprint: serviceIdentityFingerprint, + port: identityPort, + scope: { + scopeType: input.scopeType, + scopeId: input.scopeId, + executionWorkspaceId: input.executionWorkspaceId ?? null, + reuseKey: input.reuseKey, + }, + }); + const adoptedRecord = await findAdoptableLocalService({ + serviceKey, + command, + cwd: serviceCwd, + envFingerprint: serviceIdentityFingerprint, + port: identityPort, + }); + if (adoptedRecord) { + return { + id: adoptedRecord.runtimeServiceId ?? randomUUID(), + companyId: input.agent.companyId, + projectId: input.workspace.projectId, + projectWorkspaceId: input.workspace.workspaceId, + executionWorkspaceId: input.executionWorkspaceId ?? null, + issueId: input.issue?.id ?? null, + serviceName, + status: "running", + lifecycle, + scopeType: input.scopeType, + scopeId: input.scopeId, + reuseKey: input.reuseKey, + command, + cwd: serviceCwd, + port: adoptedRecord.port ?? port, + url: adoptedRecord.url ?? url, + provider: "local_process", + providerRef: String(adoptedRecord.pid), + ownerAgentId: input.agent.id ?? null, + startedByRunId: input.runId, + lastUsedAt: new Date().toISOString(), + startedAt: adoptedRecord.startedAt, + stoppedAt: null, + stopPolicy, + healthStatus: "healthy", + reused: true, + db: input.db, + child: null, + leaseRunIds: new Set([input.runId]), + idleTimer: null, + envFingerprint, + serviceKey, + profileKind: "workspace-runtime", + processGroupId: adoptedRecord.processGroupId ?? null, + }; + } + if (identityPort) { + const ownerPid = await readLocalServicePortOwner(identityPort); + if (ownerPid) { + throw new Error( + `Runtime service "${serviceName}" could not start because port ${identityPort} is already in use by pid ${ownerPid}`, + ); + } + } const shell = process.env.SHELL?.trim() || "/bin/sh"; const child = spawn(shell, ["-lc", command], { cwd: serviceCwd, @@ -1142,13 +1344,6 @@ async function startLocalRuntimeService(input: { if (input.onLog) await input.onLog("stderr", `[service:${serviceName}] ${text}`); }); - const expose = parseObject(input.service.expose); - const readiness = parseObject(input.service.readiness); - const urlTemplate = - asString(expose.urlTemplate, "") || - asString(readiness.urlTemplate, ""); - const url = urlTemplate ? renderTemplate(urlTemplate, templateData) : null; - try { await waitForReadiness({ service: input.service, url }); } catch (err) { @@ -1158,8 +1353,7 @@ async function startLocalRuntimeService(input: { ); } - const envFingerprint = createHash("sha256").update(stableStringify(envConfig)).digest("hex"); - return { + const record: RuntimeServiceRecord = { id: randomUUID(), companyId: input.agent.companyId, projectId: input.workspace.projectId, @@ -1178,12 +1372,12 @@ async function startLocalRuntimeService(input: { url, provider: "local_process", providerRef: child.pid ? String(child.pid) : null, - ownerAgentId: input.agent.id, + ownerAgentId: input.agent.id ?? null, startedByRunId: input.runId, lastUsedAt: new Date().toISOString(), startedAt: new Date().toISOString(), stoppedAt: null, - stopPolicy: parseObject(input.service.stopPolicy), + stopPolicy, healthStatus: "healthy", reused: false, db: input.db, @@ -1191,7 +1385,41 @@ async function startLocalRuntimeService(input: { leaseRunIds: new Set([input.runId]), idleTimer: null, envFingerprint, + serviceKey, + profileKind: "workspace-runtime", + processGroupId: child.pid ?? null, }; + + if (child.pid) { + await writeLocalServiceRegistryRecord({ + version: 1, + serviceKey, + profileKind: "workspace-runtime", + serviceName, + command, + cwd: serviceCwd, + envFingerprint: serviceIdentityFingerprint, + port, + url, + pid: child.pid, + processGroupId: child.pid, + provider: "local_process", + runtimeServiceId: record.id, + reuseKey: input.reuseKey, + startedAt: record.startedAt, + lastSeenAt: record.lastUsedAt, + metadata: { + projectId: record.projectId, + projectWorkspaceId: record.projectWorkspaceId, + executionWorkspaceId: record.executionWorkspaceId, + issueId: record.issueId, + scopeType: record.scopeType, + scopeId: record.scopeId, + }, + }); + } + + return record; } function scheduleIdleStop(record: RuntimeServiceRecord) { @@ -1209,15 +1437,28 @@ async function stopRuntimeService(serviceId: string) { if (!record) return; clearIdleTimer(record); record.status = "stopped"; + record.healthStatus = "unknown"; record.lastUsedAt = new Date().toISOString(); record.stoppedAt = new Date().toISOString(); - if (record.child && record.child.pid) { - terminateChildProcess(record.child); - } runtimeServicesById.delete(serviceId); - if (record.reuseKey) { + if (record.reuseKey && runtimeServicesByReuseKey.get(record.reuseKey) === record.id) { runtimeServicesByReuseKey.delete(record.reuseKey); } + if (record.child && record.child.pid) { + await terminateLocalService({ + pid: record.child.pid, + processGroupId: record.processGroupId ?? record.child.pid, + }); + } else if (record.providerRef) { + const pid = Number.parseInt(record.providerRef, 10); + if (Number.isInteger(pid) && pid > 0) { + await terminateLocalService({ + pid, + processGroupId: record.processGroupId, + }); + } + } + await removeLocalServiceRegistryRecord(record.serviceKey); await persistRuntimeServiceRecord(record.db, record); } @@ -1262,10 +1503,18 @@ function registerRuntimeService(db: Db | undefined, record: RuntimeServiceRecord if (current.reuseKey && runtimeServicesByReuseKey.get(current.reuseKey) === current.id) { runtimeServicesByReuseKey.delete(current.reuseKey); } + void removeLocalServiceRegistryRecord(current.serviceKey); void persistRuntimeServiceRecord(db, current); }); } +function readRuntimeServiceEntries(config: Record) { + const runtime = parseObject(config.workspaceRuntime); + return Array.isArray(runtime.services) + ? runtime.services.filter((entry): entry is Record => typeof entry === "object" && entry !== null) + : []; +} + export async function ensureRuntimeServicesForRun(input: { db?: Db; runId: string; @@ -1277,17 +1526,13 @@ export async function ensureRuntimeServicesForRun(input: { adapterEnv: Record; onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise; }): Promise { - const runtime = parseObject(input.config.workspaceRuntime); - const rawServices = Array.isArray(runtime.services) - ? runtime.services.filter((entry): entry is Record => typeof entry === "object" && entry !== null) - : []; + const rawServices = readRuntimeServiceEntries(input.config); const acquiredServiceIds: string[] = []; const refs: RuntimeServiceRef[] = []; runtimeServiceLeasesByRun.set(input.runId, acquiredServiceIds); try { for (const service of rawServices) { - const lifecycle = asString(service.lifecycle, "shared") === "ephemeral" ? "ephemeral" : "shared"; const { scopeType, scopeId } = resolveServiceScopeId({ service, workspace: input.workspace, @@ -1296,13 +1541,15 @@ export async function ensureRuntimeServicesForRun(input: { runId: input.runId, agent: input.agent, }); - const envConfig = parseObject(service.env); - const envFingerprint = createHash("sha256").update(stableStringify(envConfig)).digest("hex"); - const serviceName = asString(service.name, "service"); - const reuseKey = - lifecycle === "shared" - ? [scopeType, scopeId ?? "", serviceName, envFingerprint].join(":") - : null; + const reuseKey = resolveRuntimeServiceReuseIdentity({ + service, + workspace: input.workspace, + agent: input.agent, + issue: input.issue, + adapterEnv: input.adapterEnv, + scopeType, + scopeId, + }).reuseKey; if (reuseKey) { const existingId = runtimeServicesByReuseKey.get(reuseKey); @@ -1312,6 +1559,10 @@ export async function ensureRuntimeServicesForRun(input: { existing.lastUsedAt = new Date().toISOString(); existing.stoppedAt = null; clearIdleTimer(existing); + void touchLocalServiceRegistryRecord(existing.serviceKey, { + runtimeServiceId: existing.id, + lastSeenAt: existing.lastUsedAt, + }); await persistRuntimeServiceRecord(input.db, existing); acquiredServiceIds.push(existing.id); refs.push(toRuntimeServiceRef(existing, { reused: true })); @@ -1346,6 +1597,80 @@ export async function ensureRuntimeServicesForRun(input: { return refs; } +export async function startRuntimeServicesForWorkspaceControl(input: { + db?: Db; + invocationId?: string; + actor: ExecutionWorkspaceAgentRef; + issue: ExecutionWorkspaceIssueRef | null; + workspace: RealizedExecutionWorkspace; + executionWorkspaceId?: string | null; + config: Record; + adapterEnv: Record; + onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise; +}): Promise { + const rawServices = readRuntimeServiceEntries(input.config); + const refs: RuntimeServiceRef[] = []; + const invocationId = input.invocationId ?? randomUUID(); + + for (const service of rawServices) { + const { scopeType, scopeId } = resolveServiceScopeId({ + service, + workspace: input.workspace, + executionWorkspaceId: input.executionWorkspaceId, + issue: input.issue, + runId: invocationId, + agent: input.actor, + }); + const reuseKey = resolveRuntimeServiceReuseIdentity({ + service, + workspace: input.workspace, + agent: input.actor, + issue: input.issue, + adapterEnv: input.adapterEnv, + scopeType, + scopeId, + }).reuseKey; + + if (reuseKey) { + const existingId = runtimeServicesByReuseKey.get(reuseKey); + const existing = existingId ? runtimeServicesById.get(existingId) : null; + if (existing && existing.status === "running") { + existing.lastUsedAt = new Date().toISOString(); + existing.stoppedAt = null; + clearIdleTimer(existing); + void touchLocalServiceRegistryRecord(existing.serviceKey, { + runtimeServiceId: existing.id, + lastSeenAt: existing.lastUsedAt, + }); + await persistRuntimeServiceRecord(input.db, existing); + refs.push(toRuntimeServiceRef(existing, { reused: true })); + continue; + } + } + + const record = await startLocalRuntimeService({ + db: input.db, + runId: invocationId, + agent: input.actor, + issue: input.issue, + workspace: input.workspace, + executionWorkspaceId: input.executionWorkspaceId, + adapterEnv: input.adapterEnv, + service, + onLog: input.onLog, + reuseKey, + scopeType, + scopeId, + }); + record.startedByRunId = null; + registerRuntimeService(input.db, record); + await persistRuntimeServiceRecord(input.db, record); + refs.push(toRuntimeServiceRef(record)); + } + + return refs; +} + export async function releaseRuntimeServicesForRun(runId: string) { const acquired = runtimeServiceLeasesByRun.get(runId) ?? []; runtimeServiceLeasesByRun.delete(runId); @@ -1396,6 +1721,39 @@ export async function stopRuntimeServicesForExecutionWorkspace(input: { } } +export async function stopRuntimeServicesForProjectWorkspace(input: { + db?: Db; + projectWorkspaceId: string; +}) { + const matchingServiceIds = Array.from(runtimeServicesById.values()) + .filter((record) => record.projectWorkspaceId === input.projectWorkspaceId && record.scopeType === "project_workspace") + .map((record) => record.id); + + for (const serviceId of matchingServiceIds) { + await stopRuntimeService(serviceId); + } + + if (input.db) { + const now = new Date(); + await input.db + .update(workspaceRuntimeServices) + .set({ + status: "stopped", + healthStatus: "unknown", + stoppedAt: now, + lastUsedAt: now, + updatedAt: now, + }) + .where( + and( + eq(workspaceRuntimeServices.projectWorkspaceId, input.projectWorkspaceId), + eq(workspaceRuntimeServices.scopeType, "project_workspace"), + inArray(workspaceRuntimeServices.status, ["starting", "running"]), + ), + ); + } +} + export async function listWorkspaceRuntimeServicesForProjectWorkspaces( db: Db, companyId: string, @@ -1409,6 +1767,7 @@ export async function listWorkspaceRuntimeServicesForProjectWorkspaces( and( eq(workspaceRuntimeServices.companyId, companyId), inArray(workspaceRuntimeServices.projectWorkspaceId, projectWorkspaceIds), + eq(workspaceRuntimeServices.scopeType, "project_workspace"), ), ) .orderBy(desc(workspaceRuntimeServices.updatedAt), desc(workspaceRuntimeServices.createdAt)); @@ -1424,8 +1783,8 @@ export async function listWorkspaceRuntimeServicesForProjectWorkspaces( } export async function reconcilePersistedRuntimeServicesOnStartup(db: Db) { - const staleRows = await db - .select({ id: workspaceRuntimeServices.id }) + const rows = await db + .select() .from(workspaceRuntimeServices) .where( and( @@ -1434,26 +1793,171 @@ export async function reconcilePersistedRuntimeServicesOnStartup(db: Db) { ), ); - if (staleRows.length === 0) return { reconciled: 0 }; + if (rows.length === 0) return { reconciled: 0, adopted: 0, stopped: 0 }; - const now = new Date(); - await db - .update(workspaceRuntimeServices) - .set({ - status: "stopped", - healthStatus: "unknown", - stoppedAt: now, - lastUsedAt: now, - updatedAt: now, - }) - .where( - and( - eq(workspaceRuntimeServices.provider, "local_process"), - inArray(workspaceRuntimeServices.status, ["starting", "running"]), - ), - ); + let adopted = 0; + let stopped = 0; + for (const row of rows) { + const adoptedRecord = await findLocalServiceRegistryRecordByRuntimeServiceId({ + runtimeServiceId: row.id, + profileKind: "workspace-runtime", + }); + if (adoptedRecord) { + const record: RuntimeServiceRecord = { + id: row.id, + companyId: row.companyId, + projectId: row.projectId ?? null, + projectWorkspaceId: row.projectWorkspaceId ?? null, + executionWorkspaceId: row.executionWorkspaceId ?? null, + issueId: row.issueId ?? null, + serviceName: row.serviceName, + status: "running", + lifecycle: row.lifecycle as RuntimeServiceRecord["lifecycle"], + scopeType: row.scopeType as RuntimeServiceRecord["scopeType"], + scopeId: row.scopeId ?? null, + reuseKey: row.reuseKey ?? null, + command: row.command ?? null, + cwd: row.cwd ?? null, + port: adoptedRecord.port ?? row.port ?? null, + url: adoptedRecord.url ?? row.url ?? null, + provider: "local_process", + providerRef: String(adoptedRecord.pid), + ownerAgentId: row.ownerAgentId ?? null, + startedByRunId: row.startedByRunId ?? null, + lastUsedAt: new Date().toISOString(), + startedAt: row.startedAt.toISOString(), + stoppedAt: null, + stopPolicy: (row.stopPolicy as Record | null) ?? null, + healthStatus: "healthy", + reused: true, + db, + child: null, + leaseRunIds: new Set(), + idleTimer: null, + envFingerprint: row.reuseKey ?? "", + serviceKey: adoptedRecord.serviceKey, + profileKind: "workspace-runtime", + processGroupId: adoptedRecord.processGroupId ?? null, + }; + registerRuntimeService(db, record); + await touchLocalServiceRegistryRecord(adoptedRecord.serviceKey, { + runtimeServiceId: row.id, + lastSeenAt: record.lastUsedAt, + }); + await persistRuntimeServiceRecord(db, record); + adopted += 1; + continue; + } - return { reconciled: staleRows.length }; + const now = new Date(); + await db + .update(workspaceRuntimeServices) + .set({ + status: "stopped", + healthStatus: "unknown", + stoppedAt: now, + lastUsedAt: now, + updatedAt: now, + }) + .where(eq(workspaceRuntimeServices.id, row.id)); + const registryRecord = await findLocalServiceRegistryRecordByRuntimeServiceId({ + runtimeServiceId: row.id, + profileKind: "workspace-runtime", + }); + if (registryRecord) { + await removeLocalServiceRegistryRecord(registryRecord.serviceKey); + } + stopped += 1; + } + + return { reconciled: rows.length, adopted, stopped }; +} + +export async function restartDesiredRuntimeServicesOnStartup(db: Db) { + let restarted = 0; + let failed = 0; + + const projectWorkspaceRows = await db + .select() + .from(projectWorkspaces); + + for (const row of projectWorkspaceRows) { + const runtimeConfig = readProjectWorkspaceRuntimeConfig((row.metadata as Record | null) ?? null); + if (runtimeConfig?.desiredState !== "running" || !runtimeConfig.workspaceRuntime || !row.cwd) continue; + + try { + const refs = await startRuntimeServicesForWorkspaceControl({ + db, + actor: { id: null, name: "Paperclip", companyId: row.companyId }, + issue: null, + workspace: { + baseCwd: row.cwd, + source: "project_primary", + projectId: row.projectId, + workspaceId: row.id, + repoUrl: row.repoUrl ?? null, + repoRef: row.repoRef ?? null, + strategy: "project_primary", + cwd: row.cwd, + branchName: row.defaultRef ?? row.repoRef ?? null, + worktreePath: null, + warnings: [], + created: false, + }, + config: { workspaceRuntime: runtimeConfig.workspaceRuntime }, + adapterEnv: {}, + }); + if (refs.length > 0) restarted += refs.filter((ref) => !ref.reused).length; + } catch { + failed += 1; + } + } + + const executionWorkspaceRows = await db + .select() + .from(executionWorkspaces) + .where(inArray(executionWorkspaces.status, ["active", "idle", "in_review", "cleanup_failed"])); + + for (const row of executionWorkspaceRows) { + const config = readExecutionWorkspaceConfig((row.metadata as Record | null) ?? null); + if (config?.desiredState !== "running" || !config.workspaceRuntime || !row.cwd) continue; + + try { + const refs = await startRuntimeServicesForWorkspaceControl({ + db, + actor: { id: null, name: "Paperclip", companyId: row.companyId }, + issue: row.sourceIssueId + ? { + id: row.sourceIssueId, + identifier: null, + title: row.name, + } + : null, + workspace: { + baseCwd: row.cwd, + source: row.mode === "shared_workspace" ? "project_primary" : "task_session", + projectId: row.projectId, + workspaceId: row.projectWorkspaceId ?? null, + repoUrl: row.repoUrl ?? null, + repoRef: row.baseRef ?? null, + strategy: row.strategyType === "git_worktree" ? "git_worktree" : "project_primary", + cwd: row.cwd, + branchName: row.branchName ?? null, + worktreePath: row.strategyType === "git_worktree" ? row.cwd : null, + warnings: [], + created: false, + }, + executionWorkspaceId: row.id, + config: { workspaceRuntime: config.workspaceRuntime }, + adapterEnv: {}, + }); + if (refs.length > 0) restarted += refs.filter((ref) => !ref.reused).length; + } catch { + failed += 1; + } + } + + return { restarted, failed }; } export async function persistAdapterManagedRuntimeServices(input: { diff --git a/skills/paperclip/SKILL.md b/skills/paperclip/SKILL.md index 407f08da8a6..1d319ad3562 100644 --- a/skills/paperclip/SKILL.md +++ b/skills/paperclip/SKILL.md @@ -85,7 +85,7 @@ Headers: X-Paperclip-Run-Id: $PAPERCLIP_RUN_ID Status values: `backlog`, `todo`, `in_progress`, `in_review`, `done`, `blocked`, `cancelled`. Priority values: `critical`, `high`, `medium`, `low`. Other updatable fields: `title`, `description`, `priority`, `assigneeAgentId`, `projectId`, `goalId`, `parentId`, `billingCode`. -**Step 9 — Delegate if needed.** Create subtasks with `POST /api/companies/{companyId}/issues`. Always set `parentId` and `goalId`. Set `billingCode` for cross-team work. +**Step 9 — Delegate if needed.** Create subtasks with `POST /api/companies/{companyId}/issues`. Always set `parentId` and `goalId`. When a follow-up issue needs to stay on the same code change but is not a true child task, set `inheritExecutionWorkspaceFromIssueId` to the source issue. Set `billingCode` for cross-team work. ## Project Setup Workflow (CEO/Manager Common Path) @@ -147,6 +147,7 @@ If you are asked to install a skill for the company or an agent you MUST read: Resolve requesting user id from the triggering comment thread (`authorUserId`) when available; otherwise use the issue's `createdByUserId` if it matches the requester context. - **Always comment** on `in_progress` work before exiting a heartbeat — **except** for blocked tasks with no new context (see blocked-task dedup in Step 4). - **Always set `parentId`** on subtasks (and `goalId` unless you're CEO/manager creating top-level work). +- **Preserve workspace continuity for follow-ups.** Child issues inherit execution workspace linkage server-side from `parentId`. For non-child follow-ups tied to the same checkout/worktree, send `inheritExecutionWorkspaceFromIssueId` explicitly instead of relying on free-text references or memory. - **Never cancel cross-team tasks.** Reassign to your manager with a comment. - **Always update blocked issues explicitly.** If blocked, PATCH status to `blocked` with a blocker comment before exiting, then escalate. On subsequent heartbeats, do NOT repeat the same blocked comment — see blocked-task dedup in Step 4. - **@-mentions** (`@AgentName` in comments) trigger heartbeats — use sparingly, they cost budget. @@ -255,6 +256,7 @@ PATCH /api/agents/{agentId}/instructions-path | ----------------------------------------- | ------------------------------------------------------------------------------------------ | | My identity | `GET /api/agents/me` | | My compact inbox | `GET /api/agents/me/inbox-lite` | +| Report a user's Mine inbox view | `GET /api/agents/me/inbox/mine?userId=:userId` | | My assignments | `GET /api/companies/:companyId/issues?assigneeAgentId=:id&status=todo,in_progress,blocked` | | Checkout task | `POST /api/issues/:issueId/checkout` | | Get task + ancestors | `GET /api/issues/:issueId` | diff --git a/skills/paperclip/references/api-reference.md b/skills/paperclip/references/api-reference.md index 632937250e1..aea4250ce02 100644 --- a/skills/paperclip/references/api-reference.md +++ b/skills/paperclip/references/api-reference.md @@ -226,6 +226,34 @@ PATCH /api/issues/issue-99 { "comment": "JWT signing done. Still need token refresh logic. Will continue next heartbeat." } ``` +### Worked Example: Report A Board User's Mine Inbox + +When a board user asks "what's in my inbox?", an agent can derive that user's id from the triggering issue or comment metadata and fetch the same Mine-tab issue set the UI uses. + +``` +# Board user created the requesting issue. +GET /api/issues/issue-200 +-> { id: "issue-200", createdByUserId: "user-7", ... } + +# Fetch the board user's Mine inbox issues. +GET /api/agents/me/inbox/mine?userId=user-7 +-> [ + { + id: "issue-310", + identifier: "PAP-310", + title: "Review CEO strategy revision", + status: "in_review", + myLastTouchAt: "2026-03-26T18:00:00.000Z", + lastExternalCommentAt: "2026-03-26T19:10:00.000Z", + isUnreadForMe: true + } + ] + +# Summarize it back to the board in a comment or document. +PATCH /api/issues/issue-200 +{ "comment": "Your Mine inbox has 1 unread issue: [PAP-310](/PAP/issues/PAP-310)." } +``` + --- ## Worked Example: Manager Heartbeat @@ -566,6 +594,7 @@ Terminal states: `done`, `cancelled` | Method | Path | Description | | ------ | ---------------------------------- | ------------------------------------ | | GET | `/api/agents/me` | Your agent record + chain of command | +| GET | `/api/agents/me/inbox/mine?userId=:userId` | Mine-tab issue list for a specific board user | | GET | `/api/agents/:agentId` | Agent details + chain of command | | GET | `/api/companies/:companyId/agents` | List all agents in company | | GET | `/api/companies/:companyId/org` | Org chart tree | diff --git a/ui/src/App.tsx b/ui/src/App.tsx index 38b5f4bc88b..f240defcf7d 100644 --- a/ui/src/App.tsx +++ b/ui/src/App.tsx @@ -11,6 +11,7 @@ import { Agents } from "./pages/Agents"; import { AgentDetail } from "./pages/AgentDetail"; import { Projects } from "./pages/Projects"; import { ProjectDetail } from "./pages/ProjectDetail"; +import { ProjectWorkspaceDetail } from "./pages/ProjectWorkspaceDetail"; import { Issues } from "./pages/Issues"; import { IssueDetail } from "./pages/IssueDetail"; import { Routines } from "./pages/Routines"; @@ -144,6 +145,8 @@ function boardRoutes() { } /> } /> } /> + } /> + } /> } /> } /> } /> @@ -337,7 +340,10 @@ export function App() { } /> } /> } /> + } /> + } /> } /> + } /> } /> }> {boardRoutes()} diff --git a/ui/src/api/access.ts b/ui/src/api/access.ts index 46e640f0972..792278d279a 100644 --- a/ui/src/api/access.ts +++ b/ui/src/api/access.ts @@ -92,6 +92,7 @@ type CompanyInviteCreated = { expiresAt: string; allowedJoinTypes: "human" | "agent" | "both"; humanRole?: HumanCompanyRole | null; + companyName?: string | null; onboardingTextPath?: string; onboardingTextUrl?: string; inviteMessage?: string | null; diff --git a/ui/src/api/execution-workspaces.ts b/ui/src/api/execution-workspaces.ts index bf83999c93c..3644af77feb 100644 --- a/ui/src/api/execution-workspaces.ts +++ b/ui/src/api/execution-workspaces.ts @@ -1,4 +1,4 @@ -import type { ExecutionWorkspace } from "@paperclipai/shared"; +import type { ExecutionWorkspace, ExecutionWorkspaceCloseReadiness, WorkspaceOperation } from "@paperclipai/shared"; import { api } from "./client"; export const executionWorkspacesApi = { @@ -22,5 +22,14 @@ export const executionWorkspacesApi = { return api.get(`/companies/${companyId}/execution-workspaces${qs ? `?${qs}` : ""}`); }, get: (id: string) => api.get(`/execution-workspaces/${id}`), + getCloseReadiness: (id: string) => + api.get(`/execution-workspaces/${id}/close-readiness`), + listWorkspaceOperations: (id: string) => + api.get(`/execution-workspaces/${id}/workspace-operations`), + controlRuntimeServices: (id: string, action: "start" | "stop" | "restart") => + api.post<{ workspace: ExecutionWorkspace; operation: WorkspaceOperation }>( + `/execution-workspaces/${id}/runtime-services/${action}`, + {}, + ), update: (id: string, data: Record) => api.patch(`/execution-workspaces/${id}`, data), }; diff --git a/ui/src/api/issues.ts b/ui/src/api/issues.ts index 436c6dfdaa7..8d4834c3beb 100644 --- a/ui/src/api/issues.ts +++ b/ui/src/api/issues.ts @@ -11,6 +11,10 @@ import type { } from "@paperclipai/shared"; import { api } from "./client"; +export type IssueUpdateResponse = Issue & { + comment?: IssueComment | null; +}; + export const issuesApi = { list: ( companyId: string, @@ -24,6 +28,7 @@ export const issuesApi = { inboxArchivedByUserId?: string; unreadForUserId?: string; labelId?: string; + executionWorkspaceId?: string; originKind?: string; originId?: string; includeRoutineExecutions?: boolean; @@ -40,6 +45,7 @@ export const issuesApi = { if (filters?.inboxArchivedByUserId) params.set("inboxArchivedByUserId", filters.inboxArchivedByUserId); if (filters?.unreadForUserId) params.set("unreadForUserId", filters.unreadForUserId); if (filters?.labelId) params.set("labelId", filters.labelId); + if (filters?.executionWorkspaceId) params.set("executionWorkspaceId", filters.executionWorkspaceId); if (filters?.originKind) params.set("originKind", filters.originKind); if (filters?.originId) params.set("originId", filters.originId); if (filters?.includeRoutineExecutions) params.set("includeRoutineExecutions", "true"); @@ -53,13 +59,15 @@ export const issuesApi = { deleteLabel: (id: string) => api.delete(`/labels/${id}`), get: (id: string) => api.get(`/issues/${id}`), markRead: (id: string) => api.post<{ id: string; lastReadAt: Date }>(`/issues/${id}/read`, {}), + markUnread: (id: string) => api.delete<{ id: string; removed: boolean }>(`/issues/${id}/read`), archiveFromInbox: (id: string) => api.post<{ id: string; archivedAt: Date }>(`/issues/${id}/inbox-archive`, {}), unarchiveFromInbox: (id: string) => api.delete<{ id: string; archivedAt: Date } | { ok: true }>(`/issues/${id}/inbox-archive`), create: (companyId: string, data: Record) => api.post(`/companies/${companyId}/issues`, data), - update: (id: string, data: Record) => api.patch(`/issues/${id}`, data), + update: (id: string, data: Record) => + api.patch(`/issues/${id}`, data), remove: (id: string) => api.delete(`/issues/${id}`), checkout: (id: string, agentId: string) => api.post(`/issues/${id}/checkout`, { diff --git a/ui/src/api/projects.ts b/ui/src/api/projects.ts index c7177ac66f5..763718ffb51 100644 --- a/ui/src/api/projects.ts +++ b/ui/src/api/projects.ts @@ -1,4 +1,4 @@ -import type { Project, ProjectWorkspace } from "@paperclipai/shared"; +import type { Project, ProjectWorkspace, WorkspaceOperation } from "@paperclipai/shared"; import { api } from "./client"; function withCompanyScope(path: string, companyId?: string) { @@ -27,6 +27,16 @@ export const projectsApi = { projectPath(projectId, companyId, `/workspaces/${encodeURIComponent(workspaceId)}`), data, ), + controlWorkspaceRuntimeServices: ( + projectId: string, + workspaceId: string, + action: "start" | "stop" | "restart", + companyId?: string, + ) => + api.post<{ workspace: ProjectWorkspace; operation: WorkspaceOperation }>( + projectPath(projectId, companyId, `/workspaces/${encodeURIComponent(workspaceId)}/runtime-services/${action}`), + {}, + ), removeWorkspace: (projectId: string, workspaceId: string, companyId?: string) => api.delete(projectPath(projectId, companyId, `/workspaces/${encodeURIComponent(workspaceId)}`)), remove: (id: string, companyId?: string) => api.delete(projectPath(id, companyId)), diff --git a/ui/src/components/CommentThread.tsx b/ui/src/components/CommentThread.tsx index cdf0ddd24cf..af8b818f4c2 100644 --- a/ui/src/components/CommentThread.tsx +++ b/ui/src/components/CommentThread.tsx @@ -10,11 +10,16 @@ import { MarkdownEditor, type MarkdownEditorRef, type MentionOption } from "./Ma import { StatusBadge } from "./StatusBadge"; import { AgentIcon } from "./AgentIconPicker"; import { formatDateTime } from "../lib/utils"; +import { restoreSubmittedCommentDraft } from "../lib/comment-submit-draft"; import { PluginSlotOutlet } from "@/plugins/slots"; interface CommentWithRunMeta extends IssueComment { runId?: string | null; runAgentId?: string | null; + clientId?: string; + clientStatus?: "pending" | "queued"; + queueState?: "queued"; + queueTargetRunId?: string | null; } interface LinkedRunItem { @@ -32,6 +37,7 @@ interface CommentReassignment { interface CommentThreadProps { comments: CommentWithRunMeta[]; + queuedComments?: CommentWithRunMeta[]; linkedRuns?: LinkedRunItem[]; companyId?: string | null; projectId?: string | null; @@ -48,6 +54,8 @@ interface CommentThreadProps { currentAssigneeValue?: string; suggestedAssigneeValue?: string; mentions?: MentionOption[]; + onInterruptQueued?: (runId: string) => Promise; + interruptingQueuedRunId?: string | null; } const DRAFT_DEBOUNCE_MS = 800; @@ -114,6 +122,122 @@ function CopyMarkdownButton({ text }: { text: string }) { ); } +function CommentCard({ + comment, + agentMap, + companyId, + projectId, + highlightCommentId, + queued = false, +}: { + comment: CommentWithRunMeta; + agentMap?: Map; + companyId?: string | null; + projectId?: string | null; + highlightCommentId?: string | null; + queued?: boolean; +}) { + const isHighlighted = highlightCommentId === comment.id; + const isPending = comment.clientStatus === "pending"; + const isQueued = queued || comment.queueState === "queued" || comment.clientStatus === "queued"; + + return ( +
+
+ {comment.authorAgentId ? ( + + + + ) : ( + + )} + + {isQueued ? ( + + Queued + + ) : null} + {companyId && !isPending ? ( + + ) : null} + {isPending ? ( + {isQueued ? "Queueing..." : "Sending..."} + ) : ( + + {formatDateTime(comment.createdAt)} + + )} + + +
+ {comment.body} + {companyId && !isPending ? ( +
+ +
+ ) : null} + {comment.runId && !isPending ? ( +
+ {comment.runAgentId ? ( + + run {comment.runId.slice(0, 8)} + + ) : ( + + run {comment.runId.slice(0, 8)} + + )} +
+ ) : null} +
+ ); +} + type TimelineItem = | { kind: "comment"; id: string; createdAtMs: number; comment: CommentWithRunMeta } | { kind: "run"; id: string; createdAtMs: number; run: LinkedRunItem }; @@ -168,86 +292,15 @@ const TimelineList = memo(function TimelineList({ } const comment = item.comment; - const isHighlighted = highlightCommentId === comment.id; return ( -
-
- {comment.authorAgentId ? ( - - - - ) : ( - - )} - - {companyId ? ( - - ) : null} - - {formatDateTime(comment.createdAt)} - - - -
- {comment.body} - {companyId ? ( -
- -
- ) : null} - {comment.runId && ( -
- {comment.runAgentId ? ( - - run {comment.runId.slice(0, 8)} - - ) : ( - - run {comment.runId.slice(0, 8)} - - )} -
- )} -
+ comment={comment} + agentMap={agentMap} + companyId={companyId} + projectId={projectId} + highlightCommentId={highlightCommentId} + /> ); })} @@ -256,6 +309,7 @@ const TimelineList = memo(function TimelineList({ export function CommentThread({ comments, + queuedComments = [], linkedRuns = [], companyId, projectId, @@ -270,6 +324,8 @@ export function CommentThread({ currentAssigneeValue = "", suggestedAssigneeValue, mentions: providedMentions, + onInterruptQueued, + interruptingQueuedRunId = null, }: CommentThreadProps) { const [body, setBody] = useState(""); const [reopen, setReopen] = useState(true); @@ -345,7 +401,7 @@ export function CommentThread({ // Scroll to comment when URL hash matches #comment-{id} useEffect(() => { const hash = location.hash; - if (!hash.startsWith("#comment-") || comments.length === 0) return; + if (!hash.startsWith("#comment-") || comments.length + queuedComments.length === 0) return; const commentId = hash.slice("#comment-".length); // Only scroll once per hash if (hasScrolledRef.current) return; @@ -358,21 +414,30 @@ export function CommentThread({ const timer = setTimeout(() => setHighlightCommentId(null), 3000); return () => clearTimeout(timer); } - }, [location.hash, comments]); + }, [location.hash, comments, queuedComments]); async function handleSubmit() { const trimmed = body.trim(); if (!trimmed) return; const hasReassignment = enableReassign && reassignTarget !== currentAssigneeValue; const reassignment = hasReassignment ? parseReassignment(reassignTarget) : null; + const submittedBody = trimmed; setSubmitting(true); + setBody(""); try { - await onAdd(trimmed, reopen ? true : undefined, reassignment ?? undefined); - setBody(""); + await onAdd(submittedBody, reopen ? true : undefined, reassignment ?? undefined); if (draftKey) clearDraft(draftKey); setReopen(true); setReassignTarget(effectiveSuggestedAssigneeValue); + } catch { + setBody((current) => + restoreSubmittedCommentDraft({ + currentBody: current, + submittedBody, + }), + ); + // Parent mutation handlers surface the failure and the draft is restored for retry. } finally { setSubmitting(false); } @@ -401,18 +466,54 @@ export function CommentThread({ return (
-

Comments & Runs ({timeline.length})

+

Comments & Runs ({timeline.length + queuedComments.length})

- + {timeline.length > 0 ? ( + + ) : null} {liveRunSlot} + {queuedComments.length > 0 && ( +
+
+

+ Queued Comments ({queuedComments.length}) +

+ {onInterruptQueued && queuedComments[0]?.queueTargetRunId ? ( + + ) : null} +
+
+ {queuedComments.map((comment) => ( + + ))} +
+
+ )} +
void; + onClosed?: (workspace: ExecutionWorkspace) => void; +}; + +function readinessTone(state: "ready" | "ready_with_warnings" | "blocked") { + if (state === "blocked") { + return "border-destructive/30 bg-destructive/5 text-destructive"; + } + if (state === "ready_with_warnings") { + return "border-amber-500/30 bg-amber-500/10 text-amber-800 dark:text-amber-300"; + } + return "border-emerald-500/30 bg-emerald-500/10 text-emerald-700 dark:text-emerald-300"; +} + +export function ExecutionWorkspaceCloseDialog({ + workspaceId, + workspaceName, + currentStatus, + open, + onOpenChange, + onClosed, +}: ExecutionWorkspaceCloseDialogProps) { + const queryClient = useQueryClient(); + const { pushToast } = useToast(); + const actionLabel = currentStatus === "cleanup_failed" ? "Retry close" : "Close workspace"; + + const readinessQuery = useQuery({ + queryKey: queryKeys.executionWorkspaces.closeReadiness(workspaceId), + queryFn: () => executionWorkspacesApi.getCloseReadiness(workspaceId), + enabled: open, + }); + + const closeWorkspace = useMutation({ + mutationFn: () => executionWorkspacesApi.update(workspaceId, { status: "archived" }), + onSuccess: (workspace) => { + queryClient.setQueryData(queryKeys.executionWorkspaces.detail(workspace.id), workspace); + queryClient.invalidateQueries({ queryKey: queryKeys.executionWorkspaces.closeReadiness(workspace.id) }); + pushToast({ + title: currentStatus === "cleanup_failed" ? "Workspace close retried" : "Workspace closed", + tone: "success", + }); + onOpenChange(false); + onClosed?.(workspace); + }, + onError: (error) => { + pushToast({ + title: "Failed to close workspace", + body: error instanceof Error ? error.message : "Unknown error", + tone: "error", + }); + }, + }); + + const readiness = readinessQuery.data ?? null; + const blockingIssues = readiness?.linkedIssues.filter((issue) => !issue.isTerminal) ?? []; + const otherLinkedIssues = readiness?.linkedIssues.filter((issue) => issue.isTerminal) ?? []; + const confirmDisabled = + currentStatus === "archived" || + closeWorkspace.isPending || + readinessQuery.isLoading || + readiness == null || + readiness.state === "blocked"; + + return ( + { + if (!closeWorkspace.isPending) onOpenChange(nextOpen); + }}> + + + {actionLabel} + + Archive {workspaceName} and clean up any owned workspace + artifacts. Paperclip keeps the workspace record and issue history, but removes it from active workspace views. + + + + {readinessQuery.isLoading ? ( +
+ + Checking whether this workspace is safe to close... +
+ ) : readinessQuery.error ? ( +
+ {readinessQuery.error instanceof Error ? readinessQuery.error.message : "Failed to inspect workspace close readiness."} +
+ ) : readiness ? ( +
+
+
+ {readiness.state === "blocked" + ? "Close is blocked" + : readiness.state === "ready_with_warnings" + ? "Close is allowed with warnings" + : "Close is ready"} +
+
+ {readiness.isSharedWorkspace + ? "This is a shared workspace session. Archiving it removes this session record but keeps the underlying project workspace." + : readiness.git?.workspacePath && readiness.git.repoRoot && readiness.git.workspacePath !== readiness.git.repoRoot + ? "This execution workspace has its own checkout path and can be archived independently." + : readiness.isProjectPrimaryWorkspace + ? "This execution workspace currently points at the project's primary workspace path." + : "This workspace is disposable and can be archived."} +
+
+ + {blockingIssues.length > 0 ? ( +
+

Blocking issues

+
+ {blockingIssues.map((issue) => ( +
+
+ + {issue.identifier ?? issue.id} · {issue.title} + + {issue.status} +
+
+ ))} +
+
+ ) : null} + + {readiness.blockingReasons.length > 0 ? ( +
+

Blocking reasons

+
    + {readiness.blockingReasons.map((reason) => ( +
  • + {reason} +
  • + ))} +
+
+ ) : null} + + {readiness.warnings.length > 0 ? ( +
+

Warnings

+
    + {readiness.warnings.map((warning) => ( +
  • + {warning} +
  • + ))} +
+
+ ) : null} + + {readiness.git ? ( +
+

Git status

+
+
+
+
Branch
+
{readiness.git.branchName ?? "Unknown"}
+
+
+
Base ref
+
{readiness.git.baseRef ?? "Not set"}
+
+
+
Merged into base
+
{readiness.git.isMergedIntoBase == null ? "Unknown" : readiness.git.isMergedIntoBase ? "Yes" : "No"}
+
+
+
Ahead / behind
+
+ {(readiness.git.aheadCount ?? 0).toString()} / {(readiness.git.behindCount ?? 0).toString()} +
+
+
+
Dirty tracked files
+
{readiness.git.dirtyEntryCount}
+
+
+
Untracked files
+
{readiness.git.untrackedEntryCount}
+
+
+
+
+ ) : null} + + {otherLinkedIssues.length > 0 ? ( +
+

Other linked issues

+
+ {otherLinkedIssues.map((issue) => ( +
+
+ + {issue.identifier ?? issue.id} · {issue.title} + + {issue.status} +
+
+ ))} +
+
+ ) : null} + + {readiness.runtimeServices.length > 0 ? ( +
+

Attached runtime services

+
+ {readiness.runtimeServices.map((service) => ( +
+
+ {service.serviceName} + {service.status} · {service.lifecycle} +
+
+ {service.url ?? service.command ?? service.cwd ?? "No additional details"} +
+
+ ))} +
+
+ ) : null} + +
+

Cleanup actions

+
+ {readiness.plannedActions.map((action, index) => ( +
+
{action.label}
+
{action.description}
+ {action.command ? ( +
+                        {action.command}
+                      
+ ) : null} +
+ ))} +
+
+ + {currentStatus === "cleanup_failed" ? ( +
+ Cleanup previously failed on this workspace. Retrying close will rerun the cleanup flow and update the + workspace status if it succeeds. +
+ ) : null} + + {currentStatus === "archived" ? ( +
+ This workspace is already archived. +
+ ) : null} + + {readiness.git?.repoRoot ? ( +
+ Repo root: {readiness.git.repoRoot} + {readiness.git.workspacePath ? ( + <> + {" · "}Workspace path: {readiness.git.workspacePath} + + ) : null} +
+ ) : null} + +
+ Last checked {formatDateTime(new Date())} +
+
+ ) : null} + + + + + +
+
+ ); +} diff --git a/ui/src/components/IssueRow.test.tsx b/ui/src/components/IssueRow.test.tsx new file mode 100644 index 00000000000..3273d96cbfc --- /dev/null +++ b/ui/src/components/IssueRow.test.tsx @@ -0,0 +1,116 @@ +// @vitest-environment jsdom + +import { act } from "react"; +import { createRoot } from "react-dom/client"; +import type { Issue } from "@paperclipai/shared"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { IssueRow } from "./IssueRow"; + +vi.mock("@/lib/router", () => ({ + Link: ({ children, className, ...props }: React.ComponentProps<"a">) => ( + {children} + ), +})); + +// eslint-disable-next-line @typescript-eslint/no-explicit-any +(globalThis as any).IS_REACT_ACT_ENVIRONMENT = true; + +function createIssue(overrides: Partial = {}): Issue { + return { + id: "issue-1", + identifier: "PAP-1", + companyId: "company-1", + projectId: null, + projectWorkspaceId: null, + goalId: null, + parentId: null, + title: "Inbox item", + description: null, + status: "todo", + priority: "medium", + assigneeAgentId: null, + assigneeUserId: null, + createdByAgentId: null, + createdByUserId: null, + issueNumber: 1, + requestDepth: 0, + billingCode: null, + assigneeAdapterOverrides: null, + executionWorkspaceId: null, + executionWorkspacePreference: null, + executionWorkspaceSettings: null, + checkoutRunId: null, + executionRunId: null, + executionAgentNameKey: null, + executionLockedAt: null, + startedAt: null, + completedAt: null, + cancelledAt: null, + hiddenAt: null, + createdAt: new Date("2026-03-11T00:00:00.000Z"), + updatedAt: new Date("2026-03-11T00:00:00.000Z"), + labels: [], + labelIds: [], + myLastTouchAt: null, + lastExternalCommentAt: null, + isUnreadForMe: false, + ...overrides, + }; +} + +describe("IssueRow", () => { + let container: HTMLDivElement; + + beforeEach(() => { + container = document.createElement("div"); + document.body.appendChild(container); + }); + + afterEach(() => { + container.remove(); + }); + + it("suppresses accent hover styling when the row is selected", () => { + const root = createRoot(container); + const issue = createIssue(); + + act(() => { + root.render(); + }); + + const link = container.querySelector("[data-inbox-issue-link]") as HTMLAnchorElement | null; + expect(link).not.toBeNull(); + expect(link?.className).toContain("hover:bg-transparent"); + expect(link?.className).not.toContain("hover:bg-accent/50"); + + act(() => { + root.unmount(); + }); + }); + + it("neutralizes selected status and unread dot accents", () => { + const root = createRoot(container); + + act(() => { + root.render(); + }); + + const markReadButton = container.querySelector('button[aria-label="Mark as read"]'); + const unreadDot = markReadButton?.querySelector("span"); + const statusIcon = container.querySelector('span[class*="border-muted-foreground"]'); + + expect(markReadButton).not.toBeNull(); + expect(markReadButton?.className).toContain("hover:bg-muted/80"); + expect(markReadButton?.className).not.toContain("hover:bg-blue-500/20"); + expect(unreadDot).not.toBeNull(); + expect(unreadDot?.className).toContain("bg-muted-foreground/70"); + expect(unreadDot?.className).not.toContain("bg-blue-600"); + expect(statusIcon).not.toBeNull(); + expect(statusIcon?.className).toContain("!border-muted-foreground"); + expect(statusIcon?.className).toContain("!text-muted-foreground"); + + act(() => { + root.unmount(); + }); + }); +}); diff --git a/ui/src/components/IssueRow.tsx b/ui/src/components/IssueRow.tsx index d3bcfcffa78..8a01e585546 100644 --- a/ui/src/components/IssueRow.tsx +++ b/ui/src/components/IssueRow.tsx @@ -2,6 +2,7 @@ import type { ReactNode } from "react"; import type { Issue } from "@paperclipai/shared"; import { Link } from "@/lib/router"; import { X } from "lucide-react"; +import { createIssueDetailPath } from "../lib/issueDetailBreadcrumb"; import { cn } from "../lib/utils"; import { StatusIcon } from "./StatusIcon"; @@ -10,6 +11,7 @@ type UnreadState = "hidden" | "visible" | "fading"; interface IssueRowProps { issue: Issue; issueLinkState?: unknown; + selected?: boolean; mobileLeading?: ReactNode; desktopMetaLeading?: ReactNode; desktopLeadingSpacer?: boolean; @@ -26,6 +28,7 @@ interface IssueRowProps { export function IssueRow({ issue, issueLinkState, + selected = false, mobileLeading, desktopMetaLeading, desktopLeadingSpacer = false, @@ -42,18 +45,21 @@ export function IssueRow({ const identifier = issue.identifier ?? issue.id.slice(0, 8); const showUnreadSlot = unreadState !== null; const showUnreadDot = unreadState === "visible" || unreadState === "fading"; + const selectedStatusClass = selected ? "!text-muted-foreground !border-muted-foreground" : undefined; return ( - {mobileLeading ?? } + {mobileLeading ?? } @@ -66,7 +72,7 @@ export function IssueRow({ {desktopMetaLeading ?? ( <> - + {identifier} @@ -108,12 +114,16 @@ export function IssueRow({ onMarkRead?.(); } }} - className="inline-flex h-4 w-4 items-center justify-center rounded-full transition-colors hover:bg-blue-500/20" + className={cn( + "inline-flex h-4 w-4 items-center justify-center rounded-full transition-colors", + selected ? "hover:bg-muted/80" : "hover:bg-blue-500/20", + )} aria-label="Mark as read" > diff --git a/ui/src/components/IssueWorkspaceCard.tsx b/ui/src/components/IssueWorkspaceCard.tsx index 56484cab651..73ecf9eb9a0 100644 --- a/ui/src/components/IssueWorkspaceCard.tsx +++ b/ui/src/components/IssueWorkspaceCard.tsx @@ -6,7 +6,7 @@ import { executionWorkspacesApi } from "../api/execution-workspaces"; import { instanceSettingsApi } from "../api/instanceSettings"; import { useCompany } from "../context/CompanyContext"; import { queryKeys } from "../lib/queryKeys"; -import { cn } from "../lib/utils"; +import { cn, projectWorkspaceUrl } from "../lib/utils"; import { Button } from "@/components/ui/button"; import { Check, Copy, GitBranch, FolderOpen, Pencil, X } from "lucide-react"; @@ -114,6 +114,29 @@ function configuredWorkspaceLabel( } } +function projectWorkspaceDetailLink(input: { + projectId: string | null | undefined; + projectWorkspaceId: string | null | undefined; +}) { + if (!input.projectId || !input.projectWorkspaceId) return null; + return projectWorkspaceUrl({ id: input.projectId, urlKey: input.projectId }, input.projectWorkspaceId); +} + +function workspaceDetailLink(input: { + projectId: string | null | undefined; + issueProjectWorkspaceId: string | null | undefined; + workspace: ExecutionWorkspace | null | undefined; +}) { + const linkedProjectWorkspaceId = input.workspace?.projectWorkspaceId ?? input.issueProjectWorkspaceId ?? null; + if (input.workspace?.mode === "shared_workspace") { + return projectWorkspaceDetailLink({ + projectId: input.projectId, + projectWorkspaceId: linkedProjectWorkspaceId, + }); + } + return input.workspace ? `/execution-workspaces/${input.workspace.id}` : null; +} + function statusBadge(status: string) { const colors: Record = { active: "bg-green-500/15 text-green-700 dark:text-green-400", @@ -209,6 +232,17 @@ export function IssueWorkspaceCard({ issue, project, onUpdate }: IssueWorkspaceC deduplicatedReusableWorkspaces.find((w) => w.id === draftExecutionWorkspaceId) ?? (draftExecutionWorkspaceId === issue.executionWorkspaceId ? selectedReusableExecutionWorkspace : null); + const selectedReusableWorkspaceLink = workspaceDetailLink({ + projectId: project?.id, + issueProjectWorkspaceId: issue.projectWorkspaceId, + workspace: selectedReusableExecutionWorkspace, + }); + const currentWorkspaceLink = workspaceDetailLink({ + projectId: project?.id, + issueProjectWorkspaceId: issue.projectWorkspaceId, + workspace, + }); + const canSaveWorkspaceConfig = draftSelection !== "reuse_existing" || draftExecutionWorkspaceId.length > 0; const handleSave = useCallback(() => { @@ -317,18 +351,22 @@ export function IssueWorkspaceCard({ issue, project, onUpdate }: IssueWorkspaceC {currentSelection === "reuse_existing" && selectedReusableExecutionWorkspace && (
Reusing:{" "} - + {selectedReusableWorkspaceLink ? ( + + + + ) : ( - + )}
)} - {workspace && ( + {workspace && currentWorkspaceLink && (
View workspace details → @@ -385,12 +423,16 @@ export function IssueWorkspaceCard({ issue, project, onUpdate }: IssueWorkspaceC
Current:{" "} - + {currentWorkspaceLink ? ( + + + + ) : ( - + )} {" · "} {workspace.status}
diff --git a/ui/src/components/MarkdownEditor.tsx b/ui/src/components/MarkdownEditor.tsx index 68469761baa..e04ceda21d1 100644 --- a/ui/src/components/MarkdownEditor.tsx +++ b/ui/src/components/MarkdownEditor.tsx @@ -564,8 +564,11 @@ export const MarkdownEditor = forwardRef {mentionActive && filteredMentions.length > 0 && createPortal(
{filteredMentions.map((option, i) => ( + , + ); + }); + + const wrapper = container.firstElementChild as HTMLDivElement; + const button = container.querySelector("button"); + expect(button).not.toBeNull(); + + Object.defineProperty(wrapper, "offsetWidth", { configurable: true, value: 200 }); + Object.defineProperty(wrapper, "offsetHeight", { configurable: true, value: 48 }); + + act(() => { + dispatchTouchEvent(wrapper, "touchstart", { x: 180, y: 20 }); + }); + act(() => { + dispatchTouchEvent(wrapper, "touchmove", { x: 80, y: 22 }); + }); + act(() => { + dispatchTouchEvent(wrapper, "touchend", { x: 80, y: 22 }); + }); + + act(() => { + button!.dispatchEvent(new MouseEvent("click", { bubbles: true, cancelable: true })); + }); + + expect(onClick).not.toHaveBeenCalled(); + + act(() => { + vi.advanceTimersByTime(140); + }); + + expect(onArchive).toHaveBeenCalledTimes(1); + + act(() => { + root.unmount(); + }); + }); + + it("does not suppress a normal tap click", () => { + const onArchive = vi.fn(); + const onClick = vi.fn(); + const root = createRoot(container); + + act(() => { + root.render( + + + , + ); + }); + + const button = container.querySelector("button"); + expect(button).not.toBeNull(); + + act(() => { + button!.dispatchEvent(new MouseEvent("click", { bubbles: true, cancelable: true })); + }); + + expect(onClick).toHaveBeenCalledTimes(1); + expect(onArchive).not.toHaveBeenCalled(); + + act(() => { + root.unmount(); + }); + }); + + it("renders the selected inbox treatment on the swipe surface", () => { + const root = createRoot(container); + + act(() => { + root.render( + {}} selected> + + , + ); + }); + + const surface = container.querySelector("[data-inbox-row-surface]") as HTMLDivElement | null; + expect(surface).not.toBeNull(); + expect(surface?.className).toContain("bg-zinc-100"); + expect(surface?.className).toContain("dark:bg-zinc-800"); + expect(surface?.className).not.toContain("bg-card"); + expect(surface?.style.backgroundColor).toBe(""); + expect(surface?.style.boxShadow).toBe(""); + + act(() => { + root.unmount(); + }); + }); +}); diff --git a/ui/src/components/SwipeToArchive.tsx b/ui/src/components/SwipeToArchive.tsx index 17ea3707986..639179c3493 100644 --- a/ui/src/components/SwipeToArchive.tsx +++ b/ui/src/components/SwipeToArchive.tsx @@ -6,23 +6,26 @@ interface SwipeToArchiveProps { children: ReactNode; onArchive: () => void; disabled?: boolean; + selected?: boolean; className?: string; } -const COMMIT_THRESHOLD = 0.4; -const MAX_SWIPE = 0.92; -const COMMIT_DELAY_MS = 210; +const COMMIT_THRESHOLD = 0.32; +const MAX_SWIPE = 0.88; +const COMMIT_DELAY_MS = 140; export function SwipeToArchive({ children, onArchive, disabled = false, + selected = false, className, }: SwipeToArchiveProps) { const containerRef = useRef(null); const startPointRef = useRef<{ x: number; y: number } | null>(null); const widthRef = useRef(0); const timeoutRef = useRef(null); + const suppressClickRef = useRef(false); const [offsetX, setOffsetX] = useState(0); const [isDragging, setIsDragging] = useState(false); const [isCollapsing, setIsCollapsing] = useState(false); @@ -68,6 +71,7 @@ export function SwipeToArchive({ widthRef.current = node?.offsetWidth ?? 0; setLockedHeight(node?.offsetHeight ?? null); setIsCollapsing(false); + suppressClickRef.current = false; startPointRef.current = { x: touch.clientX, y: touch.clientY }; }; @@ -86,6 +90,7 @@ export function SwipeToArchive({ startPointRef.current = null; return; } + suppressClickRef.current = true; } if (deltaX >= 0) { @@ -127,6 +132,12 @@ export function SwipeToArchive({ onTouchMove={handleTouchMove} onTouchEnd={handleTouchEnd} onTouchCancel={handleTouchEnd} + onClickCapture={(event) => { + if (!suppressClickRef.current) return; + event.preventDefault(); + event.stopPropagation(); + suppressClickRef.current = false; + }} >
void; + toggleTestId?: string; }) { return (
@@ -119,6 +121,8 @@ export function ToggleField({
{/* Invites */} -
+
Invites
@@ -551,7 +549,7 @@ export function CompanySettings() {