mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-02 16:49:00 +01:00
Give metadata project tasks private remote workspaces (#18514)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A task can belong to a project without using that project's files. > - The specification gives a task without a selected or configured source its own private folder. > - The remote source planner instead rejects such a task when its project has no configured workspace. > - This stops an ordinary delegated task before native execution starts. > - This pull request selects an empty, task-owned remote source for that narrow default case. > - Explicit source choices, existing workspace authority, and repository requirements retain their current checks. ## Linked Issues or Issue Description Refs #18504 and #18351. The earlier reference-project fix covered additional project mentions. This fix covers the task's own organizational project. **What happened?** A task in a project with no configured workspace fails with `remote_workspace_source_unavailable`. Its project has no workspace policy, its four workspace selection and binding fields are null, and it requests no repository, branch, or local folder. The failure occurs before native runtime selection. A derived project folder exists in the project facade even when no source is configured; that derived path does not grant file access. **Expected behavior** Use the task's private remote workspace. Keep the project on the task for organization. Do not adopt the project folder, agent home, or another repository. Preserve explicit workspace and source requirements. **Steps to reproduce** 1. Create a project without configured workspaces or a workspace policy. 2. Assign a normal task in that project to a native sandbox agent. Leave all workspace selections and bindings unset. 3. Start the task with remote workspace storage enabled. 4. On the prior source, observe the source-unavailable error before allocation. On this source, observe a company-owned remote workspace whose `sourceIssueId` is the task and whose filesystem project and source IDs are null. **Paperclip version or commit** Reproduced against the deployed source planner at `756599e4ea069c9338c320fe8846523d3dfb92a0`. This branch starts at `987e0932bf66e703750cd04698b3c7e8993580a7`; the relevant planner and preparation inputs at that parent match the deployed source. **Deployment mode** Native sandbox execution with remote workspace storage. The regression tests use real PostgreSQL with the current migrations. ## What Changed - Allow an implicit empty source only after a company-scoped query confirms the project has no workspace rows and no explicit workspace policy. - Keep existing bindings, pending selections, explicit source choices, Git requirements, and local folder choices outside this default. - Add real PostgreSQL source controls and an assembled heartbeat preparation control through workspace reservation and the environment acquisition boundary. - Clarify the existing task-owned folder contract in the implementation specification. ## Verification - Causal RED: the new source control and the assembled allocation control both fail against the unchanged deployed planner with the observed source-unavailable error. - GREEN: 19 focused PostgreSQL controls pass across the source planner and assembled heartbeat preparation suites. The allocation control checks the persisted company/task owner, null filesystem project IDs, private remote root, original organizational project, and unchanged project and host sentinel files. - Negative controls retain configured repository sources and reject a configured source without a URL, an explicit missing source, an unknown selection, a required project policy, a required branch, an explicit local folder, a foreign project, and unsupported read-only project sources. - Current-source transitive TypeScript checks and `node scripts/check-maps.mjs` pass. The map check is structural evidence only. - Fresh exact-head Greptile is 5/5 with zero unresolved threads at `680461e180232d0f4a057528617e39b64b4f0e3a`. [Review](https://github.com/paperclipai/paperclip/pull/18514#issuecomment-6552946468). The official [Paperclip assessment](https://github.com/paperclipai/paperclip/pull/18514#issuecomment-6552931726) is 4/5 and remains recorded unchanged. A separate non-author source review found no actionable issue after the full producer/consumer trace, with manual confidence 4/5. Neither static review ran the tests or proved deployed native execution. - Full repository tests, build, and type checks passed in [canonical CI](https://github.com/paperclipai/paperclip/actions/runs/41269733360). All 59 checks are terminal: 55 success, three skipped, and the official Paperclip check neutral; no failed or pending checks. The successful build checked out merge preview `03dce9f9bc708e778dd214ca85dfa1ba58e0a806`, with parents current main `9dbbfa4fb298448db83ace946b338bff321d8b2a` and this exact head. Its four helper blobs match this head; the four added mainline document-harness paths remain unchanged. Local tests use source aliases with the current DB, shared, and Runner source, rather than borrowed generated declarations. - Actual QA staging qualification is pending deployment. The previous failed journey and cleanup evidence remain failed; these local checks do not relabel them. ### Review follow-up - Fresh workspace retry: the normal producer sets `taskDirectory:true` for this empty-source case. [Heartbeat](https://github.com/paperclipai/paperclip/blob/680461e180232d0f4a057528617e39b64b4f0e3a/server/src/services/heartbeat.ts#L2288-L2296) separates the saved filesystem project from the organizational project. [Environment selection](https://github.com/paperclipai/paperclip/blob/680461e180232d0f4a057528617e39b64b4f0e3a/server/src/services/heartbeat/environment-selection.ts#L506) selects the fresh task directory. Preparation passes that flag, and the planner's existing task-directory return runs before the new binding guard. The proposed fresh failure omits that producer flag. The existing fresh-intent and binding checks stay strict. This is source evidence, not an extra runtime test claim. - Empty policy object: the [public project policy schema](https://github.com/paperclipai/paperclip/blob/680461e180232d0f4a057528617e39b64b4f0e3a/packages/shared/src/validators/project.ts#L24-L43) requires `enabled:boolean`. The project PATCH schema makes outer fields optional and does not make nested policy fields optional. A clear uses `null`; `{}` is rejected. A defensive parser's treatment of unknown stored data does not establish a valid `{}` producer. Keep the narrow fallback closed for non-null policies. - Boolean grouping: JavaScript evaluates the existing predicate as `sourceId || (projectId && !implicitTaskDirectory)`. The explicit-source and metadata-default controls cover those branches. The optional parentheses suggestion does not identify a behavior change; this reviewed source remains frozen. - The official 4/5 assessment and its static verification limits remain visible. The independent manual review withdrew its initial fresh-retry concern after the full producer trace. Actual staging qualification still requires deployment and a new real journey result. ## Risks - The new query runs only for a source-less default admission. It reads one authorized project and uses `NOT EXISTS` for that project's source rows. It adds no recurring poll, write, provider call, or local folder access. - A project with an explicit workspace policy remains subject to the prior source requirement. This conservative rule does not reinterpret policy fields. - The allocation control uses the real planner and PostgreSQL reservation. It stops at environment acquisition, so it proves workspace ownership rather than provider filesystem execution. Live staging qualification is still required. - No API, database schema, permission, model, budget, run limit, or QA journey oracle changes are included. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. This session does not expose a more precise backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
f0ff466aaf
commit
1e8e32290c
4 files changed
+135
-4
No files matched your search
@@ -2593,6 +2593,11 @@ publication receipts, binding checks, and lifecycle locks remain authoritative.
|
||||
|
||||
A task's organizational project and filesystem workspace are independent. New tasks that need a filesystem and have no selected/configured workspace get a company/task directory, not the agent home. Plain tasks need neither Git nor a project. Existing bindings and admitted roots remain authoritative; changing project does not move files. Explicit workspace selections apply at a subsequent normal admission after prior execution and persistence obligations settle.
|
||||
|
||||
The remote default also uses a task-owned empty source for an authorized project
|
||||
with no configured workspaces or explicit workspace policy. Its derived managed
|
||||
folder is not an admitted source. Existing bindings, explicit source selections,
|
||||
and required Git or folder settings do not become empty-source fallbacks.
|
||||
|
||||
Chat connection and destination settings may supply project/workspace defaults. Destination fields override connection fields; omitted values inherit and null clears the inherited choice. Resolve and authorize these defaults once for a new conversation, then retain the resulting task intent. Existing conversations do not change when settings change. Trust confinement still takes precedence.
|
||||
|
||||
Agents inspect/select workspaces and request contained repository preparation through ordinary authorized APIs. Repository preparation is idempotent and may require the next admission; it never claims a host-only clone is available in an already-running remote session. Simultaneous tasks may still explicitly share one physical local/SSH folder. Remote sharing requires actual placement and coherent persistence capability.
|
||||
|
||||
@@ -1,9 +1,13 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { mkdtemp, mkdir, readFile, realpath, rm, writeFile } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { agents, authUsers, companies, createDb, executionWorkspaces, heartbeatRuns, issueComments, issues, projects, projectWorkspaces, type Db } from "@paperclipai/db";
|
||||
import { buildProjectMentionHref } from "@paperclipai/shared";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
|
||||
import { projectService } from "../projects.js";
|
||||
import { toExecutionWorkspace } from "../execution-workspaces.js";
|
||||
import { resolveRemoteWorkspaceSource } from "./remote-workspace-source.js";
|
||||
|
||||
@@ -11,13 +15,16 @@ const support = await getEmbeddedPostgresTestSupport();
|
||||
describe.skipIf(!support.supported)("remote referenced project sources", () => {
|
||||
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
|
||||
let db: Db;
|
||||
let home: string;
|
||||
beforeAll(async () => {
|
||||
database = await startEmbeddedPostgresTestDatabase("remote-project-reference-");
|
||||
db = createDb(database.connectionString);
|
||||
home = await realpath(await mkdtemp(path.join(os.tmpdir(), "paperclip-task-anchor-")));
|
||||
vi.stubEnv("PAPERCLIP_HOME", home);
|
||||
vi.stubEnv("PAPERCLIP_MULTI_PROJECT_WORKSPACE_SYNC", "true");
|
||||
vi.stubEnv("PAPERCLIP_MULTI_PROJECT_WORKSPACE_SYNC_REMOTE", "true");
|
||||
}, 60_000);
|
||||
afterAll(async () => { await db?.$client.end(); await database?.cleanup(); vi.unstubAllEnvs(); });
|
||||
afterAll(async () => { await db?.$client.end(); await database?.cleanup(); if (home) await rm(home, { recursive: true, force: true }); vi.unstubAllEnvs(); });
|
||||
|
||||
async function fixture(repository: boolean) {
|
||||
const companyId = randomUUID(), agentId = randomUUID(), issueId = randomUUID(), runId = randomUUID();
|
||||
@@ -38,6 +45,75 @@ describe.skipIf(!support.supported)("remote referenced project sources", () => {
|
||||
return { companyId, agentId, issueId, runId, projectId };
|
||||
}
|
||||
|
||||
async function assignedTask(repository = false) {
|
||||
const f = await fixture(repository);
|
||||
await db.update(issues).set({ projectId: f.projectId, conversationAgentId: null,
|
||||
conversationUserId: null, conversationState: null }).where(eq(issues.id, f.issueId));
|
||||
return f;
|
||||
}
|
||||
|
||||
it("gives an ordinary metadata-project task its private empty source without adopting the facade folder", async () => {
|
||||
const f = await assignedTask();
|
||||
const project = (await projectService(db).getById(f.projectId))!;
|
||||
expect(project.workspaces).toEqual([]);
|
||||
expect(project.codebase).toMatchObject({ workspaceId: null, repoUrl: null, localFolder: null, origin: "managed_checkout" });
|
||||
expect(project.codebase.managedFolder.startsWith(`${home}${path.sep}`)).toBe(true);
|
||||
await mkdir(project.codebase.managedFolder, { recursive: true });
|
||||
const sentinel = path.join(project.codebase.managedFolder, "unselected-project-files.txt");
|
||||
await writeFile(sentinel, "Preserve this unselected managed folder.");
|
||||
const result = await resolveRemoteWorkspaceSource(db, { ...f, responsibleUserId: null,
|
||||
projectId: f.projectId, projectWorkspaceId: null });
|
||||
expect(result).toMatchObject({ authority: "remote", source: { kind: "empty" }, projectId: null, projectWorkspaceId: null });
|
||||
expect(result.authority === "remote" && result.source.repositories).toBeUndefined();
|
||||
expect((await db.select().from(issues).where(eq(issues.id, f.issueId)))[0]!.projectId).toBe(f.projectId);
|
||||
expect(await readFile(sentinel, "utf8")).toBe("Preserve this unselected managed folder.");
|
||||
expect(await db.select().from(executionWorkspaces).where(eq(executionWorkspaces.companyId, f.companyId))).toEqual([]);
|
||||
});
|
||||
|
||||
it("retains a configured anchor repository instead of substituting private scratch", async () => {
|
||||
const f = await assignedTask(true);
|
||||
const result = await resolveRemoteWorkspaceSource(db, { ...f, responsibleUserId: null,
|
||||
projectId: f.projectId, projectWorkspaceId: null });
|
||||
expect(result).toMatchObject({ authority: "remote", projectId: f.projectId,
|
||||
source: { kind: "git", repoUrl: "https://example.test/team/repository.git" } });
|
||||
expect(result.authority === "remote" && result.projectWorkspaceId).toBeTruthy();
|
||||
});
|
||||
|
||||
it("does not replace a configured anchor without a repository URL", async () => {
|
||||
const f = await assignedTask();
|
||||
await db.insert(projectWorkspaces).values({ companyId: f.companyId, projectId: f.projectId, name: "Required source" });
|
||||
await expect(resolveRemoteWorkspaceSource(db, { ...f, responsibleUserId: null,
|
||||
projectId: f.projectId, projectWorkspaceId: null })).rejects.toMatchObject({
|
||||
resultJson: { reason: "remote_workspace_source_unavailable" },
|
||||
});
|
||||
});
|
||||
|
||||
it.each(["missing-source", "unknown-selection", "required-policy", "required-git", "explicit-cwd"])("does not turn %s into metadata-project scratch", async kind => {
|
||||
const f = await assignedTask();
|
||||
const input: Parameters<typeof resolveRemoteWorkspaceSource>[1] = { ...f, responsibleUserId: null,
|
||||
projectId: f.projectId, projectWorkspaceId: null };
|
||||
if (kind === "missing-source" || kind === "unknown-selection") {
|
||||
const selection = { version: 1 as const, source: "explicit" as const, selection: kind === "missing-source"
|
||||
? { kind: "configured_source", projectWorkspaceId: randomUUID() } : { kind: "unknown" } };
|
||||
await db.update(issues).set({ workspaceSelection: selection as never }).where(eq(issues.id, f.issueId));
|
||||
input.selection = selection as never;
|
||||
} else if (kind === "required-policy") {
|
||||
await db.update(projects).set({ executionWorkspacePolicy: { enabled: true, defaultProjectWorkspaceId: randomUUID() } }).where(eq(projects.id, f.projectId));
|
||||
} else if (kind === "required-git") input.branchName = "required-task-branch";
|
||||
else input.configuredCwd = home;
|
||||
await expect(resolveRemoteWorkspaceSource(db, input)).rejects.toThrow();
|
||||
expect(await db.select().from(executionWorkspaces).where(eq(executionWorkspaces.companyId, f.companyId))).toEqual([]);
|
||||
});
|
||||
|
||||
it("does not use a foreign organizational project as private scratch authority", async () => {
|
||||
const f = await assignedTask(), foreignCompanyId = randomUUID(), foreignProjectId = randomUUID();
|
||||
await db.insert(companies).values({ id: foreignCompanyId, name: "Foreign owner", issuePrefix: foreignCompanyId.slice(0, 8) });
|
||||
await db.insert(projects).values({ id: foreignProjectId, companyId: foreignCompanyId, name: "Foreign project" });
|
||||
await db.update(issues).set({ projectId: foreignProjectId }).where(eq(issues.id, f.issueId));
|
||||
await expect(resolveRemoteWorkspaceSource(db, { ...f, responsibleUserId: null,
|
||||
projectId: foreignProjectId, projectWorkspaceId: null })).rejects.toThrow();
|
||||
});
|
||||
|
||||
it.each(["true", "false"])("keeps a metadata-only project mention without a folder when the remote-source flag is %s", async (remoteEnabled) => {
|
||||
const f = await fixture(false);
|
||||
vi.stubEnv("PAPERCLIP_MULTI_PROJECT_WORKSPACE_SYNC_REMOTE", remoteEnabled);
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { asc, and, eq } from "drizzle-orm";
|
||||
import { projectWorkspaces, type Db } from "@paperclipai/db";
|
||||
import { asc, and, eq, isNull, notExists } from "drizzle-orm";
|
||||
import { projects, projectWorkspaces, type Db } from "@paperclipai/db";
|
||||
import type { TaskWorkspaceIntent } from "@paperclipai/shared";
|
||||
import { assertTaskWorkspaceExecutionAccess } from "../task-workspace-access.js";
|
||||
import { managedProjectCheckoutName, projectRepositoryIdentity } from "../managed-project-checkout-name.js";
|
||||
@@ -113,7 +113,20 @@ export async function resolveRemoteWorkspaceSource(db: Db, input: {
|
||||
eq(projectWorkspaces.projectId, projectId), eq(projectWorkspaces.companyId, input.companyId),
|
||||
)).orderBy(asc(projectWorkspaces.createdAt), asc(projectWorkspaces.id)).limit(1) : [];
|
||||
if (!source) {
|
||||
if (sourceId || projectId) throw new WorkspaceValidationFailure(
|
||||
let implicitTaskDirectory = false;
|
||||
if (projectId && !sourceId && !input.selection && !selected.workspace
|
||||
&& selected.task.executionWorkspaceId == null && selected.task.projectWorkspaceId == null
|
||||
&& selected.task.workspaceSelection == null && selected.task.workspacePendingSelection == null
|
||||
&& input.ref == null && input.branchName == null && input.branchMode !== "existing") {
|
||||
const [metadataProject] = await db.select({ id: projects.id }).from(projects).where(and(
|
||||
eq(projects.id, projectId), eq(projects.companyId, input.companyId), isNull(projects.executionWorkspacePolicy),
|
||||
notExists(db.select({ id: projectWorkspaces.id }).from(projectWorkspaces).where(and(
|
||||
eq(projectWorkspaces.projectId, projects.id), eq(projectWorkspaces.companyId, input.companyId),
|
||||
))),
|
||||
)).limit(1);
|
||||
implicitTaskDirectory = Boolean(metadataProject);
|
||||
}
|
||||
if (sourceId || projectId && !implicitTaskDirectory) throw new WorkspaceValidationFailure(
|
||||
"The selected project has no repository source for a remote workspace.",
|
||||
{ reason: "remote_workspace_source_unavailable" });
|
||||
return withReferences({ kind: "empty" }, null, null);
|
||||
|
||||
@@ -42,6 +42,7 @@ import { InMemorySpanExporter, SimpleSpanProcessor, TracerProvider } from "@open
|
||||
import { selectHeartbeatEnvironment } from "./environment-selection.js";
|
||||
import { assertNativeSessionPreparationAuthority } from "../native-runtime/native-execution-lease.js";
|
||||
import { currentNativeControllerIdentity } from "../native-runtime/native-restart-recovery.js";
|
||||
import { projectService } from "../projects.js";
|
||||
|
||||
const execFile = promisify(execFileCallback);
|
||||
const externalDatabaseUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_WORKSPACE_PREPARATION_TEST_DATABASE_URL");
|
||||
@@ -511,6 +512,42 @@ describe.skipIf(!support.supported && !externalDatabaseUrl)("heartbeat workspace
|
||||
expect(await readFile(join(source, "README.md"), "utf8")).toBe("source content\n");
|
||||
});
|
||||
|
||||
it("reserves private remote task files for an organizational project without a configured source", async () => {
|
||||
const { input, issue, source, envOrchestrator } = await fixture();
|
||||
await db.update(issues).set({ projectWorkspaceId: null }).where(eq(issues.id, issue.id));
|
||||
await db.delete(projectWorkspaces).where(eq(projectWorkspaces.projectId, issue.projectId!));
|
||||
input.task.issueRef = await readIssue(issue.id);
|
||||
const project = (await projectService(db).getById(issue.projectId!))!;
|
||||
expect(project.workspaces).toEqual([]);
|
||||
expect(project.codebase).toMatchObject({ workspaceId: null, repoUrl: null, localFolder: null, origin: "managed_checkout" });
|
||||
await mkdir(project.codebase.managedFolder, { recursive: true });
|
||||
const sentinel = join(project.codebase.managedFolder, "unselected-project-files.txt");
|
||||
await writeFile(sentinel, "Unselected project files\n");
|
||||
configureRemotePersistence();
|
||||
input.agent.adapterType = "paperclip_runner";
|
||||
input.policy.requestedExecutionWorkspaceMode = "agent_default";
|
||||
input.environment.selectedEnvironmentForConfig = { ...input.environment.localEnvironment,
|
||||
driver: "sandbox", config: { provider: "daytona" } };
|
||||
const resolveHost = vi.spyOn(input.services, "resolveWorkspaceForRun");
|
||||
const acquire = vi.spyOn(envOrchestrator, "acquireForRun").mockImplementation(async args => {
|
||||
expect(args.persistedExecutionWorkspace).toMatchObject({ companyId: issue.companyId, sourceIssueId: issue.id,
|
||||
projectId: null, projectWorkspaceId: null, providerType: "cloud_sandbox", repoUrl: null,
|
||||
metadata: { remoteWorkspace: { source: { kind: "empty" }, writerRunId: input.run.id } } });
|
||||
throw new Error("private task workspace reached acquisition");
|
||||
});
|
||||
await expect(prepareHeartbeatWorkspace(db, input)).rejects.toThrow("private task workspace reached acquisition");
|
||||
expect(acquire).toHaveBeenCalledOnce();
|
||||
expect(resolveHost).not.toHaveBeenCalled();
|
||||
const [workspace] = await db.select().from(executionWorkspaces).where(eq(executionWorkspaces.sourceIssueId, issue.id));
|
||||
expect(workspace).toMatchObject({ companyId: issue.companyId, sourceIssueId: issue.id,
|
||||
projectId: null, projectWorkspaceId: null, providerType: "cloud_sandbox", repoUrl: null });
|
||||
expect(workspace.cwd).toBe(`/workspaces/${workspace.id}`);
|
||||
expect((await readIssue(issue.id))).toMatchObject({ projectId: issue.projectId, executionWorkspaceId: workspace.id });
|
||||
await expect(access(workspace.cwd!)).rejects.toThrow();
|
||||
expect(await readFile(sentinel, "utf8")).toBe("Unselected project files\n");
|
||||
expect(await readFile(join(source, "README.md"), "utf8")).toBe("source content\n");
|
||||
});
|
||||
|
||||
it("rejects read-only project references before reserving or acquiring a remote workspace", async () => {
|
||||
const { input, source, issue, envOrchestrator } = await fixture();
|
||||
const previous = new Map(["PAPERCLIP_MULTI_PROJECT_WORKSPACE_SYNC", "PAPERCLIP_MULTI_PROJECT_WORKSPACE_SYNC_REMOTE"]
|
||||
|
||||
Reference in new issue
Block a user