mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-22 11:54:35 +01:00
fix(files): only highlight accessible workspace file links (#11090)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task comments can contain references to files in project and execution workspaces. > - Paperclip detected path-shaped inline code and showed it as an actionable file chip. > - The UI did not first confirm that the current board session could open the file. > - Missing, denied, ambiguous, remote, and unsupported files therefore looked actionable and failed after a click. > - This pull request adds an issue-scoped availability check and promotes only confirmed files to chips. > - The benefit is that the task thread shows a file action only when that action can succeed. ## Linked Issues or Issue Description **What happened?** Task comments promoted path-shaped inline code to file chips before Paperclip checked the file. A chip could point to a missing, denied, ambiguous, remote, or non-previewable file. The action then failed after the user selected it. **Expected behavior** Paperclip must show a file chip only after the server confirms that the current board session can open the exact file reference. All other path-shaped text must stay ordinary inline code. **Steps to reproduce** 1. Add a task comment that contains inline code with a missing or inaccessible workspace path. 2. Open the task thread as a board user. 3. Observe that the path looks like an actionable file chip. 4. Select the chip and observe that the file cannot open. **Paperclip version or commit** `19be4cf927` and earlier. **Deployment mode** Local dev and self-hosted server. **Access context** Board user. ## What Changed - Added shared request, response, and validation contracts for batched workspace-file availability checks. - Added an issue-scoped server endpoint that resolves file references with company, issue, workspace, and preview-access checks. - Added bounded batch concurrency and tests for missing, denied, ambiguous, remote, unsupported, and available files. - Added an issue-scoped UI availability registry that deduplicates, batches, caches, and invalidates file checks. - Changed task-comment markdown rendering so only confirmed files get chip styling and file-viewer behavior. - Bound each chip to the exact workspace target that passed the availability check. ## Verification - `pnpm exec vitest run packages/shared/src/workspace-file-resource.test.ts server/src/__tests__/file-resources.test.ts ui/src/components/MarkdownBody.test.tsx ui/src/components/WorkspaceFileMarkdownBody.availability.test.tsx ui/src/lib/remark-workspace-file-refs.test.ts ui/src/lib/workspace-file-availability.test.ts` — 93 passed, 35 skipped. - `pnpm check:token-gates` — clean. - `pnpm -r typecheck` — passed. - `pnpm build` — passed. - `pnpm test:run` — all server and UI groups passed. One unchanged CLI test saw the run-injected static AWS credentials and expected only its local `AWS_PROFILE`. The same test passed, 8 of 8, after removing only `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` from its process environment. ## Risks - File chips now appear after an asynchronous availability check, so path-shaped text can briefly render as inline code. - Availability results use the existing 30-second file-resource cache window. File-resource invalidation forces a new check. - The endpoint limits each request to 100 references and the client chunks larger sets. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with GPT-5. The service did not expose a more specific model ID or context-window size. The agent used high-reasoning mode, repository tools, command execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
815e49bb7c
commit
7ea2068ef8
25 files changed
+1967
-30
No files matched your search
@@ -11,6 +11,7 @@ import { activityLog, agents, companies, createDb, executionWorkspaces, goals, i
|
||||
import { eq } from "drizzle-orm";
|
||||
import { errorHandler } from "../middleware/index.js";
|
||||
import {
|
||||
createFileResourceAvailabilityLimiter,
|
||||
createFileResourceLimiter,
|
||||
createFileResourceListLimiter,
|
||||
fileResourceRoutes,
|
||||
@@ -1160,6 +1161,9 @@ describeEmbeddedPostgres("workspace file resources", () => {
|
||||
});
|
||||
const resolveLimitedService: WorkspaceFileResourceService = {
|
||||
getIssue: vi.fn(async () => ({ companyId: graph.companyId })),
|
||||
availability: vi.fn(async () => {
|
||||
throw new Error("not used");
|
||||
}),
|
||||
list: vi.fn(async () => {
|
||||
throw new Error("not used");
|
||||
}),
|
||||
@@ -1221,6 +1225,9 @@ describeEmbeddedPostgres("workspace file resources", () => {
|
||||
});
|
||||
const contentLimitedService: WorkspaceFileResourceService = {
|
||||
getIssue: vi.fn(async () => ({ companyId: graph.companyId })),
|
||||
availability: vi.fn(async () => {
|
||||
throw new Error("not used");
|
||||
}),
|
||||
list: vi.fn(async () => {
|
||||
throw new Error("not used");
|
||||
}),
|
||||
@@ -1294,6 +1301,9 @@ describeEmbeddedPostgres("workspace file resources", () => {
|
||||
});
|
||||
const service: WorkspaceFileResourceService = {
|
||||
getIssue: vi.fn(async () => ({ companyId: graph.companyId })),
|
||||
availability: vi.fn(async () => {
|
||||
throw new Error("not used");
|
||||
}),
|
||||
list: vi.fn(async () => {
|
||||
throw new Error("not used");
|
||||
}),
|
||||
@@ -1380,6 +1390,9 @@ describeEmbeddedPostgres("workspace file resources", () => {
|
||||
});
|
||||
const service: WorkspaceFileResourceService = {
|
||||
getIssue: vi.fn(async () => ({ companyId: graph.companyId })),
|
||||
availability: vi.fn(async () => {
|
||||
throw new Error("not used");
|
||||
}),
|
||||
list: vi.fn(async () => {
|
||||
slowListStarted?.();
|
||||
await slowList;
|
||||
@@ -1438,6 +1451,189 @@ describeEmbeddedPostgres("workspace file resources", () => {
|
||||
const third = await request(app).get(`/api/issues/${graph.issueId}/file-resources/list`);
|
||||
expect(third.status).toBe(429);
|
||||
});
|
||||
|
||||
it("returns mixed deduplicated availability results with one aggregate audit event", async () => {
|
||||
const { root, projectRoot, targetProjectRoot, executionRoot } = await makeWorkspace();
|
||||
const graph = await seedGraph(db, {
|
||||
projectRoot,
|
||||
targetProjectRoot,
|
||||
executionRoot,
|
||||
targetProjectSourceType: "remote_managed",
|
||||
});
|
||||
await fs.mkdir(path.join(projectRoot, "docs"), { recursive: true });
|
||||
await fs.writeFile(path.join(projectRoot, "README.md"), "# Visible\n", "utf8");
|
||||
await fs.writeFile(path.join(projectRoot, "docs", "guide.md"), "# Guide\n", "utf8");
|
||||
await fs.writeFile(path.join(projectRoot, "archive.bin"), Buffer.from([0, 1, 2, 3]));
|
||||
await fs.writeFile(path.join(projectRoot, "large.txt"), Buffer.alloc(WORKSPACE_FILE_TEXT_MAX_BYTES + 1, "a"));
|
||||
await fs.writeFile(path.join(projectRoot, ".env"), "TOKEN=secret\n", "utf8");
|
||||
await fs.writeFile(path.join(root, "outside-secret.txt"), "outside\n", "utf8");
|
||||
await fs.symlink(path.join(root, "outside-secret.txt"), path.join(projectRoot, "escape.txt"));
|
||||
|
||||
const app = createApp(db, {
|
||||
type: "board",
|
||||
userId: "board-user",
|
||||
companyIds: [graph.companyId],
|
||||
source: "session",
|
||||
isInstanceAdmin: false,
|
||||
});
|
||||
const response = await request(app)
|
||||
.post(`/api/issues/${graph.issueId}/file-resources/availability`)
|
||||
.send({
|
||||
queries: [
|
||||
{ workspace: "project", path: "README.md" },
|
||||
{ workspace: "project", path: "./README.md" },
|
||||
{ workspace: "project", path: "docs/" },
|
||||
{ workspace: "project", path: "archive.bin" },
|
||||
{ workspace: "project", path: "large.txt" },
|
||||
{ workspace: "project", path: ".env" },
|
||||
{ workspace: "project", path: "../outside-secret.txt" },
|
||||
{ workspace: "project", path: path.join(root, "host-secret.txt") },
|
||||
{ workspace: "project", path: "escape.txt" },
|
||||
{ workspace: "project", path: "missing.ts" },
|
||||
{
|
||||
workspace: "project",
|
||||
projectId: graph.targetProjectId,
|
||||
workspaceId: graph.targetProjectWorkspaceId,
|
||||
path: "remote.txt",
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
expect(response.body.kind).toBe("workspace_file_availability");
|
||||
expect(response.body.results).toHaveLength(10);
|
||||
const byPath = new Map(response.body.results.map((result: { query: { path: string } }) => [result.query.path, result]));
|
||||
expect(byPath.get("README.md")).toMatchObject({ openable: true, resource: { kind: "file" } });
|
||||
expect(byPath.get("docs/")).toMatchObject({ openable: true, resource: { kind: "directory" } });
|
||||
expect(byPath.get("archive.bin")).toMatchObject({ openable: false, unavailableReason: "unsupported_content" });
|
||||
expect(byPath.get("large.txt")).toMatchObject({ openable: false, unavailableReason: "too_large" });
|
||||
expect(byPath.get(".env")).toMatchObject({ openable: false, unavailableReason: "denied_secret", resource: null });
|
||||
expect(byPath.get("../outside-secret.txt")).toMatchObject({
|
||||
openable: false,
|
||||
unavailableReason: "outside_workspace",
|
||||
resource: null,
|
||||
});
|
||||
expect(byPath.get("host-secret.txt")).toMatchObject({
|
||||
openable: false,
|
||||
unavailableReason: "invalid_path",
|
||||
resource: null,
|
||||
});
|
||||
expect(byPath.get("escape.txt")).toMatchObject({
|
||||
openable: false,
|
||||
unavailableReason: "outside_workspace",
|
||||
resource: null,
|
||||
});
|
||||
expect(byPath.get("missing.ts")).toMatchObject({ openable: false, unavailableReason: "not_found", resource: null });
|
||||
expect(byPath.get("remote.txt")).toMatchObject({
|
||||
openable: false,
|
||||
unavailableReason: "remote_workspace",
|
||||
resource: { kind: "remote_resource" },
|
||||
});
|
||||
expect(JSON.stringify(response.body)).not.toContain(root);
|
||||
|
||||
const rows = await db.select().from(activityLog).where(eq(activityLog.entityId, graph.issueId));
|
||||
const availabilityRows = rows.filter((row) => row.action === "issue.file_resource_availability");
|
||||
expect(availabilityRows).toHaveLength(1);
|
||||
expect(availabilityRows[0]?.details).toMatchObject({
|
||||
outcome: "success",
|
||||
requestedCount: 11,
|
||||
uniqueCount: 10,
|
||||
openableCount: 2,
|
||||
unavailableCount: 8,
|
||||
});
|
||||
expect(JSON.stringify(availabilityRows[0]?.details)).not.toContain(root);
|
||||
});
|
||||
|
||||
it("reports ambiguous auto-discovery as one unavailable result", async () => {
|
||||
const { projectRoot, targetProjectRoot, executionRoot, root } = await makeWorkspace();
|
||||
const graph = await seedGraph(db, { projectRoot, targetProjectRoot, executionRoot });
|
||||
const extraRoot = path.join(root, "extra-project");
|
||||
await fs.mkdir(extraRoot, { recursive: true });
|
||||
await fs.writeFile(path.join(targetProjectRoot, "shared.md"), "target\n", "utf8");
|
||||
await fs.writeFile(path.join(extraRoot, "shared.md"), "extra\n", "utf8");
|
||||
const extraProjectId = crypto.randomUUID();
|
||||
await db.insert(projects).values({
|
||||
id: extraProjectId,
|
||||
companyId: graph.companyId,
|
||||
name: "Extra project",
|
||||
status: "in_progress",
|
||||
});
|
||||
await db.insert(projectWorkspaces).values({
|
||||
id: crypto.randomUUID(),
|
||||
companyId: graph.companyId,
|
||||
projectId: extraProjectId,
|
||||
name: "Extra workspace",
|
||||
sourceType: "local_path",
|
||||
cwd: extraRoot,
|
||||
isPrimary: true,
|
||||
});
|
||||
|
||||
const app = createApp(db, {
|
||||
type: "board",
|
||||
userId: "board-user",
|
||||
companyIds: [graph.companyId],
|
||||
source: "session",
|
||||
isInstanceAdmin: false,
|
||||
});
|
||||
const response = await request(app)
|
||||
.post(`/api/issues/${graph.issueId}/file-resources/availability`)
|
||||
.send({ queries: [{ path: "shared.md" }] });
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
expect(response.body.results).toEqual([
|
||||
{
|
||||
query: { path: "shared.md", workspace: "auto", projectId: null, workspaceId: null },
|
||||
openable: false,
|
||||
unavailableReason: "ambiguous_workspace_path",
|
||||
resource: null,
|
||||
},
|
||||
]);
|
||||
expect(JSON.stringify(response.body)).not.toContain(root);
|
||||
});
|
||||
|
||||
it("enforces board access, company boundaries, and the 100-query cap", async () => {
|
||||
const { projectRoot, executionRoot } = await makeWorkspace();
|
||||
const graph = await seedGraph(db, { projectRoot, executionRoot });
|
||||
const agentId = crypto.randomUUID();
|
||||
await db.insert(agents).values({
|
||||
id: agentId,
|
||||
companyId: graph.companyId,
|
||||
name: "Availability audit agent",
|
||||
role: "engineer",
|
||||
adapterType: "process",
|
||||
adapterConfig: {},
|
||||
});
|
||||
const agentApp = createApp(db, {
|
||||
type: "agent",
|
||||
agentId,
|
||||
companyId: graph.companyId,
|
||||
source: "agent_key",
|
||||
});
|
||||
const otherCompanyApp = createApp(db, {
|
||||
type: "board",
|
||||
userId: "other-board",
|
||||
companyIds: [graph.otherCompanyId],
|
||||
source: "session",
|
||||
isInstanceAdmin: false,
|
||||
});
|
||||
const boardApp = createApp(db, {
|
||||
type: "board",
|
||||
userId: "board-user",
|
||||
companyIds: [graph.companyId],
|
||||
source: "session",
|
||||
isInstanceAdmin: false,
|
||||
});
|
||||
|
||||
expect((await request(agentApp)
|
||||
.post(`/api/issues/${graph.issueId}/file-resources/availability`)
|
||||
.send({ queries: [{ path: "README.md" }] })).status).toBe(403);
|
||||
expect((await request(otherCompanyApp)
|
||||
.post(`/api/issues/${graph.issueId}/file-resources/availability`)
|
||||
.send({ queries: [{ path: "README.md" }] })).status).toBe(404);
|
||||
expect((await request(boardApp)
|
||||
.post(`/api/issues/${graph.issueId}/file-resources/availability`)
|
||||
.send({ queries: Array.from({ length: 101 }, (_, index) => ({ path: `file-${index}.ts` })) })).status).toBe(400);
|
||||
});
|
||||
});
|
||||
|
||||
describeEmbeddedPostgres("file resource route guards", () => {
|
||||
@@ -1470,6 +1666,9 @@ describeEmbeddedPostgres("file resource route guards", () => {
|
||||
});
|
||||
const service: WorkspaceFileResourceService = {
|
||||
getIssue: vi.fn(async () => ({ companyId })),
|
||||
availability: vi.fn(async () => {
|
||||
throw new Error("not used");
|
||||
}),
|
||||
list: vi.fn(async () => {
|
||||
throw new Error("not used");
|
||||
}),
|
||||
@@ -1522,4 +1721,65 @@ describeEmbeddedPostgres("file resource route guards", () => {
|
||||
const third = await request(app).get("/api/issues/issue-1/file-resources/resolve").query({ path: "README.md" });
|
||||
expect(third.status).toBe(429);
|
||||
});
|
||||
|
||||
it("uses a batch-specific availability limiter", async () => {
|
||||
const companyId = crypto.randomUUID();
|
||||
await db.insert(companies).values({
|
||||
id: companyId,
|
||||
name: "Availability rate limit company",
|
||||
issuePrefix: "AVL",
|
||||
});
|
||||
let releaseSlowAvailability: (() => void) | null = null;
|
||||
let slowAvailabilityStarted: (() => void) | null = null;
|
||||
const slowAvailability = new Promise<void>((resolve) => {
|
||||
releaseSlowAvailability = resolve;
|
||||
});
|
||||
const availabilityStarted = new Promise<void>((resolve) => {
|
||||
slowAvailabilityStarted = resolve;
|
||||
});
|
||||
const service: WorkspaceFileResourceService = {
|
||||
getIssue: vi.fn(async () => ({ companyId })),
|
||||
availability: vi.fn(async () => {
|
||||
slowAvailabilityStarted?.();
|
||||
await slowAvailability;
|
||||
return { kind: "workspace_file_availability", results: [] };
|
||||
}),
|
||||
list: vi.fn(async () => { throw new Error("not used"); }),
|
||||
resolve: vi.fn(async () => { throw new Error("not used"); }),
|
||||
readContent: vi.fn(async () => { throw new Error("not used"); }),
|
||||
prepareDownload: vi.fn(async () => { throw new Error("not used"); }),
|
||||
};
|
||||
const app = createApp(
|
||||
db,
|
||||
{
|
||||
type: "board",
|
||||
userId: "board-user",
|
||||
companyIds: [companyId],
|
||||
source: "session",
|
||||
isInstanceAdmin: false,
|
||||
},
|
||||
{
|
||||
service,
|
||||
availabilityLimiter: createFileResourceAvailabilityLimiter({
|
||||
maxConcurrent: 1,
|
||||
maxRequests: 2,
|
||||
windowMs: 60_000,
|
||||
}),
|
||||
},
|
||||
);
|
||||
|
||||
const firstRequest = request(app)
|
||||
.post("/api/issues/issue-availability/file-resources/availability")
|
||||
.send({ queries: [{ path: "README.md" }] });
|
||||
const firstResponse = firstRequest.then((response) => response);
|
||||
await availabilityStarted;
|
||||
expect((await request(app)
|
||||
.post("/api/issues/issue-availability/file-resources/availability")
|
||||
.send({ queries: [{ path: "README.md" }] })).status).toBe(429);
|
||||
releaseSlowAvailability?.();
|
||||
expect((await firstResponse).status).toBe(200);
|
||||
expect((await request(app)
|
||||
.post("/api/issues/issue-availability/file-resources/availability")
|
||||
.send({ queries: [{ path: "README.md" }] })).status).toBe(429);
|
||||
});
|
||||
});
|
||||
@@ -4,19 +4,27 @@ import { Router } from "express";
|
||||
import { ZodError } from "zod";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
workspaceFileAvailabilityRequestSchema,
|
||||
workspaceFileListQuerySchema,
|
||||
workspaceFileResourceQuerySchema,
|
||||
type ResolvedWorkspaceResource,
|
||||
type WorkspaceFileAvailabilityRequestInput,
|
||||
type WorkspaceFileAvailabilityResponse,
|
||||
type WorkspaceFileContent,
|
||||
type WorkspaceFileListResponse,
|
||||
} from "@paperclipai/shared";
|
||||
import { HttpError, notFound, unprocessable } from "../errors.js";
|
||||
import { badRequest, HttpError, notFound, unprocessable } from "../errors.js";
|
||||
import { workspaceFileResourceService } from "../services/index.js";
|
||||
import { assertBoard, getActorInfo, hasCompanyAccess } from "./authz.js";
|
||||
import { logActivity } from "../services/activity-log.js";
|
||||
|
||||
export type WorkspaceFileResourceService = {
|
||||
getIssue(issueId: string): Promise<{ companyId: string }>;
|
||||
availability(
|
||||
issueId: string,
|
||||
input: WorkspaceFileAvailabilityRequestInput,
|
||||
opts?: { issue?: Awaited<ReturnType<WorkspaceFileResourceService["getIssue"]>> },
|
||||
): Promise<WorkspaceFileAvailabilityResponse>;
|
||||
list(issueId: string, input: {
|
||||
workspace?: "auto" | "execution" | "project" | null;
|
||||
projectId?: string | null;
|
||||
@@ -107,6 +115,20 @@ export function createFileResourceListLimiter(opts: {
|
||||
});
|
||||
}
|
||||
|
||||
export function createFileResourceAvailabilityLimiter(opts: {
|
||||
maxConcurrent?: number;
|
||||
maxRequests?: number;
|
||||
windowMs?: number;
|
||||
} = {}): FileResourceLimiter {
|
||||
return createFileResourceLimiter({
|
||||
maxConcurrent: opts.maxConcurrent ?? 2,
|
||||
maxRequests: opts.maxRequests ?? 60,
|
||||
windowMs: opts.windowMs,
|
||||
requestLimitMessage: "Too many workspace file availability requests",
|
||||
concurrencyLimitMessage: "Too many concurrent workspace file availability requests",
|
||||
});
|
||||
}
|
||||
|
||||
function limiterKey(companyId: string, actorId: string, issueId: string) {
|
||||
return `${companyId}:${actorId}:${issueId}`;
|
||||
}
|
||||
@@ -170,6 +192,20 @@ function readListQuery(query: unknown) {
|
||||
};
|
||||
}
|
||||
|
||||
function readAvailabilityBody(body: unknown) {
|
||||
try {
|
||||
return workspaceFileAvailabilityRequestSchema.parse(body);
|
||||
} catch (error) {
|
||||
if (error instanceof ZodError) {
|
||||
throw badRequest("Workspace file availability request is invalid", {
|
||||
code: "invalid_availability_request",
|
||||
issues: error.issues,
|
||||
});
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
function activityDetails(input: {
|
||||
outcome: "success" | "denied" | "unavailable";
|
||||
workspaceKind?: string | null;
|
||||
@@ -222,6 +258,30 @@ function listActivityDetails(input: {
|
||||
};
|
||||
}
|
||||
|
||||
function availabilityActivityDetails(input: {
|
||||
outcome: "success" | "denied";
|
||||
requestedCount: number;
|
||||
uniqueCount?: number;
|
||||
openableCount?: number;
|
||||
unavailableCount?: number;
|
||||
denialReason?: string | null;
|
||||
}) {
|
||||
return {
|
||||
outcome: input.outcome,
|
||||
requestedCount: input.requestedCount,
|
||||
...(typeof input.uniqueCount === "number" ? { uniqueCount: input.uniqueCount } : {}),
|
||||
...(typeof input.openableCount === "number" ? { openableCount: input.openableCount } : {}),
|
||||
...(typeof input.unavailableCount === "number" ? { unavailableCount: input.unavailableCount } : {}),
|
||||
...(input.denialReason ? { denialReason: input.denialReason } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
function safeAvailabilityRequestCount(body: unknown) {
|
||||
if (!body || typeof body !== "object") return 0;
|
||||
const queries = (body as { queries?: unknown }).queries;
|
||||
return Array.isArray(queries) ? queries.length : 0;
|
||||
}
|
||||
|
||||
function safeListAuditQuery(query: unknown): {
|
||||
workspace: "auto" | "execution" | "project";
|
||||
mode: "all" | "recent" | "changed";
|
||||
@@ -268,11 +328,46 @@ export function fileResourceRoutes(db: Db, opts: {
|
||||
service?: WorkspaceFileResourceService;
|
||||
limiter?: FileResourceLimiter;
|
||||
listLimiter?: FileResourceLimiter;
|
||||
availabilityLimiter?: FileResourceLimiter;
|
||||
} = {}) {
|
||||
const router = Router();
|
||||
const svc = opts.service ?? workspaceFileResourceService(db);
|
||||
const limiter = opts.limiter ?? createFileResourceLimiter();
|
||||
const listLimiter = opts.listLimiter ?? createFileResourceListLimiter();
|
||||
const availabilityLimiter = opts.availabilityLimiter ?? createFileResourceAvailabilityLimiter();
|
||||
|
||||
async function logAvailabilityAttempt(input: {
|
||||
companyId: string;
|
||||
actor: ReturnType<typeof getActorInfo>;
|
||||
issueId: string;
|
||||
outcome: "success" | "denied";
|
||||
requestedCount: number;
|
||||
result?: WorkspaceFileAvailabilityResponse;
|
||||
error?: unknown;
|
||||
}) {
|
||||
const openableCount = input.result?.results.filter((result) => result.openable).length;
|
||||
await logActivity(db, {
|
||||
companyId: input.companyId,
|
||||
actorType: input.actor.actorType,
|
||||
actorId: input.actor.actorId,
|
||||
action: input.outcome === "success"
|
||||
? "issue.file_resource_availability"
|
||||
: "issue.file_resource_availability_denied",
|
||||
entityType: "issue",
|
||||
entityId: input.issueId,
|
||||
agentId: input.actor.agentId,
|
||||
runId: input.actor.runId,
|
||||
agentApiKeyId: input.actor.agentApiKeyId,
|
||||
details: availabilityActivityDetails({
|
||||
outcome: input.outcome,
|
||||
requestedCount: input.requestedCount,
|
||||
uniqueCount: input.result?.results.length,
|
||||
openableCount,
|
||||
unavailableCount: input.result ? input.result.results.length - (openableCount ?? 0) : undefined,
|
||||
denialReason: input.error ? denialReasonFromError(input.error) : null,
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
async function logDeniedAttempt(input: {
|
||||
companyId: string;
|
||||
@@ -331,6 +426,82 @@ export function fileResourceRoutes(db: Db, opts: {
|
||||
});
|
||||
}
|
||||
|
||||
router.post("/issues/:issueId/file-resources/availability", async (req, res) => {
|
||||
const requestedCount = safeAvailabilityRequestCount(req.body);
|
||||
try {
|
||||
assertBoard(req);
|
||||
} catch (error) {
|
||||
if (req.actor.type === "agent" && req.actor.companyId) {
|
||||
await logAvailabilityAttempt({
|
||||
companyId: req.actor.companyId,
|
||||
actor: getActorInfo(req),
|
||||
issueId: req.params.issueId,
|
||||
outcome: "denied",
|
||||
requestedCount,
|
||||
error,
|
||||
});
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
|
||||
const issue = await svc.getIssue(req.params.issueId);
|
||||
const actor = getActorInfo(req);
|
||||
if (!hasCompanyAccess(req, issue.companyId)) {
|
||||
const error = notFound("Issue not found");
|
||||
await logAvailabilityAttempt({
|
||||
companyId: issue.companyId,
|
||||
actor,
|
||||
issueId: req.params.issueId,
|
||||
outcome: "denied",
|
||||
requestedCount,
|
||||
error,
|
||||
});
|
||||
throw error;
|
||||
}
|
||||
|
||||
let body: ReturnType<typeof readAvailabilityBody>;
|
||||
try {
|
||||
body = readAvailabilityBody(req.body);
|
||||
} catch (error) {
|
||||
await logAvailabilityAttempt({
|
||||
companyId: issue.companyId,
|
||||
actor,
|
||||
issueId: req.params.issueId,
|
||||
outcome: "denied",
|
||||
requestedCount,
|
||||
error,
|
||||
});
|
||||
throw error;
|
||||
}
|
||||
|
||||
let release: (() => void) | null = null;
|
||||
try {
|
||||
release = availabilityLimiter.acquire(limiterKey(issue.companyId, actor.actorId, req.params.issueId));
|
||||
const result = await svc.availability(req.params.issueId, body, { issue });
|
||||
await logAvailabilityAttempt({
|
||||
companyId: issue.companyId,
|
||||
actor,
|
||||
issueId: req.params.issueId,
|
||||
outcome: "success",
|
||||
requestedCount: body.queries.length,
|
||||
result,
|
||||
});
|
||||
res.json(result);
|
||||
} catch (error) {
|
||||
await logAvailabilityAttempt({
|
||||
companyId: issue.companyId,
|
||||
actor,
|
||||
issueId: req.params.issueId,
|
||||
outcome: "denied",
|
||||
requestedCount: body.queries.length,
|
||||
error,
|
||||
});
|
||||
throw error;
|
||||
} finally {
|
||||
release?.();
|
||||
}
|
||||
});
|
||||
|
||||
router.get("/issues/:issueId/file-resources/list", async (req, res) => {
|
||||
const auditQuery = safeListAuditQuery(req.query);
|
||||
const auditTarget = safeAuditTarget(req.query);
|
||||
|
||||
@@ -12,7 +12,12 @@ export { agentRoutes } from "./agents.js";
|
||||
export { projectRoutes } from "./projects.js";
|
||||
export { issueRoutes } from "./issues.js";
|
||||
export { issueTreeControlRoutes } from "./issue-tree-control.js";
|
||||
export { fileResourceRoutes, createFileResourceLimiter } from "./file-resources.js";
|
||||
export {
|
||||
fileResourceRoutes,
|
||||
createFileResourceAvailabilityLimiter,
|
||||
createFileResourceLimiter,
|
||||
createFileResourceListLimiter,
|
||||
} from "./file-resources.js";
|
||||
export { routineRoutes } from "./routines.js";
|
||||
export { goalRoutes } from "./goals.js";
|
||||
export { approvalRoutes } from "./approvals.js";
|
||||
|
||||
@@ -188,6 +188,8 @@ import {
|
||||
secretProviderConfigDiscoveryPreviewSchema,
|
||||
remoteSecretImportPreviewSchema,
|
||||
remoteSecretImportSchema,
|
||||
workspaceFileAvailabilityRequestSchema,
|
||||
workspaceFileAvailabilityResponseSchema,
|
||||
workspaceFileListQuerySchema,
|
||||
workspaceFileResourceQuerySchema,
|
||||
// Tool access
|
||||
@@ -812,6 +814,7 @@ const BOARD_ONLY_OPERATIONS = new Set([
|
||||
"GET /api/secrets/{id}/usage",
|
||||
"GET /api/secrets/{id}/access-events",
|
||||
"POST /api/health/dev-server/restart",
|
||||
"POST /api/issues/{issueId}/file-resources/availability",
|
||||
"GET /api/issues/{issueId}/file-resources/content",
|
||||
"GET /api/issues/{issueId}/file-resources/list",
|
||||
"GET /api/issues/{issueId}/file-resources/resolve",
|
||||
@@ -2484,6 +2487,28 @@ registry.registerPath({
|
||||
responses: { 200: r.ok(), 401: r.unauthorized, 404: r.notFound },
|
||||
});
|
||||
|
||||
registry.registerPath({
|
||||
method: "post",
|
||||
path: "/api/issues/{issueId}/file-resources/availability",
|
||||
tags: ["issues"],
|
||||
summary: "Check whether issue workspace files can be opened",
|
||||
request: {
|
||||
params: z.object({ issueId: z.string() }),
|
||||
body: {
|
||||
required: true,
|
||||
content: { "application/json": { schema: workspaceFileAvailabilityRequestSchema } },
|
||||
},
|
||||
},
|
||||
responses: {
|
||||
200: r.ok(workspaceFileAvailabilityResponseSchema),
|
||||
400: r.badRequest,
|
||||
401: r.unauthorized,
|
||||
403: r.forbidden,
|
||||
404: r.notFound,
|
||||
429: r.tooManyRequests,
|
||||
},
|
||||
});
|
||||
|
||||
registry.registerPath({
|
||||
method: "get",
|
||||
path: "/api/issues/{issueId}/file-resources/list",
|
||||
|
||||
@@ -6,7 +6,11 @@ import { and, desc, eq, inArray, isNull } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { executionWorkspaces, issues, projects, projectWorkspaces } from "@paperclipai/db";
|
||||
import type {
|
||||
NormalizedWorkspaceFileAvailabilityQuery,
|
||||
ResolvedWorkspaceResource,
|
||||
WorkspaceFileAvailabilityRequestInput,
|
||||
WorkspaceFileAvailabilityResponse,
|
||||
WorkspaceFileAvailabilityResult,
|
||||
WorkspaceFileContent,
|
||||
WorkspaceFileListItem,
|
||||
WorkspaceFileListMode,
|
||||
@@ -22,6 +26,7 @@ export const WORKSPACE_FILE_MEDIA_MAX_BYTES = 10 * 1024 * 1024;
|
||||
export const WORKSPACE_FILE_LIST_DEFAULT_LIMIT = 25;
|
||||
export const WORKSPACE_FILE_LIST_MAX_LIMIT = 100;
|
||||
export const WORKSPACE_FILE_LIST_MAX_SCANNED_ENTRIES = 5_000;
|
||||
export const WORKSPACE_FILE_AVAILABILITY_CONCURRENCY = 8;
|
||||
const MAX_RELATIVE_PATH_BYTES = 4096;
|
||||
const TEXT_SNIFF_BYTES = 4096;
|
||||
const MAX_LIST_DEPTH = 20;
|
||||
@@ -147,12 +152,122 @@ type WorkspaceTargetInput = {
|
||||
workspaceId?: string | null;
|
||||
};
|
||||
|
||||
type WorkspaceFileAvailabilityQueryInput = WorkspaceFileAvailabilityRequestInput["queries"][number];
|
||||
|
||||
type PreparedAvailabilityQuery = {
|
||||
query: NormalizedWorkspaceFileAvailabilityQuery;
|
||||
normalizedPath: NormalizedPath | null;
|
||||
directory: boolean;
|
||||
key: string;
|
||||
unavailableReason?: string;
|
||||
};
|
||||
|
||||
type PreparedAvailabilityTarget =
|
||||
| { candidate: WorkspaceCandidate; error?: never }
|
||||
| { candidate?: never; error: HttpError };
|
||||
|
||||
function previewCapForKind(kind: WorkspaceFilePreviewKind) {
|
||||
return kind === "image" || kind === "video" || kind === "pdf"
|
||||
? WORKSPACE_FILE_MEDIA_MAX_BYTES
|
||||
: WORKSPACE_FILE_TEXT_MAX_BYTES;
|
||||
}
|
||||
|
||||
function safeRejectedAvailabilityPath(input: string) {
|
||||
const trimmed = input.trim();
|
||||
const slashPath = trimmed.replaceAll("\\", "/");
|
||||
if (path.posix.isAbsolute(slashPath) || /^[a-zA-Z]:/.test(trimmed) || /^file:\/\//i.test(trimmed)) {
|
||||
return path.posix.basename(slashPath) || "[invalid path]";
|
||||
}
|
||||
return trimmed;
|
||||
}
|
||||
|
||||
function prepareAvailabilityQuery(input: WorkspaceFileAvailabilityQueryInput): PreparedAvailabilityQuery {
|
||||
const directory = input.path.trim().endsWith("/");
|
||||
const baseQuery: NormalizedWorkspaceFileAvailabilityQuery = {
|
||||
path: input.path.trim(),
|
||||
workspace: input.workspace ?? "auto",
|
||||
projectId: input.projectId ?? null,
|
||||
workspaceId: input.workspaceId ?? null,
|
||||
};
|
||||
try {
|
||||
const normalizedPath = normalizeWorkspaceRelativePath(input.path);
|
||||
const query = {
|
||||
...baseQuery,
|
||||
path: `${normalizedPath.relativePath}${directory ? "/" : ""}`,
|
||||
};
|
||||
return {
|
||||
query,
|
||||
normalizedPath,
|
||||
directory,
|
||||
key: JSON.stringify([query.workspace, query.projectId, query.workspaceId, query.path]),
|
||||
};
|
||||
} catch (error) {
|
||||
const unavailableReason = expectedAvailabilityReason(error);
|
||||
if (!unavailableReason) throw error;
|
||||
const safePath = safeRejectedAvailabilityPath(input.path);
|
||||
return {
|
||||
query: { ...baseQuery, path: safePath },
|
||||
normalizedPath: null,
|
||||
directory,
|
||||
key: JSON.stringify([baseQuery.workspace, baseQuery.projectId, baseQuery.workspaceId, baseQuery.path]),
|
||||
unavailableReason,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
function expectedAvailabilityReason(error: unknown): string | null {
|
||||
if (!(error instanceof HttpError) || ![403, 404, 409, 422].includes(error.status)) return null;
|
||||
if (error.details && typeof error.details === "object" && "code" in error.details) {
|
||||
const code = (error.details as { code?: unknown }).code;
|
||||
if (typeof code === "string" && code.length > 0) return code;
|
||||
}
|
||||
if (error.status === 403) return "forbidden";
|
||||
if (error.status === 404) return "not_found";
|
||||
if (error.status === 409) return "conflict";
|
||||
return "unprocessable";
|
||||
}
|
||||
|
||||
function availabilityResult(
|
||||
query: NormalizedWorkspaceFileAvailabilityQuery,
|
||||
resource: ResolvedWorkspaceResource,
|
||||
): WorkspaceFileAvailabilityResult {
|
||||
const openable = resource.kind === "file"
|
||||
? resource.capabilities.preview
|
||||
: resource.kind === "directory" && resource.capabilities.listChildren;
|
||||
return {
|
||||
query,
|
||||
openable,
|
||||
...(openable ? {} : { unavailableReason: resource.denialReason ?? "unsupported_resource" }),
|
||||
resource,
|
||||
};
|
||||
}
|
||||
|
||||
function unavailableAvailabilityResult(
|
||||
query: NormalizedWorkspaceFileAvailabilityQuery,
|
||||
unavailableReason: string,
|
||||
): WorkspaceFileAvailabilityResult {
|
||||
return {
|
||||
query,
|
||||
openable: false,
|
||||
unavailableReason,
|
||||
resource: null,
|
||||
};
|
||||
}
|
||||
|
||||
async function mapWithConcurrency<T, R>(items: T[], concurrency: number, mapper: (item: T) => Promise<R>) {
|
||||
const results = new Array<R>(items.length);
|
||||
let nextIndex = 0;
|
||||
async function worker() {
|
||||
while (nextIndex < items.length) {
|
||||
const index = nextIndex;
|
||||
nextIndex += 1;
|
||||
results[index] = await mapper(items[index]!);
|
||||
}
|
||||
}
|
||||
await Promise.all(Array.from({ length: Math.min(concurrency, items.length) }, () => worker()));
|
||||
return results;
|
||||
}
|
||||
|
||||
function relativePathFromReal(rootReal: string, targetReal: string) {
|
||||
return path.relative(rootReal, targetReal).split(path.sep).join(path.posix.sep);
|
||||
}
|
||||
@@ -994,6 +1109,51 @@ export function workspaceFileResourceService(db: Db) {
|
||||
return candidates;
|
||||
}
|
||||
|
||||
async function loadAvailabilityTargets(
|
||||
issue: IssueRow,
|
||||
queries: PreparedAvailabilityQuery[],
|
||||
): Promise<Map<string, PreparedAvailabilityTarget>> {
|
||||
const targetQueries = queries.filter(
|
||||
(item) => !item.unavailableReason && item.query.projectId && item.query.workspaceId,
|
||||
);
|
||||
const projectIds = [...new Set(targetQueries.map((item) => item.query.projectId!))];
|
||||
const workspaceIds = [...new Set(targetQueries.map((item) => item.query.workspaceId!))];
|
||||
const [projectRows, workspaceRows] = await Promise.all([
|
||||
projectIds.length > 0
|
||||
? db.select().from(projects).where(inArray(projects.id, projectIds))
|
||||
: Promise.resolve([]),
|
||||
workspaceIds.length > 0
|
||||
? db.select().from(projectWorkspaces).where(inArray(projectWorkspaces.id, workspaceIds))
|
||||
: Promise.resolve([]),
|
||||
]);
|
||||
const projectById = new Map(projectRows.map((row) => [row.id, row]));
|
||||
const workspaceById = new Map(workspaceRows.map((row) => [row.id, row]));
|
||||
const targets = new Map<string, PreparedAvailabilityTarget>();
|
||||
|
||||
for (const item of targetQueries) {
|
||||
const targetKey = `${item.query.projectId}:${item.query.workspaceId}`;
|
||||
if (targets.has(targetKey)) continue;
|
||||
const project = projectById.get(item.query.projectId!);
|
||||
const workspace = workspaceById.get(item.query.workspaceId!);
|
||||
if (!project || !workspace) {
|
||||
targets.set(targetKey, { error: notFound("Project workspace not found") });
|
||||
} else if (project.companyId !== issue.companyId || workspace.companyId !== issue.companyId) {
|
||||
targets.set(targetKey, {
|
||||
error: new HttpError(403, "Project workspace belongs to another company", { code: "cross_company_workspace" }),
|
||||
});
|
||||
} else if (workspace.projectId !== project.id) {
|
||||
targets.set(targetKey, {
|
||||
error: unprocessable("Workspace does not belong to the selected project", { code: "workspace_project_mismatch" }),
|
||||
});
|
||||
} else {
|
||||
targets.set(targetKey, {
|
||||
candidate: candidateFromProjectWorkspace(workspace, { id: project.id, name: project.name }),
|
||||
});
|
||||
}
|
||||
}
|
||||
return targets;
|
||||
}
|
||||
|
||||
async function sameCompanyProjectWorkspaceCandidates(
|
||||
issue: IssueRow,
|
||||
excludedWorkspaceIds: Set<string>,
|
||||
@@ -1051,6 +1211,108 @@ export function workspaceFileResourceService(db: Db) {
|
||||
});
|
||||
}
|
||||
|
||||
async function availability(
|
||||
issueId: string,
|
||||
input: WorkspaceFileAvailabilityRequestInput,
|
||||
opts: { issue?: IssueRow } = {},
|
||||
): Promise<WorkspaceFileAvailabilityResponse> {
|
||||
const issue = opts.issue ?? await getIssue(issueId);
|
||||
const uniqueQueries = new Map<string, PreparedAvailabilityQuery>();
|
||||
for (const inputQuery of input.queries) {
|
||||
const prepared = prepareAvailabilityQuery(inputQuery);
|
||||
if (!uniqueQueries.has(prepared.key)) uniqueQueries.set(prepared.key, prepared);
|
||||
}
|
||||
const queries = [...uniqueQueries.values()];
|
||||
if (queries.length === 0) return { kind: "workspace_file_availability", results: [] };
|
||||
|
||||
const untargetedQueries = queries.filter(
|
||||
(item) => !item.unavailableReason && !item.query.projectId && !item.query.workspaceId,
|
||||
);
|
||||
const initialCandidates = untargetedQueries.length > 0 ? await listCandidates(issue, "auto") : [];
|
||||
const needsDiscovery = initialCandidates.some((candidate) => !candidate.remote)
|
||||
&& untargetedQueries.some((item) => item.query.workspace === "auto");
|
||||
const discoveryCandidates = needsDiscovery
|
||||
? await sameCompanyProjectWorkspaceCandidates(issue, new Set(initialCandidates.map((candidate) => candidate.workspaceId)))
|
||||
: [];
|
||||
const explicitTargets = await loadAvailabilityTargets(issue, queries);
|
||||
|
||||
const results = await mapWithConcurrency(
|
||||
queries,
|
||||
WORKSPACE_FILE_AVAILABILITY_CONCURRENCY,
|
||||
async (item): Promise<WorkspaceFileAvailabilityResult> => {
|
||||
try {
|
||||
if (item.unavailableReason || !item.normalizedPath) {
|
||||
return unavailableAvailabilityResult(item.query, item.unavailableReason ?? "invalid_path");
|
||||
}
|
||||
const explicitTarget = item.query.projectId && item.query.workspaceId
|
||||
? explicitTargets.get(`${item.query.projectId}:${item.query.workspaceId}`)
|
||||
: null;
|
||||
if (explicitTarget?.error) throw explicitTarget.error;
|
||||
|
||||
const candidates = explicitTarget?.candidate
|
||||
? [explicitTarget.candidate]
|
||||
: initialCandidates.filter((candidate) => {
|
||||
if (item.query.workspace === "execution") return candidate.workspaceKind === "execution_workspace";
|
||||
if (item.query.workspace === "project") return candidate.workspaceKind === "project_workspace";
|
||||
return true;
|
||||
});
|
||||
if (candidates.length === 0) {
|
||||
throw unprocessable("No workspace is available for this issue", { code: "no_workspace" });
|
||||
}
|
||||
|
||||
const hasExplicitTarget = Boolean(explicitTarget?.candidate);
|
||||
let lastNotFound: unknown = null;
|
||||
for (const candidate of candidates) {
|
||||
if (candidate.remote) {
|
||||
if (hasExplicitTarget || item.query.workspace !== "auto") {
|
||||
return availabilityResult(item.query, remoteResource(candidate, item.normalizedPath.relativePath));
|
||||
}
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
const resource = item.directory
|
||||
? (await statLocalDirectory(candidate, item.normalizedPath)).resource
|
||||
: (await statLocalCandidate(candidate, item.normalizedPath)).resource;
|
||||
return availabilityResult(item.query, resource);
|
||||
} catch (error) {
|
||||
if (!hasExplicitTarget && item.query.workspace === "auto" && isHttpStatus(error, 404)) {
|
||||
lastNotFound = error;
|
||||
continue;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
if (lastNotFound && !hasExplicitTarget && item.query.workspace === "auto") {
|
||||
const matches: ResolvedWorkspaceResource[] = [];
|
||||
for (const candidate of discoveryCandidates) {
|
||||
if (candidate.remote) continue;
|
||||
try {
|
||||
matches.push(item.directory
|
||||
? (await statLocalDirectory(candidate, item.normalizedPath)).resource
|
||||
: (await statLocalCandidate(candidate, item.normalizedPath)).resource);
|
||||
} catch (error) {
|
||||
if (isHttpStatus(error, 404)) continue;
|
||||
throw error;
|
||||
}
|
||||
if (matches.length > 1) throwAmbiguousWorkspacePath(matches.length);
|
||||
}
|
||||
if (matches[0]) return availabilityResult(item.query, matches[0]);
|
||||
}
|
||||
|
||||
if (lastNotFound) throw lastNotFound;
|
||||
throw unprocessable("No local-readable workspace is available for this issue", { code: "no_local_workspace" });
|
||||
} catch (error) {
|
||||
const reason = expectedAvailabilityReason(error);
|
||||
if (!reason) throw error;
|
||||
return unavailableAvailabilityResult(item.query, reason);
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
return { kind: "workspace_file_availability", results };
|
||||
}
|
||||
|
||||
async function resolve(issueId: string, input: {
|
||||
path: string;
|
||||
workspace?: WorkspaceFileSelector | null;
|
||||
@@ -1443,6 +1705,7 @@ export function workspaceFileResourceService(db: Db) {
|
||||
|
||||
return {
|
||||
getIssue,
|
||||
availability,
|
||||
list,
|
||||
resolve,
|
||||
readContent,
|
||||
|
||||
Reference in new issue
Block a user