mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-29 12:59:13 +01:00
fix(runner): enable API tools by default (#14186)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The native runner exposes tools for company tasks.
> - API search and call tools cover operations without a dedicated tool.
> - The current default hides these tools unless an operator sets an
environment variable.
> - This pull request enables the tools when that variable is absent.
> - Operators can still disable the tools or restrict them to selected
companies.
## Linked Issues or Issue Description
Refs #13003, which added the guarded API tools.
**What happened?**
The native runner does not advertise `search_api` or `call_api` with the
default server configuration.
**Expected behavior**
The tools are available without a special environment variable. Existing
authorization checks still apply.
**Steps to reproduce**
Remove `PAPERCLIP_RUNNER_API_TOOLS_ENABLED` and
`PAPERCLIP_RUNNER_API_TOOLS_COMPANY_IDS`. Create a normal runner
authority. Inspect its tool definitions.
**Paperclip version or commit**
a6c4e7a.
## What Changed
- Enable API tools when the server flag is absent.
- Keep explicit disable, invalid-value rejection, company restrictions,
and binding restrictions.
- Test default tool definitions and run HTTP integration tests without
the enabling flag.
- Update operator and hiring documentation. The existing shared gate
also controls `hire_agent`.
- Use the same policy in the E2E evidence summary so an unset flag is
not reported as disabled.
- Isolate default-availability tests from operator environment
variables.
## Verification
- Red test: two new rollout policy assertions failed before the fix.
- Focused policy, authority, and HTTP tests: 3 files passed; 41 tests
passed, 2 skipped. The two runnerd transport cases require a Rust-built
binary absent from this workspace.
- Command: `pnpm exec vitest run
server/src/services/native-runtime/runner-api-rollout.test.ts
server/src/services/native-runtime/paperclip-runner-tool-authority.test.ts
server/src/services/native-runtime/runner-api.integration.test.ts`.
- Attempted `pnpm -r typecheck`: blocked by missing `cargo` in this
workspace.
- Attempted `pnpm build`: terminated at the 4 GiB memory limit.
- Attempted `pnpm test:run`: stopped after memory pressure to run
focused tests alone.
- Repeated the 41 passing focused tests with an inherited disabled flag
and a foreign-company restriction; test isolation passed.
- `pnpm test:e2e:runner:unit`: 44 files and 543 tests passed.
- `pnpm test:e2e:runner:typecheck` exceeded the workspace memory limit,
including a retry with bounded Go memory settings.
- CI results will be recorded before handoff.
## Risks
- More native runs can discover API tools and the existing `hire_agent`
tool by default.
- The change does not remove company, run, mode, credential, lifecycle,
or approval checks.
- Explicit operator restrictions still take precedence. No database
migration is required.
## Model Used
OpenAI Codex agent. The runtime does not expose the exact model ID or
context-window size. Used reasoning, repository tools, shell execution,
and tests.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
a6c4e7a8d1
commit
f2ed0b65c4
9 files changed
+69
-49
No files matched your search
@@ -1,5 +1,5 @@
|
||||
import * as cloudIdentity from "../cloud-runtime-identity.js";
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { and, eq, inArray } from "drizzle-orm";
|
||||
import {
|
||||
@@ -33,6 +33,12 @@ describe("PaperclipRunnerToolAuthority", () => {
|
||||
const issueId = "00000000-0000-4000-8000-000000000103";
|
||||
const runId = "00000000-0000-4000-8000-000000000104";
|
||||
|
||||
beforeEach(() => {
|
||||
vi.stubEnv("PAPERCLIP_RUNNER_API_TOOLS_ENABLED", undefined);
|
||||
vi.stubEnv("PAPERCLIP_RUNNER_API_TOOLS_COMPANY_IDS", undefined);
|
||||
});
|
||||
afterEach(() => vi.unstubAllEnvs());
|
||||
|
||||
beforeAll(async () => {
|
||||
temporary = await startEmbeddedPostgresTestDatabase(
|
||||
"paperclip-runner-tools-",
|
||||
@@ -91,7 +97,7 @@ describe("PaperclipRunnerToolAuthority", () => {
|
||||
issueId,
|
||||
runId,
|
||||
});
|
||||
expect(authority.definitions()).toHaveLength(27);
|
||||
expect(authority.definitions()).toHaveLength(30);
|
||||
const questions = authority.definitions().find(tool => tool.name === "request_human_input")!;
|
||||
expect(questions.description).toContain("ask only the next unanswered question");
|
||||
expect(questions.description).toContain("Never infer answers");
|
||||
@@ -103,6 +109,7 @@ describe("PaperclipRunnerToolAuthority", () => {
|
||||
expect.arrayContaining([
|
||||
"connections_search",
|
||||
"connection_request", "create_project", "list_project_repositories", "list_projects",
|
||||
"search_api", "call_api", "hire_agent",
|
||||
"get_task_context",
|
||||
"get_task_history",
|
||||
"search_tasks",
|
||||
@@ -216,7 +223,7 @@ describe("PaperclipRunnerToolAuthority", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("preserves direct-chat file tools across the guarded API rollout", () => {
|
||||
it("advertises API tools by default and preserves direct-chat file tools when disabled", () => {
|
||||
const previousEnabled = process.env.PAPERCLIP_RUNNER_API_TOOLS_ENABLED;
|
||||
const previousCompanies =
|
||||
process.env.PAPERCLIP_RUNNER_API_TOOLS_COMPANY_IDS;
|
||||
@@ -240,15 +247,15 @@ describe("PaperclipRunnerToolAuthority", () => {
|
||||
try {
|
||||
delete process.env.PAPERCLIP_RUNNER_API_TOOLS_ENABLED;
|
||||
delete process.env.PAPERCLIP_RUNNER_API_TOOLS_COMPANY_IDS;
|
||||
const disabledNames = createAuthority()
|
||||
const defaultNames = createAuthority()
|
||||
.definitions()
|
||||
.map((tool) => tool.name);
|
||||
expect(disabledNames).toEqual(
|
||||
expect(defaultNames).toEqual(
|
||||
expect.arrayContaining(requiredChatFileTools),
|
||||
);
|
||||
expect(disabledNames).not.toContain("search_api");
|
||||
expect(disabledNames).not.toContain("call_api");
|
||||
expect(disabledNames).not.toContain("hire_agent");
|
||||
expect(defaultNames).toContain("search_api");
|
||||
expect(defaultNames).toContain("call_api");
|
||||
expect(defaultNames).toContain("hire_agent");
|
||||
|
||||
process.env.PAPERCLIP_RUNNER_API_TOOLS_ENABLED = "true";
|
||||
process.env.PAPERCLIP_RUNNER_API_TOOLS_COMPANY_IDS = companyId;
|
||||
@@ -268,6 +275,13 @@ describe("PaperclipRunnerToolAuthority", () => {
|
||||
expect(hireSchema.properties).toEqual(expect.objectContaining({ name: expect.any(Object), role: expect.any(Object) }));
|
||||
expect(hireSchema.properties).not.toHaveProperty("adapterConfig");
|
||||
expect(hireSchema.properties).not.toHaveProperty("env");
|
||||
|
||||
process.env.PAPERCLIP_RUNNER_API_TOOLS_ENABLED = "false";
|
||||
const disabledNames = createAuthority().definitions().map((tool) => tool.name);
|
||||
expect(disabledNames).toEqual(expect.arrayContaining(requiredChatFileTools));
|
||||
for (const name of ["search_api", "call_api", "hire_agent"]) {
|
||||
expect(disabledNames).not.toContain(name);
|
||||
}
|
||||
} finally {
|
||||
if (previousEnabled === undefined) {
|
||||
delete process.env.PAPERCLIP_RUNNER_API_TOOLS_ENABLED;
|
||||
|
||||
@@ -2,14 +2,19 @@ import { describe, expect, it } from "vitest";
|
||||
import { runnerApiToolsEnabled } from "./runner-api-rollout.js";
|
||||
|
||||
describe("runner API rollout", () => {
|
||||
it("requires operator opt-in and fails closed for invalid settings", () => {
|
||||
for (const value of [undefined, "", "TRUE", "1", "invalid"]) {
|
||||
it("enables API tools without operator configuration", () => {
|
||||
expect(runnerApiToolsEnabled("company", undefined, {})).toBe(true);
|
||||
expect(runnerApiToolsEnabled("company", true, {})).toBe(true);
|
||||
expect(runnerApiToolsEnabled("company", false, {})).toBe(false);
|
||||
});
|
||||
it("fails closed for explicit disabled or invalid settings", () => {
|
||||
for (const value of ["false", "", "TRUE", "1", "invalid"]) {
|
||||
for (const binding of [undefined, false, true]) expect(runnerApiToolsEnabled("company", binding, { PAPERCLIP_RUNNER_API_TOOLS_ENABLED: value })).toBe(false);
|
||||
}
|
||||
expect(runnerApiToolsEnabled("company", undefined, { PAPERCLIP_RUNNER_API_TOOLS_ENABLED: "true" })).toBe(true);
|
||||
});
|
||||
it("restricts an enabled rollout to exact company IDs", () => {
|
||||
const env = { PAPERCLIP_RUNNER_API_TOOLS_ENABLED: "true", PAPERCLIP_RUNNER_API_TOOLS_COMPANY_IDS: " alpha, beta " };
|
||||
it.each([undefined, "true"])("restricts enabled tools to exact company IDs (flag: %s)", (enabled) => {
|
||||
const env = { PAPERCLIP_RUNNER_API_TOOLS_ENABLED: enabled, PAPERCLIP_RUNNER_API_TOOLS_COMPANY_IDS: " alpha, beta " };
|
||||
expect(runnerApiToolsEnabled("alpha", undefined, env)).toBe(true);
|
||||
expect(runnerApiToolsEnabled("alph", undefined, env)).toBe(false);
|
||||
expect(runnerApiToolsEnabled("foreign", true, env)).toBe(false);
|
||||
@@ -17,7 +22,7 @@ describe("runner API rollout", () => {
|
||||
});
|
||||
it("keeps baseline tools disabled and honors an operator stop over explicit bindings", () => {
|
||||
expect(runnerApiToolsEnabled("company", false, { PAPERCLIP_RUNNER_API_TOOLS_ENABLED: "true" })).toBe(false);
|
||||
expect(runnerApiToolsEnabled("company", true, {})).toBe(false);
|
||||
expect(runnerApiToolsEnabled("company", false, {})).toBe(false);
|
||||
expect(runnerApiToolsEnabled("company", true, { PAPERCLIP_RUNNER_API_TOOLS_ENABLED: "false" })).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -5,8 +5,8 @@ export function runnerApiToolsEnabled(
|
||||
environment: NodeJS.ProcessEnv = process.env,
|
||||
): boolean {
|
||||
const enabled = environment.PAPERCLIP_RUNNER_API_TOOLS_ENABLED;
|
||||
// A binding can only narrow operator permission, never opt into this surface.
|
||||
if (enabled !== "true" || bindingOverride === false) return false;
|
||||
// Enabled by default. Explicit settings fail closed; bindings can only narrow access.
|
||||
if ((enabled !== undefined && enabled !== "true") || bindingOverride === false) return false;
|
||||
const companies = environment.PAPERCLIP_RUNNER_API_TOOLS_COMPANY_IDS;
|
||||
if (companies === undefined) return true;
|
||||
return companies.split(",").map(value => value.trim()).filter(Boolean).includes(companyId);
|
||||
|
||||
@@ -4,7 +4,7 @@ import { chmod, writeFile, symlink } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { documents, heartbeatRuns, issues, routineDocuments, routines } from "@paperclipai/db";
|
||||
import { beforeAll, afterAll, describe, expect, it, vi } from "vitest";
|
||||
import { beforeAll, afterAll, beforeEach, afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { startRunnerApiTestServer } from "../../__tests__/helpers/runner-api-server.js";
|
||||
import { createRunnerdCodexTransport, defaultCapabilityRunnerdBinary } from "../../vendor/paperclip-runner/index.js";
|
||||
import { runnerApiCatalog } from "./runner-api-catalog.js";
|
||||
@@ -13,18 +13,19 @@ import { registerRunnerPrpAuthority } from "../../realtime/runner-prp-ws.js";
|
||||
describe("runner API against real HTTP routes", () => {
|
||||
let server: Awaited<ReturnType<typeof startRunnerApiTestServer>>;
|
||||
const oldSecret = process.env.PAPERCLIP_AGENT_JWT_SECRET;
|
||||
const oldEnabled = process.env.PAPERCLIP_RUNNER_API_TOOLS_ENABLED;
|
||||
beforeEach(() => {
|
||||
vi.stubEnv("PAPERCLIP_RUNNER_API_TOOLS_ENABLED", undefined);
|
||||
vi.stubEnv("PAPERCLIP_RUNNER_API_TOOLS_COMPANY_IDS", undefined);
|
||||
});
|
||||
afterEach(() => vi.unstubAllEnvs());
|
||||
beforeAll(async () => {
|
||||
process.env.PAPERCLIP_AGENT_JWT_SECRET = randomUUID();
|
||||
process.env.PAPERCLIP_RUNNER_API_TOOLS_ENABLED = "true";
|
||||
server = await startRunnerApiTestServer();
|
||||
}, 60_000);
|
||||
afterAll(async () => {
|
||||
await server?.close();
|
||||
if (oldSecret === undefined) delete process.env.PAPERCLIP_AGENT_JWT_SECRET;
|
||||
else process.env.PAPERCLIP_AGENT_JWT_SECRET = oldSecret;
|
||||
if (oldEnabled === undefined) delete process.env.PAPERCLIP_RUNNER_API_TOOLS_ENABLED;
|
||||
else process.env.PAPERCLIP_RUNNER_API_TOOLS_ENABLED = oldEnabled;
|
||||
});
|
||||
|
||||
it.skipIf(!process.env.PAPERCLIP_REQUIRE_RUNNER_API_INTEGRATION && !existsSync(defaultCapabilityRunnerdBinary())).each(["current", "legacy_http"])("runs runnerd → PRP → authority → actual authenticated HTTP (%s receipt)", async (receiptFormat) => {
|
||||
@@ -92,16 +93,16 @@ else if(m.id!==undefined) send({id:m.id,result:{}});
|
||||
} finally { await bundle.transport.close(); }
|
||||
}, 30_000);
|
||||
|
||||
it("cannot opt into API tools through a binding when the operator flag is absent", async () => {
|
||||
it("cannot opt into API tools through a binding when the operator flag is false", async () => {
|
||||
const fixture = await server.fixture({ apiToolsEnabled: true });
|
||||
delete process.env.PAPERCLIP_RUNNER_API_TOOLS_ENABLED;
|
||||
process.env.PAPERCLIP_RUNNER_API_TOOLS_ENABLED = "false";
|
||||
try {
|
||||
const names = (await fixture.authority.definitions()).map(tool => tool.name);
|
||||
expect(names).toContain("get_task_context");
|
||||
expect(names).not.toContain("search_api");
|
||||
expect(names).not.toContain("call_api");
|
||||
await expect(fixture.authority.execute({ tool: "call_api", callId: "disabled", arguments: { operationId: "GET /api/companies/{companyId}/projects" } })).rejects.toThrow("not_advertised");
|
||||
} finally { process.env.PAPERCLIP_RUNNER_API_TOOLS_ENABLED = "true"; }
|
||||
} finally { delete process.env.PAPERCLIP_RUNNER_API_TOOLS_ENABLED; }
|
||||
});
|
||||
|
||||
it("rejects credential calls before any durable receipt or secret result exists", async () => {
|
||||
|
||||
Reference in new issue
Block a user