mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-29 21:07:40 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native Codex agents use the saved agent settings to prepare each provider launch. > - Onboarding saves web search as off, but native configuration conversion removes that setting. > - Codex 0.160.0 includes a hosted web-search tool by default, even in an authentication probe that asks it not to use tools. > - A gateway that permits only priced function tools can reject that request before budget admission. > - This change preserves the explicit restriction through native execution and disables web search for authentication probes. ## Linked Issues or Issue Description **What happened?** A native Codex authentication probe can include hosted web search. Normal task execution also loses a saved `search: false` setting. A restricted Responses gateway can reject these requests before it makes a model call. Related repairs: Refs #17445, Refs #17529. The roadmap and related PRs were checked; no duplicate repair was found. **Expected behavior** Authentication probes check the selected account and model without hosted web search. A saved search-off setting remains effective in native task launches, prepared launches, and session recovery. Existing behavior for an absent setting remains unchanged. **Steps to reproduce** 1. Create a Codex agent with search off and choose Paperclip Runner. 2. Use a Responses gateway that rejects hosted tools. 3. Inspect the request from the native authentication probe or task launch. Before this fix, the pinned Codex SDK includes `web_search` despite the saved setting. ## What Changed - Preserve an explicit search-off setting when native agent configuration is built and converted. - Carry the restriction in the durable native input and provider launch requirements. - Apply the restriction to authentication probes and normal launch paths, including custom arguments and recovery. - Reject prepared launches that discard the required restriction. Keep gateway billing rules unchanged. - Admit only the restrictive literal value in the strict plugin task API. Exercise the real plugin setup path. - Add regression coverage and document the policy. - Add the immutable Copilot v60 source identity required by the changed shared files. Preserve v59 and all original live evidence. Copilot authentication probes keep their prior launch requirements. ## Verification - Regression cases were run against the unfixed source and failed at the configuration and launch boundaries. - An isolated capture using the real Codex 0.160.0 app-server reproduced hosted-tool rejection. Disabling web search reached a mocked budget reservation. The helper and child processes permitted only loopback network access and used synthetic credentials. It made zero upstream calls. - Focused checks passed: 284 server cases, 61 native form cases, 198 backend/security/launch/parser cases, and 36 authentication/launch cases. A separate complete transport/launch/parser run passed 393 cases. These groups overlap. - Head `617b98f` passed workspace typecheck/build and 28/28 plugin contract and actual Runner probe tests with no skips. Final rebased head `20ff0c1` passed all current-head CI checks, including full isolated test lanes, and fresh Greptile and Paperclip reviews at 5/5. - The additive Copilot identity update passed 134 compatibility/policy/protocol checks and nine authentication probe checks. The real pinned Codex SDK capture was repeated after the final policy change: all five cases passed with zero upstream calls. - Two local full-suite attempts stopped after native PostgreSQL loader failures. Seven local dependency aliases were restored, and an isolated cluster initialized successfully. A third run later encountered host PostgreSQL semaphore exhaustion and was interrupted before source-profile regeneration. Its zero exit code after SIGINT is explicitly not a full-suite pass. Final full-suite verification runs in isolated CI; the local attempts remain recorded. - The first CI run exposed a missing strict plugin API field. Its five failures were reproduced locally, then passed after repair. The next run passed every underlying job except the source-identity guard. That guard now binds an additive identity with v59 retained. Final-head CI and both fresh reviews passed; addressed review threads are resolved. - The offline capture runs on Darwin ARM64. It does not prove the contents of a prior unretained Linux request or complete Cloud onboarding acceptance. ## Risks - The restriction must survive prepared launches and resumed sessions. Regression tests cover those boundaries. - Existing absent settings retain their prior behavior. This change does not add native search-on support or loosen gateway admission. - Older durable inputs without the optional restriction must remain valid. - Copilot now uses profile v60 for exact launch admission. Historical v59 inputs remain readable. The new declaration adds no live Copilot qualification and does not change its executable or admission flags. - A deployed Cloud acceptance run is still required after this source repair. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, code execution, and delegated review. The exact backend model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
692 lines
31 KiB
TypeScript
692 lines
31 KiB
TypeScript
import { describe, expect, it } from "vitest";
|
|
import type { ExecutionContinuationEnvelope, AskUserQuestionsInteraction } from "@paperclipai/shared";
|
|
|
|
import { formatDurableQuestionResponseSummary } from "../question-response-delivery.js";
|
|
import { nativeCompletionSource, buildNativeCompletionContract } from "./completion-contracts.js";
|
|
import { renderPaperclipWakePrompt } from "@paperclipai/adapter-utils/server-utils";
|
|
import { buildNativeExecutionInput } from "./native-execution-input.js";
|
|
import { nativeRuntimeContextFixture } from "./runtime-context.test-fixture.js";
|
|
import { buildNativeModelEnvelope, canonicalNativeRuntimeContextDigest } from "../../vendor/paperclip-runner/index.js";
|
|
|
|
describe("LCA-05 explicit native work mode", () => {
|
|
it.each(["standard", "planning", "ask"])("title and description cannot override %s mode", (workMode) => {
|
|
for (const text of ["Inspect files", "Making a plan", "Create a report", "Research proposal", "Implement now; no plan needed"]) {
|
|
const input = buildNativeExecutionInput({
|
|
companyId: "10000000-0000-4000-8000-000000000001",
|
|
runId: "50000000-0000-4000-8000-000000000005",
|
|
agentId: "30000000-0000-4000-8000-000000000003",
|
|
issue: { id: "20000000-0000-4000-8000-000000000002", identifier: "MODE-1", title: text, description: text, workMode },
|
|
taskPrompt: text,
|
|
workspace: { id: "50000000-0000-4000-8000-000000000005", cwd: "/workspace", repoUrl: null, repoRef: null, branchName: null },
|
|
normalizedSessionId: null,
|
|
provider: "codex",
|
|
codexReasoningEffort: "high",
|
|
codexWebSearch: "disabled",
|
|
planningContext: workMode === "planning" ? { documentId: null, baseRevisionId: null, baseRevisionNumber: 0, markdown: "", sha256: "a".repeat(64), reviewContext: {} } : null,
|
|
completionContract: { id: "70000000-0000-4000-8000-000000000007", sha256: `sha256:${"a".repeat(64)}`, schemaVersion: "paperclip.run-result.v1", contract: { revision: "1", objective: "Deliver the requested work", criteria: [{ id: "output", requirement: "Deliver the requested work" }] } },
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
});
|
|
expect(input.task.workMode).toBe(workMode);
|
|
expect(input.provider).toMatchObject({ kind: "codex", reasoningEffort: "high", webSearch: "disabled" });
|
|
expect(input.executionMode).toBe(workMode === "planning" ? "plan" : "default");
|
|
expect(input.task.title).toBe(text);
|
|
}
|
|
});
|
|
});
|
|
|
|
describe("native execution input external-chat framing", () => {
|
|
it.each([false, true])(
|
|
"projects the authoritative selected answer into an attested native chat prompt (resumed: %s)",
|
|
(resumedSession) => {
|
|
const interactionId = "80000000-0000-4000-8000-000000000008";
|
|
const sourceRunId = "90000000-0000-4000-8000-000000000009";
|
|
const wakePayload = {
|
|
reason: "issue_commented",
|
|
externalChatProvider: "telegram",
|
|
externalChatExecutionBound: true,
|
|
interactionId,
|
|
sourceRunId,
|
|
interactionKind: "ask_user_questions",
|
|
interactionStatus: "answered",
|
|
externalInteractionContinuation: true,
|
|
issue: {
|
|
id: "20000000-0000-4000-8000-000000000002",
|
|
workMode: "standard",
|
|
status: "in_review",
|
|
},
|
|
externalChatQuestionResponse: {
|
|
schema: "paperclip.external_chat_question_response.v1",
|
|
interactionId,
|
|
sourceRunId,
|
|
responseDeliveryId: "10000000-0000-4000-8000-000000000011",
|
|
sourceCommentId: "10000000-0000-4000-8000-000000000012",
|
|
endpointId: "10000000-0000-4000-8000-000000000013",
|
|
conversationId: "10000000-0000-4000-8000-000000000014",
|
|
bindingSha256: "a".repeat(64),
|
|
},
|
|
commentIds: ["10000000-0000-4000-8000-000000000012"],
|
|
latestCommentId: "10000000-0000-4000-8000-000000000012",
|
|
comments: [
|
|
{
|
|
id: "10000000-0000-4000-8000-000000000012",
|
|
body: "Ask for a color, then tell me the selected color.",
|
|
},
|
|
],
|
|
commentWindow: { requestedCount: 1, includedCount: 1, missingCount: 0 },
|
|
fallbackFetchNeeded: false,
|
|
};
|
|
const args: Parameters<typeof buildNativeExecutionInput>[0] = {
|
|
companyId: "10000000-0000-4000-8000-000000000001",
|
|
runId: "50000000-0000-4000-8000-000000000005",
|
|
agentId: "30000000-0000-4000-8000-000000000003",
|
|
issue: {
|
|
id: "20000000-0000-4000-8000-000000000002",
|
|
identifier: "CHAT-5",
|
|
title: "Old exact-output request",
|
|
description: null,
|
|
workMode: "standard",
|
|
},
|
|
taskPrompt:
|
|
"Continue the user's original request with their selected answer.",
|
|
initialCommunicationGuidance: "Initial Slack communication preferences.",
|
|
wakePayload,
|
|
resumedSession,
|
|
workspace: {
|
|
id: "50000000-0000-4000-8000-000000000005",
|
|
cwd: "/workspace",
|
|
repoUrl: null,
|
|
repoRef: null,
|
|
branchName: null,
|
|
},
|
|
normalizedSessionId: "60000000-0000-4000-8000-000000000006",
|
|
provider: "codex",
|
|
completionContract: {
|
|
id: "70000000-0000-4000-8000-000000000007",
|
|
sha256: `sha256:${"a".repeat(64)}`,
|
|
schemaVersion: "paperclip.run-result.v1",
|
|
contract: {
|
|
revision: "1",
|
|
objective: "Answer the user's selected choice",
|
|
criteria: [
|
|
{ id: "answer", requirement: "Return the actual color." },
|
|
],
|
|
},
|
|
},
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
interactionResponses: [
|
|
{
|
|
interactionId,
|
|
kind: "ask_user_questions",
|
|
response: {
|
|
status: "answered",
|
|
result: {
|
|
version: 1,
|
|
answers: [{ questionId: "color", optionIds: ["amber"] }],
|
|
summaryMarkdown:
|
|
"Resolved questions and answers:\n- Choose a color: Amber",
|
|
},
|
|
},
|
|
},
|
|
],
|
|
};
|
|
const input = buildNativeExecutionInput(args);
|
|
expect(input.initialCommunicationGuidance).toBe("Initial Slack communication preferences.");
|
|
// The runtime adds this only after deciding whether provider recovery succeeded.
|
|
expect(input.task.prompt).not.toContain("Initial Slack communication preferences.");
|
|
expect(input.task.title).toBe("External chat follow-up");
|
|
expect(input.task.prompt).toContain("Amber");
|
|
expect(input.task.prompt).not.toContain("## Questions that need a user response");
|
|
expect(input.task.prompt).toContain(
|
|
"semantic completion summary is the user-visible final answer",
|
|
);
|
|
expect(input.task.prompt).not.toContain(
|
|
"including marking the task done",
|
|
);
|
|
expect(input.task.prompt.indexOf("Choose a color: Amber")).toBeLessThan(
|
|
input.task.prompt.indexOf(
|
|
"Ask for a color, then tell me the selected color.",
|
|
),
|
|
);
|
|
expect(wakePayload).not.toHaveProperty("questionResponse");
|
|
|
|
const mixedModalSummary = formatDurableQuestionResponseSummary({
|
|
id: interactionId,
|
|
kind: "ask_user_questions",
|
|
status: "answered",
|
|
sourceRunId,
|
|
payload: {
|
|
version: 1,
|
|
questions: [
|
|
{
|
|
id: "environment",
|
|
prompt: "Choose an environment",
|
|
selectionMode: "single",
|
|
options: [
|
|
{ id: "maple", label: "Maple" },
|
|
{ id: "cedar", label: "Cedar" },
|
|
],
|
|
},
|
|
{
|
|
id: "label",
|
|
prompt: "Enter the verification label",
|
|
selectionMode: "single",
|
|
options: [
|
|
{
|
|
id: "custom",
|
|
label: "Custom label",
|
|
freeText: true,
|
|
},
|
|
],
|
|
},
|
|
],
|
|
},
|
|
result: {
|
|
version: 1,
|
|
answers: [
|
|
{ questionId: "environment", optionIds: ["maple"] },
|
|
{
|
|
questionId: "label",
|
|
optionIds: [],
|
|
otherText: "amber compass 93",
|
|
},
|
|
],
|
|
summaryMarkdown: null,
|
|
},
|
|
} as unknown as AskUserQuestionsInteraction);
|
|
const mixedModal = buildNativeExecutionInput({
|
|
...args,
|
|
wakePayload: {
|
|
...wakePayload,
|
|
comments: [
|
|
{
|
|
...wakePayload.comments[0],
|
|
body: "Ask a NEW environment and verification-label question.",
|
|
},
|
|
],
|
|
},
|
|
interactionResponses: [
|
|
{
|
|
interactionId,
|
|
kind: "ask_user_questions",
|
|
response: {
|
|
status: "answered",
|
|
result: {
|
|
version: 1,
|
|
answers: [
|
|
{ questionId: "environment", optionIds: ["maple"] },
|
|
{
|
|
questionId: "label",
|
|
optionIds: [],
|
|
otherText: "amber compass 93",
|
|
},
|
|
],
|
|
summaryMarkdown: mixedModalSummary,
|
|
},
|
|
},
|
|
},
|
|
],
|
|
});
|
|
expect(mixedModalSummary).toContain("Maple");
|
|
expect(mixedModalSummary).toContain("amber compass 93");
|
|
expect(mixedModal.task.prompt.indexOf("Maple")).toBeLessThan(
|
|
mixedModal.task.prompt.indexOf("Ask a NEW environment"),
|
|
);
|
|
expect(mixedModal.task.prompt.indexOf("amber compass 93")).toBeLessThan(
|
|
mixedModal.task.prompt.indexOf("Ask a NEW environment"),
|
|
);
|
|
expect(mixedModal.task.prompt).toContain(
|
|
"only the answered questions listed above are resolved",
|
|
);
|
|
const unbound = buildNativeExecutionInput({
|
|
...args,
|
|
interactionResponses: [
|
|
{
|
|
...args.interactionResponses![0]!,
|
|
interactionId: "10000000-0000-4000-8000-000000000019",
|
|
},
|
|
],
|
|
});
|
|
expect(unbound.task.title).toBe("Old exact-output request");
|
|
expect(unbound.task.prompt).not.toContain(
|
|
"Native external-chat questions",
|
|
);
|
|
const earlierInteractionId = "10000000-0000-4000-8000-000000000020";
|
|
const sequential = buildNativeExecutionInput({
|
|
...args,
|
|
interactionResponses: [
|
|
{
|
|
interactionId: earlierInteractionId,
|
|
kind: "ask_user_questions",
|
|
response: {
|
|
status: "answered",
|
|
result: {
|
|
version: 1,
|
|
answers: [{ questionId: "shape", optionIds: ["circle"] }],
|
|
summaryMarkdown: "Choose a shape: Circle",
|
|
},
|
|
},
|
|
},
|
|
...args.interactionResponses!,
|
|
],
|
|
});
|
|
expect(sequential.task.title).toBe("External chat follow-up");
|
|
expect(sequential.task.prompt).toContain("Circle");
|
|
expect(sequential.task.prompt).toContain("Amber");
|
|
expect(sequential.task.prompt).not.toContain("## Questions that need a user response");
|
|
expect(
|
|
sequential.task.prompt.indexOf("Choose a shape: Circle"),
|
|
).toBeLessThan(sequential.task.prompt.indexOf("Choose a color: Amber"));
|
|
expect(
|
|
sequential.interactionResponses.map(
|
|
(response) => response.interactionId,
|
|
),
|
|
).toEqual([earlierInteractionId, interactionId]);
|
|
expect(wakePayload).not.toHaveProperty("questionResponse");
|
|
},
|
|
);
|
|
it.each([false, true])(
|
|
"uses neutral framing and real question tools for authenticated overflow chat (resumed: %s)",
|
|
(resumedSession) => {
|
|
const staleRootTitle = "Reply with exactly STALE-OVERFLOW-MARKER";
|
|
const input = buildNativeExecutionInput({
|
|
companyId: "10000000-0000-4000-8000-000000000001",
|
|
runId: "50000000-0000-4000-8000-000000000005",
|
|
issue: {
|
|
id: "20000000-0000-4000-8000-000000000002",
|
|
identifier: "CHAT-5",
|
|
title: staleRootTitle,
|
|
description: "Started from Discord.",
|
|
workMode: "standard",
|
|
},
|
|
taskPrompt: `Paperclip task context:\n- Title: ${JSON.stringify(staleRootTitle)}`,
|
|
resumedSession,
|
|
wakePayload: {
|
|
reason: "External chat message received",
|
|
externalChatProvider: "discord",
|
|
checkedOutByHarness: true,
|
|
issue: {
|
|
id: "20000000-0000-4000-8000-000000000002",
|
|
identifier: "CHAT-5",
|
|
title: staleRootTitle,
|
|
description: "Started from Discord.",
|
|
descriptionTruncated: false,
|
|
status: "in_progress",
|
|
workMode: "standard",
|
|
},
|
|
commentWindow: {
|
|
requestedCount: 2,
|
|
includedCount: 1,
|
|
missingCount: 1,
|
|
},
|
|
commentIds: ["comment-overflow-1", "comment-overflow-2"],
|
|
latestCommentId: "comment-overflow-2",
|
|
comments: [
|
|
{
|
|
id: "comment-overflow-2",
|
|
issueId: "20000000-0000-4000-8000-000000000002",
|
|
body: "Answer both queued messages.",
|
|
bodyTruncated: false,
|
|
authorType: "user",
|
|
},
|
|
],
|
|
fallbackFetchNeeded: true,
|
|
},
|
|
agentId: "30000000-0000-4000-8000-000000000003",
|
|
workspace: {
|
|
id: "50000000-0000-4000-8000-000000000005",
|
|
cwd: "/workspace",
|
|
repoUrl: null,
|
|
repoRef: null,
|
|
branchName: null,
|
|
},
|
|
normalizedSessionId: "60000000-0000-4000-8000-000000000006",
|
|
provider: "codex",
|
|
completionContract: {
|
|
id: "70000000-0000-4000-8000-000000000007",
|
|
sha256: `sha256:${"a".repeat(64)}`,
|
|
schemaVersion: "paperclip.run-result.v1",
|
|
contract: {
|
|
revision: "1",
|
|
objective: "Respond to all pending comments in order",
|
|
criteria: [
|
|
{
|
|
id: "objective",
|
|
requirement: "Read every current wake comment.",
|
|
},
|
|
],
|
|
},
|
|
},
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
});
|
|
|
|
expect(input.task.title).toBe("External chat follow-up");
|
|
expect(input.task.description).toBeNull();
|
|
expect(input.task.prompt).toContain("read_current_wake_comments");
|
|
expect(input.task.prompt).toContain(staleRootTitle);
|
|
},
|
|
);
|
|
|
|
it.each([false, true])(
|
|
"keeps GitHub's task-only files and service-owned recovery navigation in the closed native input (resumed: %s)",
|
|
(resumedSession) => {
|
|
const args: Parameters<typeof buildNativeExecutionInput>[0] = {
|
|
companyId: "10000000-0000-4000-8000-000000000001",
|
|
runId: "50000000-0000-4000-8000-000000000005",
|
|
issue: {
|
|
id: "20000000-0000-4000-8000-000000000002",
|
|
identifier: "CHAT-5",
|
|
title: "Send a file",
|
|
description: null,
|
|
workMode: "standard",
|
|
},
|
|
taskPrompt: "Answer the current line review.",
|
|
wakePayload: {
|
|
reason: "External chat message received",
|
|
externalChatProvider: "github",
|
|
checkedOutByHarness: true,
|
|
issue: {
|
|
id: "20000000-0000-4000-8000-000000000002",
|
|
workMode: "standard",
|
|
},
|
|
comments: [
|
|
{
|
|
id: "review-comment",
|
|
body: "Make a text file and attach it here.",
|
|
},
|
|
],
|
|
commentIds: ["review-comment"],
|
|
latestCommentId: "review-comment",
|
|
commentWindow: {
|
|
requestedCount: 1,
|
|
includedCount: 1,
|
|
missingCount: 0,
|
|
},
|
|
fallbackFetchNeeded: false,
|
|
},
|
|
resumedSession,
|
|
agentId: "30000000-0000-4000-8000-000000000003",
|
|
workspace: {
|
|
id: "50000000-0000-4000-8000-000000000005",
|
|
cwd: "/workspace",
|
|
repoUrl: null,
|
|
repoRef: null,
|
|
branchName: null,
|
|
},
|
|
normalizedSessionId: "60000000-0000-4000-8000-000000000006",
|
|
provider: "codex",
|
|
completionContract: {
|
|
id: "70000000-0000-4000-8000-000000000007",
|
|
sha256: `sha256:${"a".repeat(64)}`,
|
|
schemaVersion: "paperclip.run-result.v1",
|
|
contract: {
|
|
revision: "1",
|
|
objective: "Prepare a file",
|
|
criteria: [
|
|
{ id: "objective", requirement: "Describe actual delivery." },
|
|
],
|
|
},
|
|
},
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
};
|
|
const input = buildNativeExecutionInput(args);
|
|
expect(input.task.prompt).toContain(
|
|
"This GitHub App connection cannot upload file bytes into comments or review threads",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"do not say it is attached, displayed, downloadable, or available to open in this provider conversation",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"Preparation does not confirm provider delivery",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"Paperclip owns recovery navigation for unavailable GitHub attachments",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"after an accepted response, only when the current source remains authorized and a safe configured Board URL is available",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"A task URL missing from your prompt or tool results is not evidence that no task link can be provided",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"Do not invent a URL or promise that a link will appear",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"Briefly explain the unavailable input and ask the user to attach it directly to this Paperclip task or paste the needed text",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"Never infer the file's contents or substitute an older file",
|
|
);
|
|
|
|
const runtimeContext = nativeRuntimeContextFixture();
|
|
runtimeContext.skills = ["configured", "provider-mentioned"].map((name) => ({
|
|
key: `company/test/${name}`,
|
|
runtimeName: name,
|
|
versionId: null,
|
|
bundle: { ...runtimeContext.instructions.bundle },
|
|
}));
|
|
runtimeContext.aggregateDigest = canonicalNativeRuntimeContextDigest(runtimeContext);
|
|
const skillArgs = {
|
|
...args,
|
|
issue: { ...args.issue, description: "Stale provider message: /provider-mentioned" },
|
|
taskPrompt: `${args.taskPrompt}\nGitHub message: /provider-mentioned`,
|
|
runtimeContext,
|
|
githubInstructionSkillKeys: ["company/test/configured", "company/other/unknown"],
|
|
};
|
|
const skillInput = buildNativeExecutionInput(skillArgs);
|
|
expect(skillInput.task.description).toBe("Configured GitHub instruction skills:\n/configured");
|
|
expect(buildNativeModelEnvelope(skillInput)).toMatchObject({ requestedSkills: ["configured"] });
|
|
expect(buildNativeModelEnvelope(buildNativeExecutionInput({
|
|
...skillArgs, githubInstructionSkillKeys: [],
|
|
}))).toMatchObject({ requestedSkills: [] });
|
|
|
|
const wake = args.wakePayload as Record<string, unknown>;
|
|
for (const patch of [
|
|
{ externalChatProvider: "slack" },
|
|
{ externalChatProvider: "discord" },
|
|
{ externalChatProvider: "telegram" },
|
|
{ externalChatProvider: "microsoft-teams" },
|
|
{ externalChatProvider: null },
|
|
{ externalChatProvider: "GitHub" },
|
|
{ checkedOutByHarness: false },
|
|
]) {
|
|
const unrelated = buildNativeExecutionInput({
|
|
...args,
|
|
wakePayload: { ...wake, ...patch },
|
|
});
|
|
expect(unrelated.task.prompt).not.toContain(
|
|
"Paperclip owns recovery navigation for unavailable GitHub attachments",
|
|
);
|
|
if (["slack", "discord", "telegram", "microsoft-teams"].includes(patch.externalChatProvider ?? "")) {
|
|
const unrelatedSkills = buildNativeExecutionInput({
|
|
...skillArgs, wakePayload: { ...wake, ...patch },
|
|
});
|
|
expect(unrelatedSkills.task.description).toBeNull();
|
|
expect(buildNativeModelEnvelope(unrelatedSkills)).toMatchObject({ requestedSkills: [] });
|
|
}
|
|
}
|
|
},
|
|
);
|
|
it.each([
|
|
{ provider: "codex", resumedSession: false },
|
|
{ provider: "codex", resumedSession: true },
|
|
{ provider: "acpx", resumedSession: false },
|
|
{ provider: "acpx", resumedSession: true },
|
|
{ provider: "opencode", resumedSession: false },
|
|
{ provider: "opencode", resumedSession: true },
|
|
] as const)("keeps question documentation in the tool on $provider (resumed: $resumedSession)", ({ provider, resumedSession }) => {
|
|
const input = buildNativeExecutionInput({
|
|
companyId: "10000000-0000-4000-8000-000000000001",
|
|
runId: "50000000-0000-4000-8000-000000000005",
|
|
agentId: "30000000-0000-4000-8000-000000000003",
|
|
issue: { id: "20000000-0000-4000-8000-000000000002", identifier: "QA-1", title: "Welcome", description: null, workMode: "standard" },
|
|
taskPrompt: "Ask whether the welcome should sound warm or formal before writing it.",
|
|
workspace: { id: "40000000-0000-4000-8000-000000000004", cwd: "/workspace", repoUrl: null, repoRef: null, branchName: null },
|
|
normalizedSessionId: resumedSession ? "60000000-0000-4000-8000-000000000006" : null,
|
|
provider, resumedSession,
|
|
acpxAgent: "claude",
|
|
model: provider === "acpx" ? "claude-sonnet-5" : provider === "opencode" ? "openai/gpt-5.5" : "gpt-5.6-sol",
|
|
completionContract: {
|
|
id: "70000000-0000-4000-8000-000000000007", sha256: `sha256:${"a".repeat(64)}`, schemaVersion: "paperclip.run-result.v1",
|
|
contract: { revision: "1", objective: "Write a welcome after the user's answer", criteria: [{ id: "objective", requirement: "Use the selected tone" }] },
|
|
},
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
});
|
|
expect(input.provider).toMatchObject(provider === "acpx"
|
|
? { kind: "acpx", permissionMode: "approve-all" }
|
|
: provider === "opencode" ? { kind: "opencode", permissionMode: "allow" }
|
|
: { kind: "codex", approvalPolicy: "never" });
|
|
expect(input.task.prompt).not.toContain("## Questions that need a user response");
|
|
expect(input.task.prompt).toContain("Use Paperclip's request_human_input for durable task questions.");
|
|
expect(input.task.prompt).not.toContain("payload.questionSet");
|
|
});
|
|
|
|
});
|
|
|
|
|
|
describe("follow-up context size", () => {
|
|
it("keeps old messages out of resume deltas while retaining scoped human answers", () => {
|
|
const message = (id: string, body: string) => ({
|
|
id, body, authorType: "user", authorId: "board", createdAt: "2026-09-17T00:00:00Z",
|
|
updatedAt: "2026-09-17T00:00:00Z", deleted: false, sourceTrust: null,
|
|
});
|
|
const oldBody = "PREVIOUS_TASK_TEXT ".repeat(1000);
|
|
const newBody = "Actually, save the plan first.";
|
|
const answerText = "No budget. Wait for my approval.";
|
|
const continuation: ExecutionContinuationEnvelope = {
|
|
version: 1, companyId: "company", issueId: "issue", objective: "Welcome",
|
|
trigger: { reason: "issue_commented", interactionId: "answer-id", sourceRunId: null },
|
|
originCommentIds: ["new"], messages: [message("old", oldBody), message("new", newBody)],
|
|
resumeDelta: { baseRunId: "previous-run", messages: [message("new", newBody)] },
|
|
humanResponses: [{ id: "answer-id", kind: "ask_user_questions", status: "answered",
|
|
resolvedByUserId: "board", resolvedAt: "2026-09-17T00:01:00Z",
|
|
result: { answers: [{ questionId: "scope", optionIds: [], otherText: answerText }] } }],
|
|
interactionOutcomes: [], completedWork: null, unresolvedInteractionIds: [],
|
|
coverage: { kind: "full_task_history", throughCommentId: "new", summaryThroughCommentId: null },
|
|
};
|
|
const wake = { executionContinuation: continuation };
|
|
const fresh = renderPaperclipWakePrompt(wake);
|
|
const resumed = renderPaperclipWakePrompt(wake, { resumedSession: true });
|
|
const contract = buildNativeCompletionContract({ title: "Welcome", description: oldBody }, {
|
|
immediateRequest: newBody, humanResponseId: "answer-id",
|
|
});
|
|
expect(fresh).toContain(oldBody);
|
|
expect(resumed).not.toContain("PREVIOUS_TASK_TEXT");
|
|
expect(resumed.split(newBody)).toHaveLength(2);
|
|
expect(resumed.split(answerText)).toHaveLength(2);
|
|
expect(resumed).toContain("earlier history remains in this session");
|
|
expect(JSON.stringify(contract)).not.toContain(oldBody);
|
|
expect(JSON.stringify(contract)).not.toContain(answerText);
|
|
expect(resumed.length + JSON.stringify(contract).length).toBeLessThan(fresh.length);
|
|
});
|
|
});
|
|
|
|
|
|
describe("native completion references", () => {
|
|
const source = nativeCompletionSource("description", "task", "Repeat. Repeat.");
|
|
const args: Parameters<typeof buildNativeExecutionInput>[0] = {
|
|
companyId: "company", runId: "run", agentId: "agent",
|
|
issue: { id: "task", identifier: "PAP-1", title: "Brief", description: "Repeat. Repeat.", workMode: "standard" },
|
|
taskPrompt: "Issue description:\nRepeat. Repeat.",
|
|
workspace: { id: "workspace", cwd: "/workspace", repoUrl: null, repoRef: null, branchName: null },
|
|
normalizedSessionId: null,
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
completionContract: {
|
|
id: "contract", sha256: "a".repeat(64), schemaVersion: "paperclip.completion-contract.v1",
|
|
contract: { revision: "1", objective: "Brief", criteria: [{ id: "objective", requirement: "Repeat. Repeat." }] },
|
|
sources: [{ id: "objective", source }],
|
|
},
|
|
};
|
|
it("binds only source identities and revisions attested by the task renderer", () => {
|
|
const input = buildNativeExecutionInput({ ...args, turnContext: {
|
|
version: 1,
|
|
assignment: { owner: "task_markdown", description: { id: source.id, revision: source.revision } },
|
|
events: { owner: "wake_prompt", comments: [] },
|
|
} });
|
|
expect(input.completionSources).toMatchObject({ contractRevision: "1", criteria: [{ id: "objective", source }] });
|
|
expect(input.completionContract.contract).toEqual(args.completionContract.contract);
|
|
expect(input.task.description).toBe("Repeat. Repeat.");
|
|
});
|
|
it.each([
|
|
undefined,
|
|
{ version: 1, assignment: { owner: "task_markdown", description: { id: "another-task", revision: source.revision } } },
|
|
{ version: 1, assignment: { owner: "task_markdown", description: { id: source.id, revision: "stale" } } },
|
|
])("keeps the full requirement when provenance is absent or stale", (turnContext) => {
|
|
const input = buildNativeExecutionInput({ ...args, turnContext });
|
|
expect(input.completionSources).toBeUndefined();
|
|
expect(input.completionContract.contract).toEqual(args.completionContract.contract);
|
|
});
|
|
});
|
|
|
|
|
|
describe("Cursor mode native execution projection", () => {
|
|
function fixture(acpxSessionMode?: "agent" | "plan" | "ask") {
|
|
return {
|
|
companyId: "company-1", runId: "run-1", agentId: "agent-1",
|
|
issue: { id: "issue-1", identifier: "MODE-1", title: "Review a plan", description: null, workMode: "standard" },
|
|
taskPrompt: "Review the project.",
|
|
workspace: { id: "workspace-1", cwd: "/workspace", repoUrl: null, repoRef: null, branchName: null },
|
|
normalizedSessionId: null, provider: "acpx" as const, acpxAgent: "cursor" as const,
|
|
model: "exact-cursor-model", acpxSessionMode, acpxPermissionMode: "deny-all" as const,
|
|
completionContract: { id: "contract-1", sha256: `sha256:${"a".repeat(64)}`, schemaVersion: "paperclip.run-result.v1", contract: { revision: "1", objective: "Review", criteria: [{ id: "output", requirement: "Review" }] } },
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
};
|
|
}
|
|
it.each([undefined, "agent", "plan", "ask"] as const)("preserves mode %s independently of permissions and task planning", mode => {
|
|
const result = buildNativeExecutionInput(fixture(mode));
|
|
expect(result.provider).toMatchObject({ kind: "acpx", agent: "cursor", mode: mode ?? "agent", permissionMode: "deny-all" });
|
|
expect(result.executionMode).toBe("default");
|
|
expect(result.task.workMode).toBe("standard");
|
|
});
|
|
it("rejects a Cursor mode attached to another harness", () => {
|
|
expect(() => buildNativeExecutionInput({ ...fixture("plan"), acpxAgent: "copilot" })).toThrow("only for Cursor");
|
|
});
|
|
});
|
|
|
|
describe("Pi thinking native execution projection", () => {
|
|
function fixture(piThinkingLevel?: "off" | "low" | "high" | "max") {
|
|
return {
|
|
companyId: "company-1", runId: "run-1", agentId: "agent-1",
|
|
issue: { id: "issue-1", identifier: "MODE-1", title: "Review a plan", description: null, workMode: "standard" },
|
|
taskPrompt: "Review the project.",
|
|
workspace: { id: "workspace-1", cwd: "/workspace", repoUrl: null, repoRef: null, branchName: null },
|
|
normalizedSessionId: null, provider: "acpx" as const, acpxAgent: "pi" as const,
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731", piThinkingLevel, acpxPermissionMode: "deny-all" as const,
|
|
completionContract: { id: "contract-1", sha256: `sha256:${"a".repeat(64)}`, schemaVersion: "paperclip.run-result.v1", contract: { revision: "1", objective: "Review", criteria: [{ id: "output", requirement: "Review" }] } },
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
};
|
|
}
|
|
it.each([undefined, "off", "low", "high", "max"] as const)("preserves mode %s independently of permissions and task planning", mode => {
|
|
const result = buildNativeExecutionInput(fixture(mode));
|
|
expect(result.provider).toMatchObject({ kind: "acpx", agent: "pi", piThinkingLevel: mode ?? "low", permissionMode: "deny-all" });
|
|
expect(result.executionMode).toBe("default");
|
|
expect(result.task.workMode).toBe("standard");
|
|
});
|
|
it("rejects a Pi thinking level attached to another harness", () => {
|
|
expect(() => buildNativeExecutionInput({ ...fixture("low"), acpxAgent: "copilot" })).toThrow("only for Pi");
|
|
});
|
|
});
|
|
|
|
|
|
describe("Claude effort and master home-mode compatibility", () => {
|
|
it.each([
|
|
["isolated", undefined, "v5"], ["preconfigured", undefined, "v7"],
|
|
["isolated", "high", "v9"], ["preconfigured", "high", "v9"],
|
|
] as const)("retains %s home mode with effort %s in %s", (homeMode, claudeEffort, version) => {
|
|
const value = buildNativeExecutionInput({
|
|
companyId: "company", runId: "run", agentId: "agent",
|
|
issue: { id: "issue", identifier: "CLA-1", title: "Useful output", description: null, workMode: "standard" },
|
|
taskPrompt: "Create the requested file.",
|
|
workspace: { id: "workspace", cwd: "/workspace", repoUrl: null, repoRef: null, branchName: null },
|
|
normalizedSessionId: null, provider: "acpx", acpxAgent: "claude", model: "claude-opus-5-5",
|
|
acpxPermissionMode: "approve-all", homeMode, claudeEffort,
|
|
completionContract: { id: "contract", sha256: `sha256:${"a".repeat(64)}`, schemaVersion: "paperclip.run-result.v1",
|
|
contract: { revision: "1", objective: "Create a file", criteria: [{ id: "output", requirement: "Create the file" }] } },
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
});
|
|
expect(value.schema).toBe(`paperclip.native-execution-input.${version}`);
|
|
if (version !== "v5") expect(value).toHaveProperty("homeMode", homeMode);
|
|
if (claudeEffort) expect(value.provider).toHaveProperty("claudeEffort", claudeEffort);
|
|
expect(JSON.stringify(buildNativeModelEnvelope(value))).not.toContain("claudeEffort");
|
|
});
|
|
});
|