mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-23 00:53:38 +02:00
## Thinking Path > - Paperclip lets users choose an agent harness and account. > - The server resolves the runner for new agents. > - Claude still needs the same ordinary native-default journey as Codex. > - Existing agents must retain their runner, model and account. > - This PR defaults new Apple Silicon Claude agents to Paperclip Runner with Opus 5.5. > - Users can connect and work without a runner override. ## Linked Issues or Issue Description Refs #15683. This PR depends on #15994 and precedes #15997. Related scope: #15422, #13978 and #15224. ## What Changed - Resolve new Apple Silicon Claude agents to `paperclip_runner`, `provider: acpx`, `acpxAgent: claude`. Keep explicit Legacy, models, provider routing and saved execution. - Use `claude-opus-5-5` for ordinary Claude defaults and retain explicit overrides. - Reuse existing API-key/subscription connections, credential storage and checks. Generalize the shared Advanced runner control with Claude branding. - Carry configured effort into the shared v9 contract. Preserve compatible permissions, instructions, skills, timeouts and session behavior. Require Legacy for unsupported settings. - Qualify a new immutable managed Claude target at explicit Connect through the existing permission-gated probe. Use fresh scoped discovery for test and hire. Discard stale results and require an explicit Connect after settings change. - Retain merged Codex onboarding recovery, auth preparation, import/effort corrections and Grok Bot gates. - Check the Claude first-task wizard's saved default directly. Update configuration/runtime documentation and exact Opus 5.5 accounting. - Align hire fixtures with the merged host-environment restriction. Native agent-authenticated hires cannot inject host env settings; both creation endpoints verify that no extra agent is saved. ## Verification - Updated onto master `9179d8dcc3e908c217fde4f63c9d08ccf8132eeb` through #15994. Native input v9 carries effort while preserving v7 home modes and v8 Muse. Codex defaults remain unchanged. - Focused local verification passed 285 server tests, 323 UI tests, 124 setup/registry tests, and two v9 permission-pin transport cases. Direct server/UI TypeScript checks passed. An initial 30-second hook timeout was resolved by serial verification with a longer hook budget: all 122 credential/permission cases and 232 snapshot/environment cases pass. Direct server TypeScript checking also passes. - Ordinary web creation with the experiment disabled saved native Claude, Opus 5.5 and High effort without runner/model overrides. The actual product task and follow-up both succeeded on integrated source `b2c9fe43f9cd3bc5dd9d22552da0148845ba71a4` with the cloud-built daemon from `04e0202ca81387b17cbe7fd094a8338648356fe9`. The final master reconciliation also changes shared runtime source. Historical proof retains its source identity; a final-source cloud qualification is running before refreshing the preview daemon. - Both actual browser downloads were checked: v1 has three sections/six checkboxes (451 bytes); v2 preserves the original body and adds two Rollback checks (612 bytes). The original attachment stayed byte-identical. The follow-up records session resume and the same provider session. Total estimated API usage: $0.518432800, with no pending/unpriced usage in the successful fixture. - Fixed the live-discovered setup incompatibility: native API-key validation no longer depends on the global Legacy Claude CLI. The existing provider endpoint verifies the key without a paid prompt; subscription/custom routing probes remain intact. - Fixed security review's native-hire credential redirection: agent-supplied URLs, proxy/launch settings and other host environment controls are rejected before inheritance. Native credential references remain supported and partial edits use the saved harness. Added create/hire/update regression coverage. - Current-head cloud CI is green. All three stack PRs have fresh Greptile 5/5 reviews and zero unresolved review threads. Required typecheck, test and build gates ran in cloud CI; Docker work stayed remote. Earlier failed checks remain in their original runs. - Docker and native release builds ran in cloud infrastructure. An inadvertent local Rust prerequisite was stopped; no local Docker build was run. - The final master merge preserves task-loading, permission, skill-retention and OpenRouter changes. Pi uses a fresh combined profile 27, preserving both historical profile-24 declarations; Copilot remains profile 22. Focused protocol checks passed 155 cases (one skipped), the updated Pi attestation check passed 18 (one skipped), and 91 evaluation/catalog checks passed. - Latest-application manual continuation passed in 37.942 seconds on `695e5c005a85bddc8711d3fcddcaf6b7e1fb64d0`, with High effort, native Opus 5.5, Anthropic API attribution, the same resumed Claude session, an independently checked 635-byte file and persisted Done status. It uses the earlier verified Mac daemon; the preceding-source package qualification passed at `dca62c7aeb4b45f0dd91a8e18d386a16a2658ef8`. That package run predates the small TypeScript startup fix and is not relabeled as qualification for it. - Upgrade limitation: changed compiled sidecar identity rejected old durable launch state and recovery hit session-open bounds before eventually resuming. That attempt was operator-cancelled and retained; the successful manual continuation does not qualify seamless upgrades. - Local latency profile, Opus 5.5/High unchanged: three alternating real-ACP loopback samples per variant reduce median provider-ready time from 5.081s to 4.034s. The matched real API task startup improves from 5.533s to 4.348s, but total reply time changes from 17.933s to 20.221s because agent work varies. This establishes a startup saving of about one second, not a consistent total-reply speedup. A real candidate follow-up passed in 20.146s and resumed the same saved session. The loopback fixture makes no external model requests; failed initial probes remain retained. - The startup change passes 107 focused runtime/effort/lease tests, 53 profile checks (one skipped), TypeScript checking and the TypeScript-only runtime build. Native hire fixture checks pass 70 tests; their first local run retained a socket-hangup failure and stale OpenCode expectation, both accounted for before the passing rerun. Broader required checks run in cloud CI. ## Risks - Missing packages, authentication rejection, unsupported settings and unavailable models must fail explicitly. Existing agents retain saved execution; explicit runner changes use existing revision/session invalidation. - Subscription follow-up remains unverified; remaining live acceptance uses API keys per human steering. Historical accounting holds and the separate $50 ceiling remain intact. - The subscription acceptance limit remains explicit for review. No Claude database migration is introduced. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution and browser testing. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] Focused local tests and final cloud verification pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
691 lines
31 KiB
TypeScript
691 lines
31 KiB
TypeScript
import { describe, expect, it } from "vitest";
|
|
import type { ExecutionContinuationEnvelope, AskUserQuestionsInteraction } from "@paperclipai/shared";
|
|
|
|
import { formatDurableQuestionResponseSummary } from "../question-response-delivery.js";
|
|
import { nativeCompletionSource, buildNativeCompletionContract } from "./completion-contracts.js";
|
|
import { renderPaperclipWakePrompt } from "@paperclipai/adapter-utils/server-utils";
|
|
import { buildNativeExecutionInput } from "./native-execution-input.js";
|
|
import { nativeRuntimeContextFixture } from "./runtime-context.test-fixture.js";
|
|
import { buildNativeModelEnvelope, canonicalNativeRuntimeContextDigest } from "../../vendor/paperclip-runner/index.js";
|
|
|
|
describe("LCA-05 explicit native work mode", () => {
|
|
it.each(["standard", "planning", "ask"])("title and description cannot override %s mode", (workMode) => {
|
|
for (const text of ["Inspect files", "Making a plan", "Create a report", "Research proposal", "Implement now; no plan needed"]) {
|
|
const input = buildNativeExecutionInput({
|
|
companyId: "10000000-0000-4000-8000-000000000001",
|
|
runId: "50000000-0000-4000-8000-000000000005",
|
|
agentId: "30000000-0000-4000-8000-000000000003",
|
|
issue: { id: "20000000-0000-4000-8000-000000000002", identifier: "MODE-1", title: text, description: text, workMode },
|
|
taskPrompt: text,
|
|
workspace: { id: "50000000-0000-4000-8000-000000000005", cwd: "/workspace", repoUrl: null, repoRef: null, branchName: null },
|
|
normalizedSessionId: null,
|
|
provider: "codex",
|
|
codexReasoningEffort: "high",
|
|
planningContext: workMode === "planning" ? { documentId: null, baseRevisionId: null, baseRevisionNumber: 0, markdown: "", sha256: "a".repeat(64), reviewContext: {} } : null,
|
|
completionContract: { id: "70000000-0000-4000-8000-000000000007", sha256: `sha256:${"a".repeat(64)}`, schemaVersion: "paperclip.run-result.v1", contract: { revision: "1", objective: "Deliver the requested work", criteria: [{ id: "output", requirement: "Deliver the requested work" }] } },
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
});
|
|
expect(input.task.workMode).toBe(workMode);
|
|
expect(input.provider).toMatchObject({ kind: "codex", reasoningEffort: "high" });
|
|
expect(input.executionMode).toBe(workMode === "planning" ? "plan" : "default");
|
|
expect(input.task.title).toBe(text);
|
|
}
|
|
});
|
|
});
|
|
|
|
describe("native execution input external-chat framing", () => {
|
|
it.each([false, true])(
|
|
"projects the authoritative selected answer into an attested native chat prompt (resumed: %s)",
|
|
(resumedSession) => {
|
|
const interactionId = "80000000-0000-4000-8000-000000000008";
|
|
const sourceRunId = "90000000-0000-4000-8000-000000000009";
|
|
const wakePayload = {
|
|
reason: "issue_commented",
|
|
externalChatProvider: "telegram",
|
|
externalChatExecutionBound: true,
|
|
interactionId,
|
|
sourceRunId,
|
|
interactionKind: "ask_user_questions",
|
|
interactionStatus: "answered",
|
|
externalInteractionContinuation: true,
|
|
issue: {
|
|
id: "20000000-0000-4000-8000-000000000002",
|
|
workMode: "standard",
|
|
status: "in_review",
|
|
},
|
|
externalChatQuestionResponse: {
|
|
schema: "paperclip.external_chat_question_response.v1",
|
|
interactionId,
|
|
sourceRunId,
|
|
responseDeliveryId: "10000000-0000-4000-8000-000000000011",
|
|
sourceCommentId: "10000000-0000-4000-8000-000000000012",
|
|
endpointId: "10000000-0000-4000-8000-000000000013",
|
|
conversationId: "10000000-0000-4000-8000-000000000014",
|
|
bindingSha256: "a".repeat(64),
|
|
},
|
|
commentIds: ["10000000-0000-4000-8000-000000000012"],
|
|
latestCommentId: "10000000-0000-4000-8000-000000000012",
|
|
comments: [
|
|
{
|
|
id: "10000000-0000-4000-8000-000000000012",
|
|
body: "Ask for a color, then tell me the selected color.",
|
|
},
|
|
],
|
|
commentWindow: { requestedCount: 1, includedCount: 1, missingCount: 0 },
|
|
fallbackFetchNeeded: false,
|
|
};
|
|
const args: Parameters<typeof buildNativeExecutionInput>[0] = {
|
|
companyId: "10000000-0000-4000-8000-000000000001",
|
|
runId: "50000000-0000-4000-8000-000000000005",
|
|
agentId: "30000000-0000-4000-8000-000000000003",
|
|
issue: {
|
|
id: "20000000-0000-4000-8000-000000000002",
|
|
identifier: "CHAT-5",
|
|
title: "Old exact-output request",
|
|
description: null,
|
|
workMode: "standard",
|
|
},
|
|
taskPrompt:
|
|
"Continue the user's original request with their selected answer.",
|
|
initialCommunicationGuidance: "Initial Slack communication preferences.",
|
|
wakePayload,
|
|
resumedSession,
|
|
workspace: {
|
|
id: "50000000-0000-4000-8000-000000000005",
|
|
cwd: "/workspace",
|
|
repoUrl: null,
|
|
repoRef: null,
|
|
branchName: null,
|
|
},
|
|
normalizedSessionId: "60000000-0000-4000-8000-000000000006",
|
|
provider: "codex",
|
|
completionContract: {
|
|
id: "70000000-0000-4000-8000-000000000007",
|
|
sha256: `sha256:${"a".repeat(64)}`,
|
|
schemaVersion: "paperclip.run-result.v1",
|
|
contract: {
|
|
revision: "1",
|
|
objective: "Answer the user's selected choice",
|
|
criteria: [
|
|
{ id: "answer", requirement: "Return the actual color." },
|
|
],
|
|
},
|
|
},
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
interactionResponses: [
|
|
{
|
|
interactionId,
|
|
kind: "ask_user_questions",
|
|
response: {
|
|
status: "answered",
|
|
result: {
|
|
version: 1,
|
|
answers: [{ questionId: "color", optionIds: ["amber"] }],
|
|
summaryMarkdown:
|
|
"Resolved questions and answers:\n- Choose a color: Amber",
|
|
},
|
|
},
|
|
},
|
|
],
|
|
};
|
|
const input = buildNativeExecutionInput(args);
|
|
expect(input.initialCommunicationGuidance).toBe("Initial Slack communication preferences.");
|
|
// The runtime adds this only after deciding whether provider recovery succeeded.
|
|
expect(input.task.prompt).not.toContain("Initial Slack communication preferences.");
|
|
expect(input.task.title).toBe("External chat follow-up");
|
|
expect(input.task.prompt).toContain("Amber");
|
|
expect(input.task.prompt).not.toContain("## Questions that need a user response");
|
|
expect(input.task.prompt).toContain(
|
|
"semantic completion summary is the user-visible final answer",
|
|
);
|
|
expect(input.task.prompt).not.toContain(
|
|
"including marking the task done",
|
|
);
|
|
expect(input.task.prompt.indexOf("Choose a color: Amber")).toBeLessThan(
|
|
input.task.prompt.indexOf(
|
|
"Ask for a color, then tell me the selected color.",
|
|
),
|
|
);
|
|
expect(wakePayload).not.toHaveProperty("questionResponse");
|
|
|
|
const mixedModalSummary = formatDurableQuestionResponseSummary({
|
|
id: interactionId,
|
|
kind: "ask_user_questions",
|
|
status: "answered",
|
|
sourceRunId,
|
|
payload: {
|
|
version: 1,
|
|
questions: [
|
|
{
|
|
id: "environment",
|
|
prompt: "Choose an environment",
|
|
selectionMode: "single",
|
|
options: [
|
|
{ id: "maple", label: "Maple" },
|
|
{ id: "cedar", label: "Cedar" },
|
|
],
|
|
},
|
|
{
|
|
id: "label",
|
|
prompt: "Enter the verification label",
|
|
selectionMode: "single",
|
|
options: [
|
|
{
|
|
id: "custom",
|
|
label: "Custom label",
|
|
freeText: true,
|
|
},
|
|
],
|
|
},
|
|
],
|
|
},
|
|
result: {
|
|
version: 1,
|
|
answers: [
|
|
{ questionId: "environment", optionIds: ["maple"] },
|
|
{
|
|
questionId: "label",
|
|
optionIds: [],
|
|
otherText: "amber compass 93",
|
|
},
|
|
],
|
|
summaryMarkdown: null,
|
|
},
|
|
} as unknown as AskUserQuestionsInteraction);
|
|
const mixedModal = buildNativeExecutionInput({
|
|
...args,
|
|
wakePayload: {
|
|
...wakePayload,
|
|
comments: [
|
|
{
|
|
...wakePayload.comments[0],
|
|
body: "Ask a NEW environment and verification-label question.",
|
|
},
|
|
],
|
|
},
|
|
interactionResponses: [
|
|
{
|
|
interactionId,
|
|
kind: "ask_user_questions",
|
|
response: {
|
|
status: "answered",
|
|
result: {
|
|
version: 1,
|
|
answers: [
|
|
{ questionId: "environment", optionIds: ["maple"] },
|
|
{
|
|
questionId: "label",
|
|
optionIds: [],
|
|
otherText: "amber compass 93",
|
|
},
|
|
],
|
|
summaryMarkdown: mixedModalSummary,
|
|
},
|
|
},
|
|
},
|
|
],
|
|
});
|
|
expect(mixedModalSummary).toContain("Maple");
|
|
expect(mixedModalSummary).toContain("amber compass 93");
|
|
expect(mixedModal.task.prompt.indexOf("Maple")).toBeLessThan(
|
|
mixedModal.task.prompt.indexOf("Ask a NEW environment"),
|
|
);
|
|
expect(mixedModal.task.prompt.indexOf("amber compass 93")).toBeLessThan(
|
|
mixedModal.task.prompt.indexOf("Ask a NEW environment"),
|
|
);
|
|
expect(mixedModal.task.prompt).toContain(
|
|
"only the answered questions listed above are resolved",
|
|
);
|
|
const unbound = buildNativeExecutionInput({
|
|
...args,
|
|
interactionResponses: [
|
|
{
|
|
...args.interactionResponses![0]!,
|
|
interactionId: "10000000-0000-4000-8000-000000000019",
|
|
},
|
|
],
|
|
});
|
|
expect(unbound.task.title).toBe("Old exact-output request");
|
|
expect(unbound.task.prompt).not.toContain(
|
|
"Native external-chat questions",
|
|
);
|
|
const earlierInteractionId = "10000000-0000-4000-8000-000000000020";
|
|
const sequential = buildNativeExecutionInput({
|
|
...args,
|
|
interactionResponses: [
|
|
{
|
|
interactionId: earlierInteractionId,
|
|
kind: "ask_user_questions",
|
|
response: {
|
|
status: "answered",
|
|
result: {
|
|
version: 1,
|
|
answers: [{ questionId: "shape", optionIds: ["circle"] }],
|
|
summaryMarkdown: "Choose a shape: Circle",
|
|
},
|
|
},
|
|
},
|
|
...args.interactionResponses!,
|
|
],
|
|
});
|
|
expect(sequential.task.title).toBe("External chat follow-up");
|
|
expect(sequential.task.prompt).toContain("Circle");
|
|
expect(sequential.task.prompt).toContain("Amber");
|
|
expect(sequential.task.prompt).not.toContain("## Questions that need a user response");
|
|
expect(
|
|
sequential.task.prompt.indexOf("Choose a shape: Circle"),
|
|
).toBeLessThan(sequential.task.prompt.indexOf("Choose a color: Amber"));
|
|
expect(
|
|
sequential.interactionResponses.map(
|
|
(response) => response.interactionId,
|
|
),
|
|
).toEqual([earlierInteractionId, interactionId]);
|
|
expect(wakePayload).not.toHaveProperty("questionResponse");
|
|
},
|
|
);
|
|
it.each([false, true])(
|
|
"uses neutral framing and real question tools for authenticated overflow chat (resumed: %s)",
|
|
(resumedSession) => {
|
|
const staleRootTitle = "Reply with exactly STALE-OVERFLOW-MARKER";
|
|
const input = buildNativeExecutionInput({
|
|
companyId: "10000000-0000-4000-8000-000000000001",
|
|
runId: "50000000-0000-4000-8000-000000000005",
|
|
issue: {
|
|
id: "20000000-0000-4000-8000-000000000002",
|
|
identifier: "CHAT-5",
|
|
title: staleRootTitle,
|
|
description: "Started from Discord.",
|
|
workMode: "standard",
|
|
},
|
|
taskPrompt: `Paperclip task context:\n- Title: ${JSON.stringify(staleRootTitle)}`,
|
|
resumedSession,
|
|
wakePayload: {
|
|
reason: "External chat message received",
|
|
externalChatProvider: "discord",
|
|
checkedOutByHarness: true,
|
|
issue: {
|
|
id: "20000000-0000-4000-8000-000000000002",
|
|
identifier: "CHAT-5",
|
|
title: staleRootTitle,
|
|
description: "Started from Discord.",
|
|
descriptionTruncated: false,
|
|
status: "in_progress",
|
|
workMode: "standard",
|
|
},
|
|
commentWindow: {
|
|
requestedCount: 2,
|
|
includedCount: 1,
|
|
missingCount: 1,
|
|
},
|
|
commentIds: ["comment-overflow-1", "comment-overflow-2"],
|
|
latestCommentId: "comment-overflow-2",
|
|
comments: [
|
|
{
|
|
id: "comment-overflow-2",
|
|
issueId: "20000000-0000-4000-8000-000000000002",
|
|
body: "Answer both queued messages.",
|
|
bodyTruncated: false,
|
|
authorType: "user",
|
|
},
|
|
],
|
|
fallbackFetchNeeded: true,
|
|
},
|
|
agentId: "30000000-0000-4000-8000-000000000003",
|
|
workspace: {
|
|
id: "50000000-0000-4000-8000-000000000005",
|
|
cwd: "/workspace",
|
|
repoUrl: null,
|
|
repoRef: null,
|
|
branchName: null,
|
|
},
|
|
normalizedSessionId: "60000000-0000-4000-8000-000000000006",
|
|
provider: "codex",
|
|
completionContract: {
|
|
id: "70000000-0000-4000-8000-000000000007",
|
|
sha256: `sha256:${"a".repeat(64)}`,
|
|
schemaVersion: "paperclip.run-result.v1",
|
|
contract: {
|
|
revision: "1",
|
|
objective: "Respond to all pending comments in order",
|
|
criteria: [
|
|
{
|
|
id: "objective",
|
|
requirement: "Read every current wake comment.",
|
|
},
|
|
],
|
|
},
|
|
},
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
});
|
|
|
|
expect(input.task.title).toBe("External chat follow-up");
|
|
expect(input.task.description).toBeNull();
|
|
expect(input.task.prompt).toContain("read_current_wake_comments");
|
|
expect(input.task.prompt).toContain(staleRootTitle);
|
|
},
|
|
);
|
|
|
|
it.each([false, true])(
|
|
"keeps GitHub's task-only files and service-owned recovery navigation in the closed native input (resumed: %s)",
|
|
(resumedSession) => {
|
|
const args: Parameters<typeof buildNativeExecutionInput>[0] = {
|
|
companyId: "10000000-0000-4000-8000-000000000001",
|
|
runId: "50000000-0000-4000-8000-000000000005",
|
|
issue: {
|
|
id: "20000000-0000-4000-8000-000000000002",
|
|
identifier: "CHAT-5",
|
|
title: "Send a file",
|
|
description: null,
|
|
workMode: "standard",
|
|
},
|
|
taskPrompt: "Answer the current line review.",
|
|
wakePayload: {
|
|
reason: "External chat message received",
|
|
externalChatProvider: "github",
|
|
checkedOutByHarness: true,
|
|
issue: {
|
|
id: "20000000-0000-4000-8000-000000000002",
|
|
workMode: "standard",
|
|
},
|
|
comments: [
|
|
{
|
|
id: "review-comment",
|
|
body: "Make a text file and attach it here.",
|
|
},
|
|
],
|
|
commentIds: ["review-comment"],
|
|
latestCommentId: "review-comment",
|
|
commentWindow: {
|
|
requestedCount: 1,
|
|
includedCount: 1,
|
|
missingCount: 0,
|
|
},
|
|
fallbackFetchNeeded: false,
|
|
},
|
|
resumedSession,
|
|
agentId: "30000000-0000-4000-8000-000000000003",
|
|
workspace: {
|
|
id: "50000000-0000-4000-8000-000000000005",
|
|
cwd: "/workspace",
|
|
repoUrl: null,
|
|
repoRef: null,
|
|
branchName: null,
|
|
},
|
|
normalizedSessionId: "60000000-0000-4000-8000-000000000006",
|
|
provider: "codex",
|
|
completionContract: {
|
|
id: "70000000-0000-4000-8000-000000000007",
|
|
sha256: `sha256:${"a".repeat(64)}`,
|
|
schemaVersion: "paperclip.run-result.v1",
|
|
contract: {
|
|
revision: "1",
|
|
objective: "Prepare a file",
|
|
criteria: [
|
|
{ id: "objective", requirement: "Describe actual delivery." },
|
|
],
|
|
},
|
|
},
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
};
|
|
const input = buildNativeExecutionInput(args);
|
|
expect(input.task.prompt).toContain(
|
|
"This GitHub App connection cannot upload file bytes into comments or review threads",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"do not say it is attached, displayed, downloadable, or available to open in this provider conversation",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"Preparation does not confirm provider delivery",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"Paperclip owns recovery navigation for unavailable GitHub attachments",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"after an accepted response, only when the current source remains authorized and a safe configured Board URL is available",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"A task URL missing from your prompt or tool results is not evidence that no task link can be provided",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"Do not invent a URL or promise that a link will appear",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"Briefly explain the unavailable input and ask the user to attach it directly to this Paperclip task or paste the needed text",
|
|
);
|
|
expect(input.task.prompt).toContain(
|
|
"Never infer the file's contents or substitute an older file",
|
|
);
|
|
|
|
const runtimeContext = nativeRuntimeContextFixture();
|
|
runtimeContext.skills = ["configured", "provider-mentioned"].map((name) => ({
|
|
key: `company/test/${name}`,
|
|
runtimeName: name,
|
|
versionId: null,
|
|
bundle: { ...runtimeContext.instructions.bundle },
|
|
}));
|
|
runtimeContext.aggregateDigest = canonicalNativeRuntimeContextDigest(runtimeContext);
|
|
const skillArgs = {
|
|
...args,
|
|
issue: { ...args.issue, description: "Stale provider message: /provider-mentioned" },
|
|
taskPrompt: `${args.taskPrompt}\nGitHub message: /provider-mentioned`,
|
|
runtimeContext,
|
|
githubInstructionSkillKeys: ["company/test/configured", "company/other/unknown"],
|
|
};
|
|
const skillInput = buildNativeExecutionInput(skillArgs);
|
|
expect(skillInput.task.description).toBe("Configured GitHub instruction skills:\n/configured");
|
|
expect(buildNativeModelEnvelope(skillInput)).toMatchObject({ requestedSkills: ["configured"] });
|
|
expect(buildNativeModelEnvelope(buildNativeExecutionInput({
|
|
...skillArgs, githubInstructionSkillKeys: [],
|
|
}))).toMatchObject({ requestedSkills: [] });
|
|
|
|
const wake = args.wakePayload as Record<string, unknown>;
|
|
for (const patch of [
|
|
{ externalChatProvider: "slack" },
|
|
{ externalChatProvider: "discord" },
|
|
{ externalChatProvider: "telegram" },
|
|
{ externalChatProvider: "microsoft-teams" },
|
|
{ externalChatProvider: null },
|
|
{ externalChatProvider: "GitHub" },
|
|
{ checkedOutByHarness: false },
|
|
]) {
|
|
const unrelated = buildNativeExecutionInput({
|
|
...args,
|
|
wakePayload: { ...wake, ...patch },
|
|
});
|
|
expect(unrelated.task.prompt).not.toContain(
|
|
"Paperclip owns recovery navigation for unavailable GitHub attachments",
|
|
);
|
|
if (["slack", "discord", "telegram", "microsoft-teams"].includes(patch.externalChatProvider ?? "")) {
|
|
const unrelatedSkills = buildNativeExecutionInput({
|
|
...skillArgs, wakePayload: { ...wake, ...patch },
|
|
});
|
|
expect(unrelatedSkills.task.description).toBeNull();
|
|
expect(buildNativeModelEnvelope(unrelatedSkills)).toMatchObject({ requestedSkills: [] });
|
|
}
|
|
}
|
|
},
|
|
);
|
|
it.each([
|
|
{ provider: "codex", resumedSession: false },
|
|
{ provider: "codex", resumedSession: true },
|
|
{ provider: "acpx", resumedSession: false },
|
|
{ provider: "acpx", resumedSession: true },
|
|
{ provider: "opencode", resumedSession: false },
|
|
{ provider: "opencode", resumedSession: true },
|
|
] as const)("keeps question documentation in the tool on $provider (resumed: $resumedSession)", ({ provider, resumedSession }) => {
|
|
const input = buildNativeExecutionInput({
|
|
companyId: "10000000-0000-4000-8000-000000000001",
|
|
runId: "50000000-0000-4000-8000-000000000005",
|
|
agentId: "30000000-0000-4000-8000-000000000003",
|
|
issue: { id: "20000000-0000-4000-8000-000000000002", identifier: "QA-1", title: "Welcome", description: null, workMode: "standard" },
|
|
taskPrompt: "Ask whether the welcome should sound warm or formal before writing it.",
|
|
workspace: { id: "40000000-0000-4000-8000-000000000004", cwd: "/workspace", repoUrl: null, repoRef: null, branchName: null },
|
|
normalizedSessionId: resumedSession ? "60000000-0000-4000-8000-000000000006" : null,
|
|
provider, resumedSession,
|
|
acpxAgent: "claude",
|
|
model: provider === "acpx" ? "claude-sonnet-5" : provider === "opencode" ? "openai/gpt-5.5" : "gpt-5.6-sol",
|
|
completionContract: {
|
|
id: "70000000-0000-4000-8000-000000000007", sha256: `sha256:${"a".repeat(64)}`, schemaVersion: "paperclip.run-result.v1",
|
|
contract: { revision: "1", objective: "Write a welcome after the user's answer", criteria: [{ id: "objective", requirement: "Use the selected tone" }] },
|
|
},
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
});
|
|
expect(input.provider).toMatchObject(provider === "acpx"
|
|
? { kind: "acpx", permissionMode: "approve-all" }
|
|
: provider === "opencode" ? { kind: "opencode", permissionMode: "allow" }
|
|
: { kind: "codex", approvalPolicy: "never" });
|
|
expect(input.task.prompt).not.toContain("## Questions that need a user response");
|
|
expect(input.task.prompt).toContain("Use Paperclip's request_human_input for durable task questions.");
|
|
expect(input.task.prompt).not.toContain("payload.questionSet");
|
|
});
|
|
|
|
});
|
|
|
|
|
|
describe("follow-up context size", () => {
|
|
it("keeps old messages out of resume deltas while retaining scoped human answers", () => {
|
|
const message = (id: string, body: string) => ({
|
|
id, body, authorType: "user", authorId: "board", createdAt: "2026-09-17T00:00:00Z",
|
|
updatedAt: "2026-09-17T00:00:00Z", deleted: false, sourceTrust: null,
|
|
});
|
|
const oldBody = "PREVIOUS_TASK_TEXT ".repeat(1000);
|
|
const newBody = "Actually, save the plan first.";
|
|
const answerText = "No budget. Wait for my approval.";
|
|
const continuation: ExecutionContinuationEnvelope = {
|
|
version: 1, companyId: "company", issueId: "issue", objective: "Welcome",
|
|
trigger: { reason: "issue_commented", interactionId: "answer-id", sourceRunId: null },
|
|
originCommentIds: ["new"], messages: [message("old", oldBody), message("new", newBody)],
|
|
resumeDelta: { baseRunId: "previous-run", messages: [message("new", newBody)] },
|
|
humanResponses: [{ id: "answer-id", kind: "ask_user_questions", status: "answered",
|
|
resolvedByUserId: "board", resolvedAt: "2026-09-17T00:01:00Z",
|
|
result: { answers: [{ questionId: "scope", optionIds: [], otherText: answerText }] } }],
|
|
interactionOutcomes: [], completedWork: null, unresolvedInteractionIds: [],
|
|
coverage: { kind: "full_task_history", throughCommentId: "new", summaryThroughCommentId: null },
|
|
};
|
|
const wake = { executionContinuation: continuation };
|
|
const fresh = renderPaperclipWakePrompt(wake);
|
|
const resumed = renderPaperclipWakePrompt(wake, { resumedSession: true });
|
|
const contract = buildNativeCompletionContract({ title: "Welcome", description: oldBody }, {
|
|
immediateRequest: newBody, humanResponseId: "answer-id",
|
|
});
|
|
expect(fresh).toContain(oldBody);
|
|
expect(resumed).not.toContain("PREVIOUS_TASK_TEXT");
|
|
expect(resumed.split(newBody)).toHaveLength(2);
|
|
expect(resumed.split(answerText)).toHaveLength(2);
|
|
expect(resumed).toContain("earlier history remains in this session");
|
|
expect(JSON.stringify(contract)).not.toContain(oldBody);
|
|
expect(JSON.stringify(contract)).not.toContain(answerText);
|
|
expect(resumed.length + JSON.stringify(contract).length).toBeLessThan(fresh.length);
|
|
});
|
|
});
|
|
|
|
|
|
describe("native completion references", () => {
|
|
const source = nativeCompletionSource("description", "task", "Repeat. Repeat.");
|
|
const args: Parameters<typeof buildNativeExecutionInput>[0] = {
|
|
companyId: "company", runId: "run", agentId: "agent",
|
|
issue: { id: "task", identifier: "PAP-1", title: "Brief", description: "Repeat. Repeat.", workMode: "standard" },
|
|
taskPrompt: "Issue description:\nRepeat. Repeat.",
|
|
workspace: { id: "workspace", cwd: "/workspace", repoUrl: null, repoRef: null, branchName: null },
|
|
normalizedSessionId: null,
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
completionContract: {
|
|
id: "contract", sha256: "a".repeat(64), schemaVersion: "paperclip.completion-contract.v1",
|
|
contract: { revision: "1", objective: "Brief", criteria: [{ id: "objective", requirement: "Repeat. Repeat." }] },
|
|
sources: [{ id: "objective", source }],
|
|
},
|
|
};
|
|
it("binds only source identities and revisions attested by the task renderer", () => {
|
|
const input = buildNativeExecutionInput({ ...args, turnContext: {
|
|
version: 1,
|
|
assignment: { owner: "task_markdown", description: { id: source.id, revision: source.revision } },
|
|
events: { owner: "wake_prompt", comments: [] },
|
|
} });
|
|
expect(input.completionSources).toMatchObject({ contractRevision: "1", criteria: [{ id: "objective", source }] });
|
|
expect(input.completionContract.contract).toEqual(args.completionContract.contract);
|
|
expect(input.task.description).toBe("Repeat. Repeat.");
|
|
});
|
|
it.each([
|
|
undefined,
|
|
{ version: 1, assignment: { owner: "task_markdown", description: { id: "another-task", revision: source.revision } } },
|
|
{ version: 1, assignment: { owner: "task_markdown", description: { id: source.id, revision: "stale" } } },
|
|
])("keeps the full requirement when provenance is absent or stale", (turnContext) => {
|
|
const input = buildNativeExecutionInput({ ...args, turnContext });
|
|
expect(input.completionSources).toBeUndefined();
|
|
expect(input.completionContract.contract).toEqual(args.completionContract.contract);
|
|
});
|
|
});
|
|
|
|
|
|
describe("Cursor mode native execution projection", () => {
|
|
function fixture(acpxSessionMode?: "agent" | "plan" | "ask") {
|
|
return {
|
|
companyId: "company-1", runId: "run-1", agentId: "agent-1",
|
|
issue: { id: "issue-1", identifier: "MODE-1", title: "Review a plan", description: null, workMode: "standard" },
|
|
taskPrompt: "Review the project.",
|
|
workspace: { id: "workspace-1", cwd: "/workspace", repoUrl: null, repoRef: null, branchName: null },
|
|
normalizedSessionId: null, provider: "acpx" as const, acpxAgent: "cursor" as const,
|
|
model: "exact-cursor-model", acpxSessionMode, acpxPermissionMode: "deny-all" as const,
|
|
completionContract: { id: "contract-1", sha256: `sha256:${"a".repeat(64)}`, schemaVersion: "paperclip.run-result.v1", contract: { revision: "1", objective: "Review", criteria: [{ id: "output", requirement: "Review" }] } },
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
};
|
|
}
|
|
it.each([undefined, "agent", "plan", "ask"] as const)("preserves mode %s independently of permissions and task planning", mode => {
|
|
const result = buildNativeExecutionInput(fixture(mode));
|
|
expect(result.provider).toMatchObject({ kind: "acpx", agent: "cursor", mode: mode ?? "agent", permissionMode: "deny-all" });
|
|
expect(result.executionMode).toBe("default");
|
|
expect(result.task.workMode).toBe("standard");
|
|
});
|
|
it("rejects a Cursor mode attached to another harness", () => {
|
|
expect(() => buildNativeExecutionInput({ ...fixture("plan"), acpxAgent: "copilot" })).toThrow("only for Cursor");
|
|
});
|
|
});
|
|
|
|
describe("Pi thinking native execution projection", () => {
|
|
function fixture(piThinkingLevel?: "off" | "low" | "high" | "max") {
|
|
return {
|
|
companyId: "company-1", runId: "run-1", agentId: "agent-1",
|
|
issue: { id: "issue-1", identifier: "MODE-1", title: "Review a plan", description: null, workMode: "standard" },
|
|
taskPrompt: "Review the project.",
|
|
workspace: { id: "workspace-1", cwd: "/workspace", repoUrl: null, repoRef: null, branchName: null },
|
|
normalizedSessionId: null, provider: "acpx" as const, acpxAgent: "pi" as const,
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731", piThinkingLevel, acpxPermissionMode: "deny-all" as const,
|
|
completionContract: { id: "contract-1", sha256: `sha256:${"a".repeat(64)}`, schemaVersion: "paperclip.run-result.v1", contract: { revision: "1", objective: "Review", criteria: [{ id: "output", requirement: "Review" }] } },
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
};
|
|
}
|
|
it.each([undefined, "off", "low", "high", "max"] as const)("preserves mode %s independently of permissions and task planning", mode => {
|
|
const result = buildNativeExecutionInput(fixture(mode));
|
|
expect(result.provider).toMatchObject({ kind: "acpx", agent: "pi", piThinkingLevel: mode ?? "low", permissionMode: "deny-all" });
|
|
expect(result.executionMode).toBe("default");
|
|
expect(result.task.workMode).toBe("standard");
|
|
});
|
|
it("rejects a Pi thinking level attached to another harness", () => {
|
|
expect(() => buildNativeExecutionInput({ ...fixture("low"), acpxAgent: "copilot" })).toThrow("only for Pi");
|
|
});
|
|
});
|
|
|
|
|
|
describe("Claude effort and master home-mode compatibility", () => {
|
|
it.each([
|
|
["isolated", undefined, "v5"], ["preconfigured", undefined, "v7"],
|
|
["isolated", "high", "v9"], ["preconfigured", "high", "v9"],
|
|
] as const)("retains %s home mode with effort %s in %s", (homeMode, claudeEffort, version) => {
|
|
const value = buildNativeExecutionInput({
|
|
companyId: "company", runId: "run", agentId: "agent",
|
|
issue: { id: "issue", identifier: "CLA-1", title: "Useful output", description: null, workMode: "standard" },
|
|
taskPrompt: "Create the requested file.",
|
|
workspace: { id: "workspace", cwd: "/workspace", repoUrl: null, repoRef: null, branchName: null },
|
|
normalizedSessionId: null, provider: "acpx", acpxAgent: "claude", model: "claude-opus-5-5",
|
|
acpxPermissionMode: "approve-all", homeMode, claudeEffort,
|
|
completionContract: { id: "contract", sha256: `sha256:${"a".repeat(64)}`, schemaVersion: "paperclip.run-result.v1",
|
|
contract: { revision: "1", objective: "Create a file", criteria: [{ id: "output", requirement: "Create the file" }] } },
|
|
runtimeContext: nativeRuntimeContextFixture(),
|
|
});
|
|
expect(value.schema).toBe(`paperclip.native-execution-input.${version}`);
|
|
if (version !== "v5") expect(value).toHaveProperty("homeMode", homeMode);
|
|
if (claudeEffort) expect(value.provider).toHaveProperty("claudeEffort", claudeEffort);
|
|
expect(JSON.stringify(buildNativeModelEnvelope(value))).not.toContain("claudeEffort");
|
|
});
|
|
});
|