fix(interactions): derive question storage from canonical forms (#14946)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents request human input through durable issue interactions.
> - A question form has a canonical presentation and a compatibility
storage format.
> - The creation API required agents to write both formats.
> - Tool guidance told agents to split text and choice questions across
those formats.
> - This pull request accepts one complete canonical form and derives
storage fields on the server.
> - The benefit is a complete question card with stable answer and retry
behavior.

## Linked Issues or Issue Description

Related work: Refs #13630 and #14430. PR #13630 addresses the display of
historical partial forms. This change fixes creation and keeps the check
that rejects conflicting new forms.

**What happened?**

A question save supplied three compatibility questions and one canonical
text question. The API correctly rejected the incomplete canonical form.
The Runner's tool description encouraged this split. Sending only a
complete canonical form also failed because the API required
compatibility questions.

**Expected behavior**

An agent sends one complete `payload.questionSet` with every text and
choice question. Paperclip derives `payload.questions` for storage and
answer compatibility. Existing legacy requests remain valid. Explicitly
conflicting dual forms remain invalid.

**Steps to reproduce**

1. Call `paperclip_request_human_input` with `interactionKind:
"questions"`.
2. Send `payload: { version: 1, questionSet: ... }` with a required text
question and a required choice question.
3. The old API rejects the missing compatibility questions. With this
change, it stores both questions and preserves the canonical form.
4. Retry with the same idempotency key. Confirm that only one
interaction exists.
5. Submit both answers. Confirm that the normal resolver and
continuation rules apply.

**Paperclip version or commit**

The branch is based on `cf8ad63c8`. The problem affects the native
Runner and the interaction creation API.

**Deployment mode**

Server deployment with the native Paperclip Runner. Integration tests
use the real interaction service and an embedded test database.

## What Changed

- Add one shared canonical-to-storage projection. Reuse it for native
harness question requests.
- Accept canonical-only question creation at the shared validator and
server boundary.
- Export the input type and update the plugin SDK and its RPC contract.
- Advertise a typed, complete question form in the live and scenario
tool schemas.
- Enforce canonical text and custom-answer constraints before ordinary
or native resolution. Preserve harmless display whitespace.
- Run regex matching in isolated workers with a deadline and resource
limits. Both answer paths await the result before persistence. Saved
native delivery uses the validated answer without taking another worker
slot.
- Update agent guidance and generated Runner contracts.
- Test mixed forms, option-ID collisions, retries, answers, legacy
requests, and conflicting forms.

## Verification

- Interaction service, HTTP route, native bridge, and Runner authority
suites: 221 tests passed after correcting an obsolete tool-description
assertion.
- Shared validator, plugin SDK, CLI, and UI compatibility suites: 67
tests passed.
- Runner core tool-contract suite: 20 tests passed. AJV validates live
and scenario schemas.
- Final review regressions: 172 shared, service, native bridge, and
authority tests passed. These cover text length, pattern, numeric
limits, whitespace, custom option IDs, and historical pending cards.
- Runner session suites: 67 tests passed. Published example tests: 4
tests passed.
- Server typecheck and the shared/server builds passed after the
compatibility fixes.
- Final delivery verification: 35 response-delivery tests passed. The
native delivery regression proves saved answers do not enter pattern
workers; server typecheck and build passed.
- Pattern security and answer-flow verification: 205 tests passed after
repairing the child fixture loader. These cover pathological matching,
event-loop responsiveness, worker concurrency, slot cleanup, HTTP
routes, native delivery, and the full helper in a child process.
- `pnpm -r typecheck` passed on the bounded-worker revision.
- `pnpm build` passed on the bounded-worker revision.
- All 55 GitHub checks passed on `fe457af`; four optional jobs were
skipped. An unchanged Cursor adapter test timed out once in CI, passed
locally, and passed on one failed-job rerun.
- Reviewers can send the canonical-only mixed form above and verify that
the saved interaction contains both canonical and compatibility
questions.

## Risks

- The creation API accepts a new input shape. Stored rows and answer
contracts keep the existing shape.
- The shared projection must preserve synthetic free-text option IDs.
Collision and native round-trip tests cover this behavior.
- Historical partial rows remain readable. New conflicting dual forms,
including written-answer mismatches, remain rejected.
- Existing pending cards retain the written-answer paths offered by
their stored options. Canonical text constraints still apply.
- Ordinary answers now enforce declared canonical constraints before
persistence. Invalid answers leave the card pending.
- Regex validation has a one-second deadline and a four-worker capacity
limit. A complex pattern or capacity error leaves the card pending with
a validation error.
- No database migration or change to company authorization is required.

## Model Used

- OpenAI GPT-6 through Codex. The session exposes the GPT-6 model
family; its exact runtime model identifier and context window size are
not exposed. Used reasoning, tool use, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-10-02 14:21:11 -05:00
1 parent 8fd3cf74ea
commit d034ba7491
39 files changed
+954 -481

No files matched your search

@@ -33,11 +33,14 @@ describe("published hiring and human-input examples", () => {
const skill = readFileSync(new URL("../../../skills/paperclip/SKILL.md", import.meta.url), "utf8");
const section = skill.split('<a id="asking-for-human-input"></a>')[1]!;
const body = JSON.parse(section.match(/```json\n([\s\S]*?)\n```/)![1]);
expect(createIssueThreadInteractionSchema.safeParse(substituteIds(body))).toMatchObject({ success: true });
const normalized = createIssueThreadInteractionSchema.parse(substituteIds(body));
expect(body).toMatchObject({ resolverPolicy: "human_only", continuationPolicy: "wake_assignee" });
expect(body).not.toHaveProperty("addresseeUserId");
expect(body.payload.questionSet.questions[0]).toMatchObject({ answerMode: "text" });
expect(body.payload.questions[0].id).toBe(body.payload.questionSet.questions[0].id);
expect(body.payload).not.toHaveProperty("questions");
expect(normalized.kind).toBe("ask_user_questions");
if (normalized.kind !== "ask_user_questions") throw new Error("expected questions");
expect(normalized.payload.questions[0].id).toBe(body.payload.questionSet.questions[0].id);
});
it("keeps these examples in the generated runner reference without displacing confirmations", () => {
@@ -612,6 +612,23 @@ describe.sequential("issue thread interaction routes", () => {
);
}, 10_000);
it("normalizes a canonical-only mixed question form at the HTTP boundary", async () => {
const questionSet = { schema: "paperclip.question_set.v1", questions: [
{ id: "repo", prompt: "Repository URL?", required: true, answerMode: "text" },
{ id: "scope", prompt: "Review scope?", required: true, answerMode: "single_select", options: [{ id: "all", label: "All changes" }, { id: "selected", label: "Selected changes" }] },
] };
const res = await request(await createApp())
.post("/api/issues/aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa/interactions")
.send({ kind: "ask_user_questions", payload: { version: 1, questionSet } });
expect(res.status).toBe(201);
expect(mockInteractionService.create).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({
payload: expect.objectContaining({ questionSet, questions: [
expect.objectContaining({ id: "repo", options: [{ id: "paperclip_text_answer", label: "Type an answer", freeText: true }] }),
expect.objectContaining({ id: "scope", options: questionSet.questions[1].options }),
] }),
}), expect.anything());
});
it("does not run historical-comment catch-up or queue recovery from the interaction read path", async () => {
mockIssueService.getById.mockResolvedValue(createIssue({
status: "in_review",
@@ -213,6 +213,89 @@ describeEmbeddedPostgres("issueThreadInteractionService", () => {
};
}
it("persists and answers a canonical-only mixed question form", async () => {
const { companyId, issueId } = await seedSourceQuestionFixture({});
const questionSet = {
schema: "paperclip.question_set.v1" as const,
questions: [
{ id: "repo", prompt: "Repository URL?", required: true, answerMode: "text" as const },
{ id: "scope", prompt: "Review scope?", required: true, answerMode: "single_select" as const, options: [{ id: "all", label: "All changes" }, { id: "selected", label: "Selected changes" }] },
{ id: "hosting", prompt: "Preview hosting?", required: true, answerMode: "multi_select" as const, options: [{ id: "existing", label: "Existing host" }, { id: "new", label: "New host" }] },
],
};
const input = { kind: "ask_user_questions" as const, idempotencyKey: "canonical:mixed", payload: { version: 1 as const, questionSet } };
const issue = { id: issueId, companyId };
const created = await interactionsSvc.create(issue, input, { userId: "local-board" });
if (created.kind !== "ask_user_questions") throw new Error("expected questions");
expect(created.payload.questionSet).toEqual(questionSet);
expect(created.payload.questions.map((question) => question.id)).toEqual(["repo", "scope", "hosting"]);
expect(await interactionsSvc.create(issue, input, { userId: "local-board" })).toEqual(created);
await expect(interactionsSvc.answerQuestions(issue, created.id, { answers: [{ questionId: "repo", optionIds: [], otherText: "https://example.com/repo" }] }, { userId: "local-board" })).rejects.toThrow("requires an answer");
const answered = await interactionsSvc.answerQuestions(issue, created.id, { answers: [
{ questionId: "repo", optionIds: [], otherText: "https://example.com/repo" },
{ questionId: "scope", optionIds: ["all"] },
{ questionId: "hosting", optionIds: ["existing", "new"] },
] }, { userId: "local-board" });
expect(answered.status).toBe("answered");
});
it("rejects canonical text and custom answers that violate constraints before resolution", async () => {
const { companyId, issueId } = await seedSourceQuestionFixture({});
const issue = { id: issueId, companyId };
const cases = [
{ textValidation: { minLength: 3, maxLength: 4 }, invalid: ["ab", "abcde"], valid: "abcd" },
{ textValidation: { pattern: "^https://" }, invalid: ["http://example.test"], valid: "https://example.test" },
{ textValidation: { inputType: "integer" as const, minimum: 2, maximum: 4 }, invalid: ["no", "3.5", "1", "5"], valid: "3" },
{ textValidation: { inputType: "number" as const, minimum: 2, maximum: 4 }, invalid: ["Infinity", "1.9", "4.1"], valid: "2.5" },
];
for (const [index, entry] of cases.entries()) {
const created = await interactionsSvc.create(issue, {
kind: "ask_user_questions", payload: { version: 1, questionSet: {
schema: "paperclip.question_set.v1", questions: [{ id: "value", prompt: "Value?", required: true, answerMode: "text", textValidation: entry.textValidation }],
} },
}, { userId: "local-board" });
for (const otherText of entry.invalid) {
await expect(interactionsSvc.answerQuestions(issue, created.id, { answers: [{ questionId: "value", optionIds: [], otherText }] }, { userId: "local-board" })).rejects.toThrow();
const [row] = await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.id, created.id));
expect(row?.status, `case ${index}: ${otherText}`).toBe("pending");
}
expect((await interactionsSvc.answerQuestions(issue, created.id, { answers: [{ questionId: "value", optionIds: [], otherText: entry.valid }] }, { userId: "local-board" })).status).toBe("answered");
}
const custom = await interactionsSvc.create(issue, { kind: "ask_user_questions", payload: { version: 1, questionSet: {
schema: "paperclip.question_set.v1", questions: [{ id: "scope", prompt: "Scope?", required: true, answerMode: "single_select", options: [{ id: "all", label: "All" }, { id: "selected", label: "Selected" }], customAnswer: { enabled: true }, textValidation: { minLength: 3 } }],
} } }, { userId: "local-board" });
await expect(interactionsSvc.answerQuestions(issue, custom.id, { answers: [{ questionId: "scope", optionIds: [], otherText: "ab" }] }, { userId: "local-board" })).rejects.toThrow("at least 3");
expect((await interactionsSvc.answerQuestions(issue, custom.id, { answers: [{ questionId: "scope", optionIds: [], otherText: "abc" }] }, { userId: "local-board" })).status).toBe("answered");
});
it("accepts existing custom-answer IDs in compatible dual forms", async () => {
const { companyId, issueId } = await seedSourceQuestionFixture({});
const issue = { id: issueId, companyId };
const options = [{ id: "all", label: "All" }, { id: "selected", label: "Selected" }];
const created = await interactionsSvc.create(issue, { kind: "ask_user_questions", payload: {
version: 1,
questions: [{ id: "scope", prompt: "Scope?", required: true, selectionMode: "single", options: [...options, { id: "existing-custom-id", label: "Other", freeText: true }] }],
questionSet: { schema: "paperclip.question_set.v1", questions: [{ id: "scope", prompt: "Scope?", required: true, answerMode: "single_select", options, customAnswer: { enabled: true }, textValidation: { minLength: 3 } }] },
} }, { userId: "local-board" });
const answered = await interactionsSvc.answerQuestions(issue, created.id, { answers: [{ questionId: "scope", optionIds: ["existing-custom-id"], otherText: "Specific files" }] }, { userId: "local-board" });
expect(answered).toMatchObject({ status: "answered", result: { answers: [{ questionId: "scope", optionIds: ["existing-custom-id"], otherText: "Specific files" }] } });
});
it("keeps historical pending written-answer paths usable with their text constraints", async () => {
const { companyId, issueId } = await seedSourceQuestionFixture({});
const issue = { id: issueId, companyId };
const created = await interactionsSvc.create(issue, { kind: "ask_user_questions", payload: { version: 1, questionSet: {
schema: "paperclip.question_set.v1", questions: [{ id: "scope", prompt: "Scope?", required: true, answerMode: "single_select", options: [{ id: "all", label: "All" }, { id: "selected", label: "Selected" }], customAnswer: { enabled: true }, textValidation: { minLength: 3 } }],
} } }, { userId: "local-board" });
if (created.kind !== "ask_user_questions") throw new Error("expected questions");
const historicalPayload = structuredClone(created.payload);
delete historicalPayload.questionSet!.questions[0].customAnswer;
await db.update(issueThreadInteractions).set({ payload: historicalPayload }).where(eq(issueThreadInteractions.id, created.id));
await expect(interactionsSvc.answerQuestions(issue, created.id, { answers: [{ questionId: "scope", optionIds: ["paperclip_custom_answer"], otherText: "ab" }] }, { userId: "local-board" })).rejects.toThrow("at least 3");
const answered = await interactionsSvc.answerQuestions(issue, created.id, { answers: [{ questionId: "scope", optionIds: ["paperclip_custom_answer"], otherText: "Specific files" }] }, { userId: "local-board" });
expect(answered.status).toBe("answered");
});
async function seedQuestionUser(companyId: string, userId: string, role = "member", status = "active") {
await db.insert(authUsers).values({ id: userId, name: "Question recipient", email: `${randomUUID()}@example.test`, createdAt: new Date(), updatedAt: new Date() });
await db.insert(companyMemberships).values({ companyId, principalType: "user", principalId: userId, membershipRole: role, status });
@@ -214,6 +214,7 @@ describeEmbeddedPostgres("question response delivery", () => {
id: "features",
prompt: "Which features?",
selectionMode: "multi",
allowOther: true,
options: [
{ id: "health", label: "Health check" },
{ id: "logs", label: "Request logs" },
+1 -1
View File
@@ -16477,7 +16477,7 @@ export function issueRoutes(
const actor = getActorInfo(req);
if (current.kind === "ask_user_questions") {
validateNativeQuestionResponseInput(current, req.body);
await validateNativeQuestionResponseInput(current, req.body);
}
const interaction = await interactionSvc.answerQuestions(
issue,
+1 -1
View File
@@ -8667,7 +8667,7 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) {
);
return true;
}
validateNativeQuestionResponseInput(interaction, { answers });
await validateNativeQuestionResponseInput(interaction, { answers });
// Canonical service rechecks required fields, options, audience, and
// pending -> answered atomically with the response-delivery receipt.
const issue = await db
@@ -2,6 +2,7 @@ import {
currentContinuationOrigins,
deliveredContinuationCommentIds,
} from "./execution-continuation.js";
import { parseQuestionInteractionAnswers } from "./question-interaction-answers.js";
import { isUniqueViolation } from "../db-errors.js";
import { assertAgentRunWriteAllowed } from "../agent-run-cancellation.js";
import { connectionIntentDeliveries } from "@paperclipai/db";
@@ -47,6 +48,7 @@ import type {
CancelIssueThreadInteraction,
ConnectionIntentInteraction,
CreateIssueThreadInteraction,
CreateIssueThreadInteractionInput,
InteractionResolverGovernance,
IssueReviewPolicy,
IssueThreadInteraction,
@@ -3309,7 +3311,7 @@ export function issueThreadInteractionService(
create: async (
issue: { id: string; companyId: string },
input: CreateIssueThreadInteraction,
input: CreateIssueThreadInteractionInput,
actor: InteractionActor,
options: CreateInteractionOptions = {},
) => {
@@ -4894,6 +4896,16 @@ export function issueThreadInteractionService(
questions: interaction.payload.questions,
answers: input.answers,
});
if (interaction.payload.questionSet) {
try {
await parseQuestionInteractionAnswers(interaction.payload.questionSet, normalizedAnswers, interaction.payload.questions);
} catch (error) {
throw unprocessable(
error instanceof Error ? error.message : "Invalid question response",
{ code: "invalid_question_response" },
);
}
}
const updated = await db.transaction(async (tx) => {
await assertInteractionRunWriteAllowed(tx as unknown as Db, issue, actor);
@@ -45,6 +45,7 @@ import {
type IssuePostCommitAction,
} from "../issues.js";
import { questionResponseDeliveryService } from "../question-response-delivery.js";
import * as questionPatternValidation from "../question-pattern-validation.js";
import { heartbeatService } from "../heartbeat.js";
import { DurablePrpControlPlane } from "../../vendor/paperclip-runner/index.js";
import { PaperclipControlPlanePort } from "./paperclip-control-plane-port.js";
@@ -336,6 +337,33 @@ describeEmbeddedPostgres("native question bridge", () => {
} finally { bridge.close(); }
});
it("delivers saved answers without entering pattern validation again", async () => {
await seed();
const event = runtimeRequestEvent();
(event.payload.request as any).input.questions = [{ id: "url", prompt: "URL?", required: true, answerMode: "text" }];
const interaction = await projectNativeRuntimeRequest({ db, binding: binding(), event });
const answered = await issueThreadInteractionService(db).answerQuestions(
{ id: issueId, companyId, status: "in_progress" }, interaction!.id,
{ answers: [{ questionId: "url", optionIds: [], otherText: "https://example.test" }] }, { userId: "operator-1" },
);
if (answered.kind !== "ask_user_questions") throw new Error("expected questions");
const queueCommand = vi.fn(() => ({ commandId: "question", controllerSeq: 1 }));
const release = registerNativeQuestionCommandTarget({ binding: { companyId, issueId, runId, agentId }, queueCommand });
const busy = vi.spyOn(questionPatternValidation, "validateQuestionPatterns").mockRejectedValue(new Error("Question format validation is busy; try again"));
try {
await flushNativeQuestionResponses(db, runId);
expect(queueCommand).toHaveBeenCalledWith("request.resolve", expect.objectContaining({
response: { schema: "paperclip.question_response.v1", answers: { url: { text: "https://example.test" } } },
}), `question_${interaction!.id}`);
expect(busy).not.toHaveBeenCalled();
const [delivery] = await db.select().from(issueQuestionResponseDeliveries);
expect(delivery).toMatchObject({ status: "delivered" });
} finally {
busy.mockRestore();
release();
}
});
it.each(["codex", "claude"])("materializes, validates, and durably resumes a %s question response", async (provider) => {
await seed();
const interaction = await projectNativeRuntimeRequest({
@@ -370,10 +398,10 @@ describeEmbeddedPostgres("native question bridge", () => {
expect(await db.select().from(activityLog)).toHaveLength(1);
const answer = { answers: [{ questionId: "color", optionIds: ["blue"] }] };
validateNativeQuestionResponseInput(interaction!, answer);
expect(() => validateNativeQuestionResponseInput(interaction!, {
await validateNativeQuestionResponseInput(interaction!, answer);
await expect(validateNativeQuestionResponseInput(interaction!, {
answers: [{ questionId: "color", optionIds: ["red"] }],
})).toThrow(/unknown option red/);
})).rejects.toThrow(/unknown option red/);
const answered = await issueThreadInteractionService(db).answerQuestions(
{ id: issueId, companyId, status: "in_progress" },
@@ -648,7 +676,7 @@ describeEmbeddedPostgres("native question bridge", () => {
otherText: "purple",
}],
};
validateNativeQuestionResponseInput(interaction!, answer);
await validateNativeQuestionResponseInput(interaction!, answer);
const answered = await issueThreadInteractionService(db).answerQuestions(
{ id: issueId, companyId, status: "in_progress" },
interaction!.id,
@@ -3,18 +3,15 @@ import { and, eq, inArray, sql } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import { heartbeatRuns, issueThreadInteractions } from "@paperclipai/db";
import type {
AskUserQuestionsAnswer,
AskUserQuestionsInteraction,
AskUserQuestionsQuestionOption,
PaperclipQuestionSetPayload,
RespondIssueThreadInteraction,
} from "@paperclipai/shared";
import { questionSetToAskUserQuestionsPayload } from "@paperclipai/shared";
import type { PrpEvent } from "../../vendor/paperclip-runner/index.js";
import {
parsePaperclipQuestionResponse,
parsePaperclipQuestionSet,
type PaperclipQuestionResponse,
type PaperclipQuestionSet,
} from "../../vendor/paperclip-runner/index.js";
import { logger } from "../../middleware/logger.js";
@@ -24,10 +21,10 @@ import { issueThreadInteractionService } from "../issue-thread-interactions.js";
import { questionResponseDeliveryService } from "../question-response-delivery.js";
import type { NativeRunStoreBinding } from "./native-run-coordinator-store.js";
import { parseQuestionInteractionAnswers, parseSavedQuestionInteractionAnswers } from "../question-interaction-answers.js";
const QUESTION_KEY_PREFIX = "paperclip-runner-question:";
const REQUEST_ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,159}$/;
const TEXT_ANSWER_OPTION_ID = "paperclip_text_answer";
const CUSTOM_ANSWER_OPTION_ID = "paperclip_custom_answer";
export const NATIVE_QUESTION_CANCELLATION_CONTEXT_KEY = "nativeQuestionCancellation";
type QueueCommand = (
@@ -83,100 +80,15 @@ function requestIdForInteraction(
: requestId;
}
function uniqueSyntheticOptionId(existing: readonly string[], preferred: string): string {
const ids = new Set(existing);
if (!ids.has(preferred)) return preferred;
for (let suffix = 2; suffix < 10_000; suffix += 1) {
const candidate = `${preferred}_${suffix}`;
if (!ids.has(candidate)) return candidate;
}
throw new Error("native_question_synthetic_option_exhausted");
}
function toInteractionPayload(questionSet: PaperclipQuestionSet, runtimeRequestId: string) {
return {
version: 1 as const,
...(questionSet.title ? { title: questionSet.title.slice(0, 240) } : {}),
...(questionSet.submitLabel ? { submitLabel: questionSet.submitLabel.slice(0, 120) } : {}),
questions: questionSet.questions.map((question) => {
const canonicalOptions = question.options ?? [];
const options: AskUserQuestionsQuestionOption[] = canonicalOptions.map((option) => ({
id: option.id,
label: option.label,
...(option.description ? { description: option.description } : {}),
}));
if (question.answerMode === "text") {
options.push({
id: uniqueSyntheticOptionId([], TEXT_ANSWER_OPTION_ID),
label: question.header ?? "Type an answer",
...(question.textValidation?.inputType
? { description: `Expected ${question.textValidation.inputType} input` }
: {}),
freeText: true,
});
} else if (question.customAnswer?.enabled) {
options.push({
id: uniqueSyntheticOptionId(canonicalOptions.map((option) => option.id), CUSTOM_ANSWER_OPTION_ID),
label: question.customAnswer.label ?? "Other",
...(question.customAnswer.placeholder ? { description: question.customAnswer.placeholder } : {}),
freeText: true,
});
}
return {
id: question.id,
prompt: question.prompt,
...((question.helpText || question.header)
? { helpText: question.helpText ?? question.header }
: {}),
selectionMode: question.answerMode === "multi_select" ? "multi" as const : "single" as const,
required: question.required,
allowOther: question.answerMode === "text" || question.customAnswer?.enabled === true,
options,
};
}),
questionSet: questionSet as PaperclipQuestionSetPayload,
...questionSetToAskUserQuestionsPayload(questionSet),
runtimeRequestId,
// A generic task comment cannot satisfy this provider request. Keep the
// card actionable until a validated answer or an explicit terminal action.
// A generic task comment cannot satisfy this provider request.
supersedeOnUserComment: false,
};
}
function canonicalResponse(
questionSet: PaperclipQuestionSetPayload,
answers: readonly AskUserQuestionsAnswer[],
): PaperclipQuestionResponse {
const answerByQuestionId = new Map(answers.map((answer) => [answer.questionId, answer]));
const response: PaperclipQuestionResponse = {
schema: "paperclip.question_response.v1",
answers: {},
};
for (const question of questionSet.questions) {
const answer = answerByQuestionId.get(question.id);
if (!answer) continue;
if (question.answerMode === "text") {
response.answers[question.id] = {
...(answer.otherText !== undefined && answer.otherText !== null
? { text: answer.otherText }
: {}),
};
} else {
const customOptionId = question.customAnswer?.enabled
? uniqueSyntheticOptionId(
(question.options ?? []).map((option) => option.id),
CUSTOM_ANSWER_OPTION_ID,
)
: null;
response.answers[question.id] = {
selectedOptionIds: answer.optionIds.filter((optionId) => optionId !== customOptionId),
...(answer.otherText !== undefined && answer.otherText !== null
? { customText: answer.otherText }
: {}),
};
}
}
return parsePaperclipQuestionResponse(questionSet, response);
}
async function authorizedNativeRun(
db: Pick<Db | DbTransaction, "select">,
@@ -312,13 +224,13 @@ export async function projectNativeRuntimeRequest(input: {
}
/** Validate untrusted board input before the existing interaction service persists it. */
export function validateNativeQuestionResponseInput(
export async function validateNativeQuestionResponseInput(
interaction: AskUserQuestionsInteraction,
input: RespondIssueThreadInteraction,
): void {
): Promise<void> {
if (!requestIdForInteraction(interaction) || !interaction.payload.questionSet) return;
try {
canonicalResponse(interaction.payload.questionSet, input.answers);
await parseQuestionInteractionAnswers(interaction.payload.questionSet, input.answers, interaction.payload.questions);
} catch (error) {
throw unprocessable(
error instanceof Error ? error.message : "Invalid native question response",
@@ -340,7 +252,7 @@ export async function deliverNativeQuestionResponse(
// Fall through to durable fresh-wake delivery instead of waiting forever for
// a command target that cannot return for this terminal run.
if (!run || ["succeeded", "failed", "cancelled", "timed_out"].includes(run.status)) return "not_native";
const response = canonicalResponse(interaction.payload.questionSet, interaction.result.answers);
const response = parseSavedQuestionInteractionAnswers(interaction.payload.questionSet, interaction.result.answers, interaction.payload.questions);
const target = activeTargets.get(run.id);
if (
!target
@@ -109,8 +109,10 @@ describe("PaperclipRunnerToolAuthority", () => {
expect(questions.description).toContain("resolve-from-comment");
expect(questions.description).toContain("Existing resolver permissions still apply");
expect(questions.description).toContain("Do not fabricate answer links");
expect(JSON.stringify(questions.inputSchema)).toContain("at least two distinct meaningful options");
expect(JSON.stringify(questions.inputSchema)).toContain("answerMode:'text'");
expect(JSON.stringify(questions.inputSchema)).toContain("at least two meaningful options");
expect(questions.inputSchema).toMatchObject({ properties: { payload: { properties: { questionSet: {
properties: { questions: { items: { properties: { answerMode: { enum: ["single_select", "multi_select", "text"] } } } } },
} } } } });
expect(authority.definitions().map((tool) => tool.name)).toEqual(
expect.arrayContaining([
@@ -408,7 +410,7 @@ describe("PaperclipRunnerToolAuthority", () => {
"current Paperclip task bound to this run",
);
expect(advertised.description).toContain(
"payload.questions for choices",
"one complete payload.questionSet",
);
expect(advertised.description).toContain(
"Paperclip renders it and authenticates the response",
@@ -427,7 +429,7 @@ describe("PaperclipRunnerToolAuthority", () => {
},
);
it.each(["choice", "text"] as const)("executes the advertised %s question once on the bound reviewed task", async (answerMode) => {
it.each(["choice", "text", "mixed"] as const)("executes the advertised %s question once on the bound reviewed task", async (answerMode) => {
const binding = {
companyId: randomUUID(),
agentId: randomUUID(),
@@ -437,7 +439,7 @@ describe("PaperclipRunnerToolAuthority", () => {
await db.insert(companies).values({
id: binding.companyId,
name: "Question invocation",
issuePrefix: answerMode === "choice" ? "RQA" : "RQT",
issuePrefix: answerMode === "choice" ? "RQA" : answerMode === "text" ? "RQT" : "RQM",
});
await db.insert(agents).values({
id: binding.agentId,
@@ -474,7 +476,7 @@ describe("PaperclipRunnerToolAuthority", () => {
const advertised = authority
.definitions()
.find((tool) => tool.name === "request_human_input")!;
expect(advertised.description).toContain("payload.questions");
expect(advertised.description).toContain("payload.questionSet");
const questions = [
{
id: "color",
@@ -490,28 +492,19 @@ describe("PaperclipRunnerToolAuthority", () => {
const payloadDescription = (advertised.inputSchema as {
properties: { payload: { description: string } };
}).properties.payload.description;
expect(payloadDescription).toContain("at least two distinct meaningful options");
expect(payloadDescription).toContain("at least two meaningful options");
expect(payloadDescription).toContain("questionSet");
expect(payloadDescription).not.toContain("use exactly");
const payload = answerMode === "choice"
? { version: 1, questions }
: {
version: 1,
questions: [{
id: "goal",
prompt: "What should we accomplish?",
selectionMode: "single",
required: true,
options: [{ id: "describe", label: "Your answer", freeText: true }],
}],
questionSet: {
schema: "paperclip.question_set.v1",
questions: [{
id: "goal",
prompt: "What should we accomplish?",
answerMode: "text",
required: true,
}],
questions: [
{ id: "goal", prompt: "What should we accomplish?", answerMode: "text", required: true },
...(answerMode === "mixed" ? [{ id: "color", prompt: "Choose one color", answerMode: "single_select", required: true, options: questions[0].options }] : []),
],
},
};
const call = {
@@ -547,6 +540,8 @@ describe("PaperclipRunnerToolAuthority", () => {
.from(issueThreadInteractions)
.where(eq(issueThreadInteractions.issueId, binding.issueId));
expect(rows).toHaveLength(1);
expect((rows[0].payload as { questions: { id: string }[] }).questions.map((question) => question.id))
.toEqual(answerMode === "choice" ? ["color"] : answerMode === "text" ? ["goal"] : ["goal", "color"]);
const [task] = await db
.select()
.from(issues)
@@ -198,21 +198,6 @@ export const runnerApiReference: Record<string, { section: string; description?:
"continuationPolicy": "wake_assignee",
"payload": {
"version": 1,
"questions": [
{
"id": "responsibility",
"prompt": "What should the new agent be responsible for?",
"selectionMode": "single",
"required": true,
"options": [
{
"id": "describe",
"label": "I'll describe it",
"freeText": true
}
]
}
],
"questionSet": {
"schema": "paperclip.question_set.v1",
"questions": [
@@ -236,27 +221,32 @@ export const runnerApiReference: Record<string, { section: string; description?:
"continuationPolicy": "wake_assignee",
"payload": {
"version": 1,
"questions": [
{
"id": "responsibility",
"prompt": "What should the new agent be responsible for?",
"selectionMode": "single",
"required": true,
"allowOther": true,
"options": [
{
"id": "research",
"label": "Research",
"description": "Find and summarize information."
"questionSet": {
"schema": "paperclip.question_set.v1",
"questions": [
{
"id": "responsibility",
"prompt": "What should the new agent be responsible for?",
"answerMode": "single_select",
"required": true,
"customAnswer": {
"enabled": true
},
{
"id": "writing",
"label": "Writing",
"description": "Draft and edit content."
}
]
}
]
"options": [
{
"id": "research",
"label": "Research",
"description": "Find and summarize information."
},
{
"id": "writing",
"label": "Writing",
"description": "Draft and edit content."
}
]
}
]
}
}
}
},
@@ -0,0 +1,30 @@
import { spawnSync } from "node:child_process";
import { fileURLToPath } from "node:url";
import { expect, it } from "vitest";
it("isolates pathological patterns throughout canonical answer validation", () => {
const moduleUrl = new URL("./question-interaction-answers.ts", import.meta.url).href;
const script = `
const { parseQuestionInteractionAnswers } = await import(${JSON.stringify(moduleUrl)});
let ticks = 0;
const timer = setInterval(() => { ticks += 1; }, 10);
try {
await parseQuestionInteractionAnswers({
schema: "paperclip.question_set.v1",
questions: [{ id: "value", prompt: "Value?", required: true, answerMode: "text", textValidation: { pattern: "^(a+)+$" } }],
}, [{ questionId: "value", optionIds: [], otherText: "a".repeat(99999) + "!" }], []);
process.exitCode = 1;
} catch (error) {
process.stdout.write(JSON.stringify({ message: error.message, ticks }));
} finally { clearInterval(timer); }
`;
// A future synchronous regression must fail this child deadline, not hang Vitest.
const child = spawnSync(process.execPath, ["--import", "tsx/esm", "--input-type=module", "-e", script], {
cwd: fileURLToPath(new URL("../..", import.meta.url)), encoding: "utf8", timeout: 10_000,
});
expect(child.error).toBeUndefined();
expect(child.status, child.stderr).toBe(0);
const result = JSON.parse(child.stdout);
expect(result.message).toContain("time limit");
expect(result.ticks).toBeGreaterThan(1);
}, 15_000);
@@ -0,0 +1,86 @@
import type { AskUserQuestionsAnswer, AskUserQuestionsPayload, PaperclipQuestionSetPayload } from "@paperclipai/shared";
import { parsePaperclipQuestionResponse, type PaperclipQuestionResponse } from "../vendor/paperclip-runner/index.js";
import { validateQuestionPatterns } from "./question-pattern-validation.js";
function prepareQuestionInteractionAnswers(
questionSet: PaperclipQuestionSetPayload,
answers: readonly AskUserQuestionsAnswer[],
storageQuestions: AskUserQuestionsPayload["questions"],
) {
const answerByQuestionId = new Map(answers.map((answer) => [answer.questionId, answer]));
const response: PaperclipQuestionResponse = {
schema: "paperclip.question_response.v1",
answers: {},
};
for (const question of questionSet.questions) {
const answer = answerByQuestionId.get(question.id);
if (!answer) continue;
if (question.answerMode === "text") {
response.answers[question.id] = {
...(answer.otherText !== undefined && answer.otherText !== null
? { text: answer.otherText }
: {}),
};
} else {
const customOptionId = storageQuestions.find((entry) => entry.id === question.id)
?.options.find((option) => option.freeText)?.id ?? null;
response.answers[question.id] = {
selectedOptionIds: answer.optionIds.filter((optionId) => optionId !== customOptionId),
...(answer.otherText !== undefined && answer.otherText !== null
? { customText: answer.otherText }
: {}),
};
}
}
// Historical dual forms may offer a written answer only in storage. Keep
// that pending answer path usable; new creation rejects this mismatch.
const answerableQuestionSet = {
...questionSet,
questions: questionSet.questions.map((question) => {
const storage = storageQuestions.find((entry) => entry.id === question.id);
if (question.answerMode !== "text" && !question.customAnswer
&& (storage?.allowOther === true || storage?.options.some((option) => option.freeText))) {
return { ...question, customAnswer: { enabled: true as const } };
}
return question;
}),
};
// The portable parser uses JavaScript regexes. Validate all other fields
// there, but run pattern matching only in a bounded, isolated worker.
const withoutPatterns = {
...answerableQuestionSet,
questions: answerableQuestionSet.questions.map((question) => {
if (!question.textValidation) return question;
const { pattern: _pattern, ...textValidation } = question.textValidation;
return { ...question, textValidation };
}),
};
const parsed = parsePaperclipQuestionResponse(withoutPatterns, response);
const checks = answerableQuestionSet.questions.flatMap((question) => {
const pattern = question.textValidation?.pattern;
const answer = parsed.answers[question.id];
const text = question.answerMode === "text" ? answer?.text : answer?.customText;
return pattern !== undefined && text !== undefined ? [{ questionId: question.id, pattern, text }] : [];
});
return { parsed, checks };
}
/** Validate untrusted answers before persistence, including bounded patterns. */
export async function parseQuestionInteractionAnswers(
questionSet: PaperclipQuestionSetPayload,
answers: readonly AskUserQuestionsAnswer[],
storageQuestions: AskUserQuestionsPayload["questions"],
): Promise<PaperclipQuestionResponse> {
const { parsed, checks } = prepareQuestionInteractionAnswers(questionSet, answers, storageQuestions);
await validateQuestionPatterns(checks);
return parsed;
}
/** Saved answers already passed pattern validation; delivery must not depend on worker capacity. */
export function parseSavedQuestionInteractionAnswers(
questionSet: PaperclipQuestionSetPayload,
answers: readonly AskUserQuestionsAnswer[],
storageQuestions: AskUserQuestionsPayload["questions"],
): PaperclipQuestionResponse {
return prepareQuestionInteractionAnswers(questionSet, answers, storageQuestions).parsed;
}
@@ -0,0 +1,31 @@
import { describe, expect, it } from "vitest";
import { validateQuestionPatterns } from "./question-pattern-validation.js";
const pathological = { questionId: "value", pattern: "^(a+)+$", text: "a".repeat(99_999) + "!" };
describe("isolated question pattern validation", () => {
it("preserves matching and nonmatching regex semantics", async () => {
await validateQuestionPatterns([{ questionId: "url", pattern: "^https://", text: "https://example.test" }]);
await expect(validateQuestionPatterns([{ questionId: "url", pattern: "^https://", text: "http://example.test" }])).rejects.toThrow("does not match");
});
it("terminates pathological matching while the event loop remains responsive", async () => {
let ticks = 0;
const timer = setInterval(() => { ticks += 1; }, 10);
try {
await expect(validateQuestionPatterns([pathological])).rejects.toThrow("time limit");
expect(ticks).toBeGreaterThan(1);
await validateQuestionPatterns([{ questionId: "value", pattern: "^a$", text: "a" }]);
} finally {
clearInterval(timer);
}
}, 5_000);
it("bounds concurrent workers and releases every slot after timeout", async () => {
const pending = Array.from({ length: 4 }, () => validateQuestionPatterns([pathological]));
const settled = Promise.allSettled(pending);
await expect(validateQuestionPatterns([pathological])).rejects.toThrow("busy");
expect((await settled).every(result => result.status === "rejected")).toBe(true);
await validateQuestionPatterns([{ questionId: "value", pattern: "^a$", text: "a" }]);
}, 5_000);
});
@@ -0,0 +1,53 @@
import { Worker } from "node:worker_threads";
interface PatternCheck { questionId: string; pattern: string; text: string }
const MAX_WORKERS = 4;
const DEADLINE_MS = 1_000;
let activeWorkers = 0;
const workerSource = `
const { parentPort, workerData } = require("node:worker_threads");
const invalid = workerData.find(check => !new RegExp(check.pattern).test(check.text));
parentPort.postMessage(invalid ? invalid.questionId : null);
`;
/** Keep caller-supplied regex execution outside the request-serving thread. */
export async function validateQuestionPatterns(checks: readonly PatternCheck[]): Promise<void> {
if (checks.length === 0) return;
if (activeWorkers >= MAX_WORKERS) throw new Error("Question format validation is busy; try again");
activeWorkers += 1;
let worker: Worker;
try {
worker = new Worker(workerSource, {
eval: true,
workerData: checks,
execArgv: [],
resourceLimits: { maxOldGenerationSizeMb: 32, maxYoungGenerationSizeMb: 8, stackSizeMb: 2 },
});
} catch (error) {
activeWorkers -= 1;
throw error;
}
await new Promise<void>((resolve, reject) => {
let settled = false;
const finish = (error?: Error) => {
if (settled) return;
settled = true;
clearTimeout(timer);
// Release the slot after the worker actually stops, including timeouts.
void worker.terminate().then(() => {
activeWorkers -= 1;
if (error) reject(error); else resolve();
}, (terminationError: unknown) => {
activeWorkers -= 1;
reject(error ?? terminationError);
});
};
const timer = setTimeout(() => finish(new Error("Question format validation exceeded its time limit")), DEADLINE_MS);
worker.once("message", (questionId: string | null) => finish(questionId === null ? undefined
: new Error(`Question ${questionId} does not match the required format`)));
worker.once("error", finish);
worker.once("exit", () => finish(new Error("Question format validation stopped before completion")));
});
}