mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-11 05:11:49 +01:00
fix(ui): show submitted screenshots and Markdown in steering messages (#20367)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People send follow-up messages while an agent works. > - The queue above the composer holds those messages until delivery. > - The queue showed raw Markdown and had no screenshot previews. > - Submitted screenshots stayed in the composer until the save response arrived. > - This pull request shows formatted text and previews in the queue at submission. > - It keeps upload receipts so failed saves can restore the draft and its files. ## Linked Issues or Issue Description **What happened?** Send a message with Markdown and screenshots while an agent works. The queue shows Markdown source. It has no screenshot previews. The composer keeps the submitted screenshots until the save response arrives. **Expected behavior** Show formatted text and screenshot previews in the queue. Clear the submitted previews with the text. Restore them if the save fails. Keep files added for the next message. **Steps to reproduce** 1. Open the Screenshot Submission Storybook case under Steering and queued messages. 2. Add a screenshot and formatted text. Select Send. 3. Leave the simulated save pending. Inspect the queue and composer. 4. Reject the save. Confirm the draft and screenshots return. Retry and complete the save. Related work: #19321 bounds the queued-message list. This fix keeps that scrolling behavior. ## What Changed - Hide submitted attachment previews as soon as Send clears the text. Keep their upload receipts until the request settles. - Use the shared Markdown renderer in queued rows. Show up to three compact screenshots with a count for more images. Strip standalone image syntax from the text preview. - Keep paragraph boundaries. Wrap thumbnails and controls within narrow rows. - Load queue previews only from local attachment and asset content paths. Add an opt-in image omission mode to the shared Markdown renderer for compact queue text. - Correct the case lookup API summary so the existing search test passes with the full catalog. - Add regressions for successful, rejected, and uncertain saves. Check files added to the next draft. - Add a production-component Storybook case with controlled save responses and a desktop/mobile browser test. - Document the queue display and draft recovery behavior in DESIGN.md. ## Verification - Red: the original components fail five new unit cases. The desktop and mobile browser cases also fail on raw Markdown. Screenshots show the previews still in the composer. - Green: 256 composer, queue, and shared Markdown unit tests pass. - Related coverage: 466 thread, draft, and attachment tests pass. One existing test is skipped. - Three browser cases pass. Desktop and mobile cases check pending save, rejection, retry, successful save, and steering. A 320px case checks separate paragraphs with no margins, three loaded thumbnails, the additional-image count, and auto-send controls. It checks that all controls and images fit within the queue. - UI typecheck and design token gates pass after the review fixes. The initial implementation also passed the UI production build, Storybook catalog checks, and 30 map/catalog tests. The final PR workflow passed typecheck, build, all required test suites, and browser E2E checks. - Browser command: `pnpm exec playwright test queued-message-screenshots.spec.ts --config=tests/storybook-visual/playwright.config.ts --project=stories`. Use `STORYBOOK_TEST_BASE_URL` to target an owned Storybook server. - Review regressions first failed with two unit failures and an inline-paragraph browser failure. They pass after the fixes. - Initial CI failed the case lookup search test. It reproduced locally and now passes all three cases after the API summary correction. - Initial CI also failed process identity inspection during a server shutdown test. All 22 focused shutdown tests pass locally without source changes. The cause remains unconfirmed; the final CI Runner support suite also passed without changes to process cleanup. - The browser cases mount production components with simulated upload/save and steer responses. They do not prove backend persistence or live provider delivery. - `pnpm verify` returns exit 2 because DESIGN.md is a global CI input. The required PR workflow supplies the broad CI coverage. - Native preflight cannot finish because Cargo is absent in this environment. UI dependencies were installed. The missing plugin SDK build was prepared before the successful UI checks. - Final head: `2219de45721360d36341ea1ae46641b6bcb20360`. [CI run](https://github.com/paperclipai/paperclip/actions/runs/42708839687) passed. No checks are pending or failing. Greptile and Paperclip Review each gave 5/5. All five prior review threads are resolved. ## Risks - Narrow rows can grow to show two text lines and thumbnails. The existing queue height limit still applies. - Screenshot previews use the existing image-reference parser and a local attachment path filter. Image omission is opt-in; thread and decision media keep their existing modes. - Pending uploads remain in draft state while their previews are hidden. Rejection and uncertain-response tests check that the same receipts return without losing later attachments. ## Model Used - OpenAI Codex, based on GPT-6. The runtime does not expose a more specific serving model ID or context limit. Used reasoning, tools, code execution, and browser checks. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
facd409073
commit
eebdfe71b1
8 files changed
+277
-19
No files matched your search
@@ -124,6 +124,13 @@ starts this queued turn stays in history, including when it reopens an older
|
||||
task. This placement is not a consumption receipt. Later follow-ups remain in
|
||||
the composer queue with their existing edit, order and discard controls.
|
||||
|
||||
Queued messages render Markdown and show compact local attachment thumbnails.
|
||||
Keep paragraph boundaries and wrap thumbnails and controls within narrow rows.
|
||||
Omit inline images from the compact text preview. On
|
||||
submit, move the text and attachment previews out of the composer together.
|
||||
Retain upload receipts while saving so a rejected or uncertain send restores
|
||||
the draft without discarding files added for the next message.
|
||||
|
||||
Stream the provider's final reply as the conversational answer. Completion-tool
|
||||
reports remain inspectable run data while work continues. Use their summary as
|
||||
an answer fallback only after the run ends and its final transcript fetch
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
import { expect, test } from "@playwright/test";
|
||||
|
||||
for (const mobile of [false, true]) {
|
||||
test(`screenshot steering handoff ${mobile ? "mobile" : "desktop"}`, async ({ page }, testInfo) => {
|
||||
if (mobile) await page.setViewportSize({ width: 390, height: 844 });
|
||||
const errors: string[] = [];
|
||||
page.on("pageerror", error => errors.push(error.message));
|
||||
await page.addInitScript(() => localStorage.setItem("storybook-screenshot-submit", "Check **mobile** and *desktop* with `pnpm verify`."));
|
||||
await page.goto("/iframe.html?id=composer-steering-and-queued-messages--screenshot-submission&viewMode=story&globals=theme:light");
|
||||
const composer = page.getByTestId("queued-composer");
|
||||
await expect(composer).toBeVisible({ timeout: 30_000 });
|
||||
// Attach real browser screenshots through the production file picker.
|
||||
const screenshot = await page.getByRole("main").screenshot();
|
||||
await page.route("**/api/attachments/*/content", route => route.fulfill({ contentType: "image/png", body: screenshot }));
|
||||
await composer.locator('input[type="file"]').setInputFiles([
|
||||
{ name: "mobile.png", mimeType: "image/png", buffer: screenshot },
|
||||
{ name: "desktop.png", mimeType: "image/png", buffer: screenshot },
|
||||
]);
|
||||
await expect(composer.getByRole("button", { name: "Remove mobile.png" })).toBeEnabled();
|
||||
await expect(composer.getByRole("button", { name: "Remove desktop.png" })).toBeEnabled();
|
||||
await page.screenshot({ path: testInfo.outputPath("before-submit.png"), fullPage: true });
|
||||
await composer.getByRole("button", { name: /^Send/ }).click();
|
||||
await expect(page.getByRole("button", { name: "Complete save" })).toBeVisible();
|
||||
const queue = page.getByTestId("task-chat-queued-messages");
|
||||
await expect(queue.locator("strong")).toHaveText("mobile");
|
||||
await expect(queue.locator("em")).toHaveText("desktop");
|
||||
await expect(queue.locator("code")).toHaveText("pnpm verify");
|
||||
await expect(queue.locator("img")).toHaveCount(2);
|
||||
await expect.poll(() => queue.locator("img").evaluateAll(images => images.every(image => (image as HTMLImageElement).naturalWidth > 0))).toBe(true);
|
||||
expect(await queue.evaluate(element => {
|
||||
const bounds = element.getBoundingClientRect();
|
||||
return [...element.querySelectorAll("img")].every(image => {
|
||||
const imageBounds = image.getBoundingClientRect();
|
||||
return imageBounds.top >= bounds.top && imageBounds.bottom <= bounds.bottom;
|
||||
});
|
||||
})).toBe(true);
|
||||
await expect(composer.getByTestId("task-chat-composer-images")).toHaveCount(0);
|
||||
await expect(composer.locator('[contenteditable="true"]')).toHaveText("");
|
||||
expect(await page.evaluate(() => document.documentElement.scrollWidth <= innerWidth)).toBe(true);
|
||||
await page.screenshot({ path: testInfo.outputPath("save-pending.png"), fullPage: true });
|
||||
|
||||
// A rejected save restores the same draft and screenshots for retry.
|
||||
await page.getByRole("button", { name: "Reject save" }).click();
|
||||
await expect(queue).toHaveCount(0);
|
||||
await expect(composer.getByTestId("task-chat-composer-images").locator("img")).toHaveCount(2);
|
||||
await expect(composer.locator("strong")).toContainText("mobile");
|
||||
await composer.getByRole("button", { name: /^Send/ }).click();
|
||||
await page.getByRole("button", { name: "Complete save" }).click();
|
||||
await expect(composer.getByTestId("task-chat-composer-images")).toHaveCount(0);
|
||||
await expect(queue.locator("img")).toHaveCount(2);
|
||||
await page.screenshot({ path: testInfo.outputPath("save-complete.png"), fullPage: true });
|
||||
await queue.getByRole("button", { name: "Steer", exact: true }).click();
|
||||
await expect(queue).toHaveCount(0);
|
||||
await expect(page.getByRole("status")).toHaveText("Message steered into the active turn.");
|
||||
expect(errors).toEqual([]);
|
||||
});
|
||||
}
|
||||
|
||||
test("paragraphs and four screenshots fit beside narrow auto-send controls", async ({ page }, testInfo) => {
|
||||
await page.setViewportSize({ width: 320, height: 844 });
|
||||
await page.clock.setFixedTime(new Date("2026-11-09T22:00:00Z"));
|
||||
await page.addInitScript(() => localStorage.setItem("storybook-screenshot-submit", "First paragraph.\n\nSecond paragraph."));
|
||||
await page.goto("/iframe.html?id=composer-steering-and-queued-messages--screenshot-submission&viewMode=story&args=autoSteer:true;mobile:true&globals=theme:light");
|
||||
const composer = page.getByTestId("queued-composer");
|
||||
await expect(composer).toBeVisible({ timeout: 30_000 });
|
||||
const screenshot = await page.getByRole("main").screenshot();
|
||||
await page.route("**/api/attachments/*/content", route => route.fulfill({ contentType: "image/png", body: screenshot }));
|
||||
await composer.locator('input[type="file"]').setInputFiles([1, 2, 3, 4].map(index => ({
|
||||
name: `screen-${index}.png`, mimeType: "image/png", buffer: screenshot,
|
||||
})));
|
||||
await expect(composer.getByRole("button", { name: "Remove screen-4.png" })).toBeEnabled();
|
||||
await composer.getByRole("button", { name: /^Send/ }).click();
|
||||
const queue = page.getByTestId("task-chat-queued-messages");
|
||||
const first = queue.locator("p").filter({ hasText: "First paragraph." });
|
||||
const second = queue.locator("p").filter({ hasText: "Second paragraph." });
|
||||
await expect(first).toBeVisible();
|
||||
await expect(second).toBeVisible();
|
||||
expect(await first.evaluate(element => getComputedStyle(element).display)).toBe("block");
|
||||
expect(await first.evaluate(element => getComputedStyle(element).marginBottom)).toBe("0px");
|
||||
expect(await second.evaluate(element => getComputedStyle(element).marginTop)).toBe("0px");
|
||||
expect((await second.boundingBox())!.y).toBeGreaterThan((await first.boundingBox())!.y);
|
||||
await expect(queue.getByText("Sending in 5s")).toBeVisible();
|
||||
const images = queue.getByRole("group", { name: "Queued images" });
|
||||
await expect(images.locator("img")).toHaveCount(3);
|
||||
await expect.poll(() => images.locator("img").evaluateAll(nodes => nodes.every(image => (image as HTMLImageElement).naturalWidth > 0))).toBe(true);
|
||||
await expect(images.getByText("+1")).toBeVisible();
|
||||
expect(await queue.evaluate(element => {
|
||||
const bounds = element.getBoundingClientRect();
|
||||
return [...element.querySelectorAll('[aria-label="Queued images"] > *, button, [data-testid^="task-chat-queued-auto-steer"]')].every(child => {
|
||||
const box = child.getBoundingClientRect();
|
||||
return box.left >= bounds.left && box.right <= bounds.right && box.top >= bounds.top && box.bottom <= bounds.bottom;
|
||||
});
|
||||
})).toBe(true);
|
||||
expect(await page.evaluate(() => document.documentElement.scrollWidth <= innerWidth)).toBe(true);
|
||||
await page.screenshot({ path: testInfo.outputPath("narrow-auto-steer.png"), fullPage: true });
|
||||
});
|
||||
@@ -130,8 +130,8 @@ interface MarkdownBodyProps {
|
||||
resolveLinkHref?: (href: string) => string | null;
|
||||
/** Called when a user clicks an inline image */
|
||||
onImageClick?: (src: string, image: HTMLImageElement) => void;
|
||||
/** Keep untrusted decision media inert: image references and diagram source only. */
|
||||
mediaMode?: "render" | "reference";
|
||||
/** Keep media inert as references, or omit images in compact text previews. */
|
||||
mediaMode?: "render" | "reference" | "omit";
|
||||
/**
|
||||
* Resolver that decides which inline-code workspace file paths may be linked
|
||||
* to the issue file viewer. Omitting it (or returning null) leaves every
|
||||
@@ -967,7 +967,9 @@ function MarkdownBodyImpl({
|
||||
);
|
||||
},
|
||||
};
|
||||
if (mediaMode === "reference") {
|
||||
if (mediaMode === "omit") {
|
||||
map.img = () => null;
|
||||
} else if (mediaMode === "reference") {
|
||||
map.img = ({ src, alt, title }) => (
|
||||
<span data-markdown-image-reference title={title}>
|
||||
Image: {alt || "Untitled image"}{src ? ` (${src})` : ""}
|
||||
|
||||
@@ -19,6 +19,7 @@ import { instanceSettingsApi } from "@/api/instanceSettings";
|
||||
import { CoordinatorPresentation } from "@/components/coordinator/CoordinatorPresentation";
|
||||
import { DRAFT_DEBOUNCE_MS } from "../../lib/composer-draft";
|
||||
import {
|
||||
loadDraftAttachments,
|
||||
loadDraftSubmission,
|
||||
saveDraft,
|
||||
saveDraftSubmission,
|
||||
@@ -1433,6 +1434,55 @@ describe("TaskChatComposer", () => {
|
||||
expect(onAdd.mock.calls[0]?.[0]).toBe("Review this image\n\n");
|
||||
});
|
||||
|
||||
it.each(["saved", "rejected", "unknown"] as const)("moves submitted screenshots out of the composer before the request is %s", async (outcome) => {
|
||||
let resolveSend!: () => void;
|
||||
let rejectSend!: (error: Error) => void;
|
||||
const onAdd = vi.fn().mockReturnValue(new Promise<void>((resolve, reject) => {
|
||||
resolveSend = resolve;
|
||||
rejectSend = reject;
|
||||
}));
|
||||
const draftKey = `screenshots-${outcome}`;
|
||||
const onAttachImage = vi.fn().mockResolvedValueOnce({
|
||||
id: "8c365a89-5545-40f7-8cc0-ec6b20567827", contentPath: "/api/attachments/8c365a89-5545-40f7-8cc0-ec6b20567827/content", originalFilename: "shot.png", contentType: "image/png",
|
||||
}).mockResolvedValue({
|
||||
id: "266ff1d0-6be9-47cb-b2ed-300fce5fbcd3", contentPath: "/api/attachments/266ff1d0-6be9-47cb-b2ed-300fce5fbcd3/content", originalFilename: "next.png", contentType: "image/png",
|
||||
});
|
||||
renderComposer({ onAdd, workMode: "standard", draftKey, onAttachImage });
|
||||
typeText("Check **this** screenshot");
|
||||
pasteFiles([new File(["png"], "shot.png", { type: "image/png" })]);
|
||||
await flushAsync();
|
||||
expect(container.querySelector('[data-testid="task-chat-composer-images"]')).not.toBeNull();
|
||||
|
||||
flushSync(() => sendButton().click());
|
||||
await flushAsync();
|
||||
expect(onAdd.mock.calls[0]?.[0]).toBe("Check **this** screenshot\n\n");
|
||||
expect(onAdd.mock.calls[0]?.[3]).toEqual(["8c365a89-5545-40f7-8cc0-ec6b20567827"]);
|
||||
expect(editable().textContent).toBe("");
|
||||
expect(container.querySelector('[data-testid="task-chat-composer-images"]')).toBeNull();
|
||||
// Hiding the submitted preview must not discard its reload/retry receipt.
|
||||
expect(loadDraftAttachments(draftKey).map(item => item.attachmentId)).toEqual(["8c365a89-5545-40f7-8cc0-ec6b20567827"]);
|
||||
typeText("Next draft");
|
||||
pasteFiles([new File(["next"], "next.png", { type: "image/png" })]);
|
||||
await flushAsync();
|
||||
expect(container.querySelector('img[alt="next.png"]')).not.toBeNull();
|
||||
expect(container.querySelector('img[alt="shot.png"]')).toBeNull();
|
||||
if (outcome === "saved") resolveSend();
|
||||
else rejectSend(outcome === "unknown" ? new CommentSubmissionUnknownError() : new Error("Rejected"));
|
||||
await flushAsync();
|
||||
await flushAsync();
|
||||
if (outcome === "saved") {
|
||||
expect(container.querySelector('img[alt="shot.png"]')).toBeNull();
|
||||
expect(editable().textContent).toBe("Next draft");
|
||||
expect(loadDraftAttachments(draftKey).map(item => item.attachmentId)).toEqual(["266ff1d0-6be9-47cb-b2ed-300fce5fbcd3"]);
|
||||
} else {
|
||||
expect(container.querySelectorAll('img[alt="shot.png"]')).toHaveLength(1);
|
||||
expect(editable().textContent).toBe("Check **this** screenshot\n\nNext draft");
|
||||
expect(loadDraftAttachments(draftKey).map(item => item.attachmentId)).toEqual(["8c365a89-5545-40f7-8cc0-ec6b20567827", "266ff1d0-6be9-47cb-b2ed-300fce5fbcd3"]);
|
||||
if (outcome === "unknown") expect(loadDraftSubmission(draftKey)?.request?.attachmentIds).toEqual(["8c365a89-5545-40f7-8cc0-ec6b20567827"]);
|
||||
}
|
||||
expect(container.querySelectorAll('img[alt="next.png"]')).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("keeps failed image uploads in the preview row for retry without changing the text", async () => {
|
||||
const onAdd = vi.fn();
|
||||
const onAttachImage = vi.fn().mockRejectedValueOnce(new Error("Too large"))
|
||||
|
||||
@@ -471,6 +471,9 @@ export function TaskChatComposer({
|
||||
: [],
|
||||
);
|
||||
const attachmentsRef = useRef(attachments);
|
||||
// Keep upload receipts durable while the optimistic message owns the preview.
|
||||
// A rejected or uncertain send returns these same attachments to the draft.
|
||||
const [sendingAttachmentIds, setSendingAttachmentIds] = useState<ReadonlySet<string> | null>(null);
|
||||
function setAttachments(
|
||||
update:
|
||||
| ComposerAttachment[]
|
||||
@@ -869,8 +872,9 @@ export function TaskChatComposer({
|
||||
const attachedRefs = attachments.filter(
|
||||
(item) => item.status === "attached" && item.contentPath && !item.inline,
|
||||
);
|
||||
const imageAttachments = attachments.filter(item => isImage(item) && !item.inline);
|
||||
const visibleAttachments = attachments.filter(
|
||||
const draftAttachments = attachments.filter(item => !sendingAttachmentIds?.has(item.id));
|
||||
const imageAttachments = draftAttachments.filter(item => isImage(item) && !item.inline);
|
||||
const visibleAttachments = draftAttachments.filter(
|
||||
(item) => (!item.inline || item.status !== "attached") && !isImage(item),
|
||||
);
|
||||
// Sending mid-upload would silently drop the pending file from the comment;
|
||||
@@ -1030,6 +1034,7 @@ export function TaskChatComposer({
|
||||
else clearDraft(draftKey);
|
||||
}
|
||||
setBody("");
|
||||
setSendingAttachmentIds(new Set(submittedAttachments.map(item => item.id)));
|
||||
setSubmitting(true);
|
||||
let attemptId: string | null = null;
|
||||
let request: ComposerSendRequest | null = null;
|
||||
@@ -1133,6 +1138,7 @@ export function TaskChatComposer({
|
||||
setBody(restoredBody);
|
||||
} finally {
|
||||
if (pendingDraftRef.current?.attemptId === attemptId) pendingDraftRef.current = null;
|
||||
setSendingAttachmentIds(null);
|
||||
setSubmitting(false);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,6 +5,8 @@ import { flushSync } from "react-dom";
|
||||
import { createRoot, type Root } from "react-dom/client";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import type { IssueQueuedCommentQueue } from "@paperclipai/shared";
|
||||
import { ThemeProvider } from "@/context/ThemeContext";
|
||||
import { MemoryRouter } from "@/lib/router";
|
||||
import {
|
||||
reorderQueuedMessageEntries,
|
||||
TaskChatQueuedMessages,
|
||||
@@ -76,7 +78,7 @@ describe("TaskChatQueuedMessages", () => {
|
||||
onDiscard: vi.fn().mockResolvedValue(undefined),
|
||||
...overrides,
|
||||
};
|
||||
flushSync(() => root.render(<TaskChatQueuedMessages {...props} />));
|
||||
flushSync(() => root.render(<MemoryRouter><ThemeProvider><TaskChatQueuedMessages {...props} /></ThemeProvider></MemoryRouter>));
|
||||
return props;
|
||||
}
|
||||
|
||||
@@ -116,6 +118,53 @@ describe("TaskChatQueuedMessages", () => {
|
||||
expect(container.textContent).toContain("Second queued message");
|
||||
});
|
||||
|
||||
it("renders queued Markdown and screenshot thumbnails without showing image syntax", () => {
|
||||
const body = "Check **mobile** and *desktop* with `pnpm verify`.\n\n\n";
|
||||
render({ queue: { ...queue, entries: [{ ...queue.entries[0], comment: { ...queue.entries[0].comment, body } }] } });
|
||||
const row = container.querySelector('[data-testid="task-chat-queued-message-comment-1"]')!;
|
||||
expect(row.querySelector("strong")?.textContent).toBe("mobile");
|
||||
expect(row.querySelector("em")?.textContent).toBe("desktop");
|
||||
expect(row.querySelector("code")?.textContent).toBe("pnpm verify");
|
||||
expect([...row.querySelectorAll("img")].map(img => [img.alt, img.getAttribute("src")])).toEqual([
|
||||
["mobile.png", "/api/attachments/mobile/content"],
|
||||
["desktop.png", "/api/attachments/desktop/content"],
|
||||
]);
|
||||
expect(row.textContent).not.toContain("";
|
||||
render({ queue: { ...queue, entries: [{ ...queue.entries[0], comment: { ...queue.entries[0].comment, body } }] } });
|
||||
expect(container.querySelector('img[alt="screen.png"]')?.getAttribute("src")).toBe("/api/attachments/screen/content");
|
||||
expect(container.textContent).not.toContain(body);
|
||||
});
|
||||
|
||||
it("only loads local attachment images and counts the remaining screenshots", () => {
|
||||
const urls = [
|
||||
"/api/attachments/first/content", "/api/assets/second/content?download=0",
|
||||
"/api/attachments/third/content", "/api/attachments/fourth/content",
|
||||
"https://other.example/api/attachments/pixel/content", "//other.example/pixel.png",
|
||||
"javascript:alert", "data:image/png;base64,AAAA", "/other/pixel.png",
|
||||
"/api/attachments/first/content/extra", "/api/attachments/first/content#fragment",
|
||||
];
|
||||
const body = urls.map((url, index) => ``).join("\n");
|
||||
render({ queue: { ...queue, entries: [{ ...queue.entries[0], comment: { ...queue.entries[0].comment, body } }] } });
|
||||
const row = container.querySelector('[data-testid="task-chat-queued-message-comment-1"]')!;
|
||||
expect([...row.querySelectorAll("img")].map(image => image.getAttribute("src"))).toEqual(urls.slice(0, 3));
|
||||
expect(row.querySelector('[aria-label="Queued images"]')?.textContent).toBe("+1");
|
||||
});
|
||||
|
||||
it("omits inline images from the compact text without exposing their URLs", () => {
|
||||
const body = "Check  then **retry**.";
|
||||
render({ queue: { ...queue, entries: [{ ...queue.entries[0], comment: { ...queue.entries[0].comment, body } }] } });
|
||||
const row = container.querySelector('[data-testid="task-chat-queued-message-comment-1"]')!;
|
||||
expect(row.querySelector("strong")?.textContent).toBe("retry");
|
||||
expect(row.textContent).toContain("Check then retry.");
|
||||
expect(row.textContent).not.toContain("/api/attachments");
|
||||
expect(row.querySelector("img")).toBeNull();
|
||||
});
|
||||
|
||||
it.each(["legacy", "native", "native-plan"] as const)("shows a read-only response and sends it only on click for %s", async (runtime) => {
|
||||
const entry = { ...queue.entries[0], canEdit: false, canDiscard: false,
|
||||
comment: { ...queue.entries[0].comment, body: 'Accepted: Build the app\n\n{"revision": "v1"}' },
|
||||
|
||||
@@ -29,6 +29,8 @@ import type {
|
||||
IssueQueuedCommentQueue,
|
||||
} from "@paperclipai/shared";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { MarkdownBody } from "@/components/MarkdownBody";
|
||||
import { extractImageRefs, stripStandaloneImageEmbeds } from "./task-chat-attachments";
|
||||
import { TaskChatComposerBar } from "./TaskChatComposerBar";
|
||||
import {
|
||||
queuedMessageAutoSteerDeadline,
|
||||
@@ -121,7 +123,10 @@ function SortableQueuedMessage({
|
||||
onCancelAutoSteer: () => void;
|
||||
}) {
|
||||
const immutableResponse = entry.source?.kind === "interaction";
|
||||
const label = queuedMessageLabel(entry);
|
||||
const body = queuedMessageLabel(entry);
|
||||
const images = extractImageRefs(body).filter(image =>
|
||||
/^\/api\/(?:attachments|assets)\/[^/?#\\]+\/content(?:\?[^#\\]*)?$/.test(image.url));
|
||||
const label = stripStandaloneImageEmbeds(body);
|
||||
const sortable = useSortable({
|
||||
id: entry.comment.id,
|
||||
disabled: queueMutationDisabled || immutableResponse,
|
||||
@@ -144,7 +149,7 @@ function SortableQueuedMessage({
|
||||
ref={sortable.setNodeRef}
|
||||
style={style}
|
||||
className={cn(
|
||||
"group flex h-11 min-w-0 items-center gap-1 border-b border-border/55 bg-card/95 px-2.5 text-sm last:border-b-0",
|
||||
"group flex min-h-11 min-w-0 flex-wrap items-center gap-1 border-b border-border/55 bg-card/95 px-2.5 py-1 text-sm last:border-b-0",
|
||||
sortable.isDragging &&
|
||||
"relative z-20 rounded-lg border border-border shadow-lg",
|
||||
)}
|
||||
@@ -156,7 +161,7 @@ function SortableQueuedMessage({
|
||||
{...sortable.attributes}
|
||||
{...sortable.listeners}
|
||||
disabled={queueMutationDisabled || immutableResponse}
|
||||
aria-label={`Reorder queued message: ${label}`}
|
||||
aria-label={`Reorder queued message: ${label || "Images"}`}
|
||||
className="flex h-7 w-7 shrink-0 cursor-grab items-center justify-center rounded-md text-muted-foreground/70 transition-colors hover:bg-accent hover:text-foreground active:cursor-grabbing disabled:cursor-default disabled:opacity-40"
|
||||
>
|
||||
<GripVertical className="h-3.5 w-3.5" aria-hidden />
|
||||
@@ -166,9 +171,20 @@ function SortableQueuedMessage({
|
||||
className="h-3.5 w-3.5 shrink-0 text-muted-foreground"
|
||||
aria-hidden
|
||||
/>
|
||||
<span className="min-w-0 flex-1 truncate px-1" title={label}>
|
||||
{label}
|
||||
</span>
|
||||
<div className="flex min-w-24 max-w-full flex-1 flex-col items-start gap-1 overflow-hidden px-1 sm:flex-row sm:items-center sm:gap-2" title={label || undefined}>
|
||||
{label ? <MarkdownBody
|
||||
className="w-full min-w-0 line-clamp-2 sm:w-auto sm:flex-1 sm:line-clamp-1 [&_p]:m-0!"
|
||||
linkIssueReferences={false}
|
||||
mediaMode="omit"
|
||||
>{label}</MarkdownBody> : null}
|
||||
{images.length ? <div className="flex min-w-0 max-w-full flex-wrap items-center gap-1" role="group" aria-label="Queued images">
|
||||
{images.slice(0, 3).map(image => <img
|
||||
key={image.url} src={image.url} alt={image.name} loading="lazy"
|
||||
className="size-8 shrink-0 rounded-md border border-border object-cover"
|
||||
/>)}
|
||||
{images.length > 3 ? <span className="text-xs text-muted-foreground">+{images.length - 3}</span> : null}
|
||||
</div> : null}
|
||||
</div>
|
||||
|
||||
{autoSteerSecondsLeft !== null ? (
|
||||
<span className="flex shrink-0 items-center gap-1" data-testid={`task-chat-queued-auto-steer-${entry.comment.id}`}>
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
import { useState } from "react";
|
||||
import { useRef, useState } from "react";
|
||||
import type { Meta, StoryObj } from "@storybook/react-vite";
|
||||
import { expect, userEvent, within } from "storybook/test";
|
||||
import type { IssueQueuedCommentEntry, IssueQueuedCommentQueue } from "@paperclipai/shared";
|
||||
import { TaskChatComposer } from "@/components/task-chat/TaskChatComposer";
|
||||
import { TaskChatQueuedMessages } from "@/components/task-chat/TaskChatQueuedMessages";
|
||||
import { QUEUED_MESSAGE_AUTO_STEER_DELAY_MS } from "@/components/task-chat/queued-message-auto-steer";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { composerAgentAppearance, composerAgents } from "../../prototypes/composer-model-picker/fixtures";
|
||||
|
||||
const agentMap = new Map(composerAgents.map((agent) => [agent.id, {
|
||||
@@ -46,10 +47,13 @@ type QueueStoryProps = {
|
||||
mobile: boolean;
|
||||
/** Production delay before an eligible queued message is steered automatically. */
|
||||
autoSteer: boolean;
|
||||
screenshotSubmission: boolean;
|
||||
};
|
||||
|
||||
function QueuedComposer({ protocol, steeringDisposition, initialEdit, mobile, autoSteer }: QueueStoryProps) {
|
||||
const [entries, setEntries] = useState(initialEntries);
|
||||
function QueuedComposer({ protocol, steeringDisposition, initialEdit, mobile, autoSteer, screenshotSubmission }: QueueStoryProps) {
|
||||
const [entries, setEntries] = useState(screenshotSubmission ? [] : initialEntries);
|
||||
const [pendingSave, setPendingSave] = useState<{ resolve: () => void; reject: () => void } | null>(null);
|
||||
const uploadIndex = useRef(0);
|
||||
const [editingId, setEditingId] = useState<string | null>(initialEdit ? "queued-1" : null);
|
||||
const [sent, setSent] = useState<string[]>([]);
|
||||
const [notice, setNotice] = useState<string | null>(null);
|
||||
@@ -76,6 +80,10 @@ function QueuedComposer({ protocol, steeringDisposition, initialEdit, mobile, au
|
||||
<header className="border-b border-border pb-4">
|
||||
<p className="text-xs text-muted-foreground">PAP-204 · Codie is working</p>
|
||||
<h1 className="mt-1 text-base font-semibold">Steering and queued messages</h1>
|
||||
{pendingSave ? <div className="mt-2 flex gap-2">
|
||||
<Button size="sm" onClick={() => { pendingSave.resolve(); setPendingSave(null); }}>Complete save</Button>
|
||||
<Button size="sm" variant="secondary" onClick={() => { pendingSave.reject(); setPendingSave(null); }}>Reject save</Button>
|
||||
</div> : null}
|
||||
</header>
|
||||
<div className="flex-1 space-y-3 py-6 text-sm">
|
||||
<p className="rounded-lg bg-card px-4 py-3">I’m updating the composer and checking the responsive layout.</p>
|
||||
@@ -95,17 +103,38 @@ function QueuedComposer({ protocol, steeringDisposition, initialEdit, mobile, au
|
||||
editingCommentId={editingId}
|
||||
autoSteerDelayMs={autoSteer ? QUEUED_MESSAGE_AUTO_STEER_DELAY_MS : null}
|
||||
/> : null}
|
||||
<div className="relative z-10">
|
||||
<div className="relative z-10" data-testid="queued-composer">
|
||||
<TaskChatComposer
|
||||
onAdd={async (body) => {
|
||||
if (running) {
|
||||
setEntries((current) => [...current, queuedEntry(body, `queued-${Date.now()}`, current.length)]);
|
||||
const id = `queued-${Date.now()}`;
|
||||
setEntries((current) => [...current, queuedEntry(body, id, current.length)]);
|
||||
setNotice("Message queued for the next turn.");
|
||||
if (screenshotSubmission) {
|
||||
try {
|
||||
await new Promise<void>((resolve, reject) => setPendingSave({ resolve, reject: () => reject(new Error("Save rejected")) }));
|
||||
} catch (error) {
|
||||
remove(id);
|
||||
setNotice("Save rejected. The draft is restored.");
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
setSent((current) => [...current, body]);
|
||||
}
|
||||
}}
|
||||
workMode="standard"
|
||||
draftKey={screenshotSubmission ? "storybook-screenshot-submit" : undefined}
|
||||
onAttachImage={screenshotSubmission ? async (file) => {
|
||||
const id = `00000000-0000-4000-8000-00000000000${uploadIndex.current++ % 7}`;
|
||||
return {
|
||||
id, companyId: "storybook", issueId: queue.issueId, issueCommentId: null,
|
||||
assetId: id, provider: "storybook", objectKey: file.name, contentType: file.type,
|
||||
byteSize: file.size, sha256: "0".repeat(64), originalFilename: file.name,
|
||||
createdByAgentId: null, createdByUserId: "storybook-user", createdAt: new Date(), updatedAt: new Date(),
|
||||
contentPath: `/api/attachments/${id}/content`,
|
||||
};
|
||||
} : undefined}
|
||||
onStop={running ? async () => setRunning(false) : undefined}
|
||||
queuedEdit={editingId ? {
|
||||
commentId: editingId,
|
||||
@@ -147,16 +176,19 @@ const meta = {
|
||||
],
|
||||
"status": "current",
|
||||
"implementation": "production",
|
||||
"boundary": "fixtures"
|
||||
"boundary": "fixtures",
|
||||
"notes": "Production composer and queue. ScreenshotSubmission uses static attachment receipts and controls the simulated pending, accepted or rejected save. No backend or provider runs."
|
||||
},
|
||||
},
|
||||
args: { protocol: "paperclip_runner_v1", steeringDisposition: "available", initialEdit: false, mobile: false, autoSteer: false },
|
||||
args: { protocol: "paperclip_runner_v1", steeringDisposition: "available", initialEdit: false, mobile: false, autoSteer: false, screenshotSubmission: false },
|
||||
} satisfies Meta<typeof QueuedComposer>;
|
||||
|
||||
export default meta;
|
||||
type Story = StoryObj<typeof meta>;
|
||||
|
||||
export const QueuedWhileRunning: Story = {};
|
||||
/** Production composer and queue; the buttons control only the simulated save response. */
|
||||
export const ScreenshotSubmission: Story = { args: { screenshotSubmission: true } };
|
||||
export const SteerIntoActiveTurn: Story = {
|
||||
play: async ({ canvasElement }) => {
|
||||
const page = within(canvasElement.ownerDocument.body);
|
||||
|
||||
Reference in new issue
Block a user