mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-06 10:48:12 +02:00
feat(ui): integrate agent personas across the app (#13171)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A stable agent persona is useful only when the same identity appears across the app. > - Lists, task messages, selectors, and activity feeds need inexpensive static avatars. > - Onboarding and agent headers need a larger character with expressions and pointer tracking. > - This pull request connects the persona foundation to those existing views and preserves onboarding draft assignments. > - Full-page stories and Linux checks make the placements and performance contract reviewable. ## Linked Issues or Issue Description **Problem or motivation** Agents need a stable visual identity in lists, tasks, onboarding, and configuration. External tools also need an image URL for that identity. **Proposed solution** Assign each agent a permanent palette from a fixed ClipLab character library. Store the assignment on the agent. Render and cache preset PNG URLs on demand. Use static images in dense views and one animated character in larger placements. **Alternatives considered** A generated image bundle requires a separate asset build. A live renderer in every avatar adds unnecessary work in large lists. Arbitrary uploaded images do not provide the requested shared character system. **Roadmap alignment** This improves agent identity across existing control-plane views. It preserves agent permissions, company boundaries, and status labels. ROADMAP.md has no separate ClipLab persona milestone. Related approaches: #2422 adds configurable image URLs and DiceBear generation; #5578 adds optional uploaded avatars. This work uses a fixed, versioned character library and preset URLs. ## What Changed - Replace agent icons with static persona images across lists, the sidebar, org charts, tasks, comments, selectors, activity, and dashboard views. - Put one animated character in the agent header. Let it follow the pointer across the page, with reduced-motion and touch fallbacks. - Add larger padded characters to agent creation. Keep the palette stable across draft refreshes and connection retries, then reveal it after success. - Pass appearance through shared projections rather than fetching each agent separately. - Add real full-page Storybook examples for the agent list, overview, task, dashboard, new-agent dialog, and connection page. - Add Linux screenshot, clipping, density, and 500-avatar performance checks. ## Verification - `pnpm -r typecheck`, `pnpm build`, and token gates pass on the rebased tree. Persona lifecycle tests pass. - The rebased feature passes 38 Linux screenshot/performance checks, including both display densities, corner pointer positions, and the no-WebGL/no-live-download contract for 500 avatars. - The final Linux persona suite passes all 38 visual, lifecycle, density, and full-page checks using the standard Storybook configuration and real on-demand avatar endpoint. - Final local focused verification: 45 avatar/native-recovery tests pass; UI identity/routine tests, typecheck/build, token gates, and Storybook build pass. - Current-head CI passes: full workspace/server tests, all serialized server groups, typecheck/release checks, build, canary validation, and end-to-end shards. The build passed after retrying a native-runner concurrency-test failure; its three targeted cases also pass locally. - Manual inspection covered stable identities in the app, header placement, full-page mouse tracking, onboarding size, and task/dashboard placements. ### Screenshots Linux captures use synthetic Storybook fixtures. Full-page captures use reduced motion. The live character, mouse tracking, and disposal are checked separately. <details> <summary>Agent overview with the character in its header</summary> <img src="https://raw.githubusercontent.com/paperclipai/paperclip/8c68f42b268ada22b67d79d0fe1bb0a2f84ec25c/screenshots/full-page-agent-overview.png" width="900" alt="Agent overview with the character in its header" /> </details> <details> <summary>Task messages and assignee identity</summary> <img src="https://raw.githubusercontent.com/paperclipai/paperclip/8c68f42b268ada22b67d79d0fe1bb0a2f84ec25c/screenshots/full-page-task.png" width="900" alt="Task messages and assignee identity" /> </details> <details> <summary>Larger onboarding character with room for expressions</summary> <img src="https://raw.githubusercontent.com/paperclipai/paperclip/8c68f42b268ada22b67d79d0fe1bb0a2f84ec25c/screenshots/full-page-meet-your-next-agent.png" width="900" alt="Larger onboarding character with room for expressions" /> </details> <details> <summary>Dashboard agent activity</summary> <img src="https://raw.githubusercontent.com/paperclipai/paperclip/8c68f42b268ada22b67d79d0fe1bb0a2f84ec25c/screenshots/full-page-company-dashboard.png" width="900" alt="Dashboard agent activity" /> </details> ## Risks - This PR depends on #13170, the persona foundation. Merge the foundation first, then retarget this PR to master. - Many placements change from icons to character silhouettes. Human avatars and authoritative agent status labels retain their existing behavior. - Only one character can render live per view. Reduced motion, hidden/offscreen content, touch input, and renderer failures use the defined fallbacks. - The full-page stories use fixture data. They do not contact a real company or complete real provider sign-in. ## Model Used OpenAI Codex, GPT-6 family. The exact model identifier and context window are not exposed in this session. Used code editing, shell execution, browser inspection, and Linux visual testing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Tonio <tonework@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
166 files changed
+51724
-436
No files matched your search
@@ -0,0 +1,178 @@
|
||||
# Agent personas
|
||||
|
||||
Agents have a persisted visual identity independent of prompts and runtime
|
||||
configuration: `{ schemaVersion: 1, characterVersion: "cap-v1", paletteId }`.
|
||||
The cap-v1 library contains 17 permanent palettes and a presentation-only gray
|
||||
Muted dream palette. New agents get one random assignment; existing rows are
|
||||
backfilled with the same ID-based mapping used by legacy clients. Duplicating
|
||||
an agent generates another assignment. Export/import preserves it.
|
||||
|
||||
## Rendering and URLs
|
||||
|
||||
`GET /api/agent-avatars/cap-v1/bubblegum-sky/rest.png?size=24&scale=2`
|
||||
|
||||
The endpoint is public preset artwork, contains no agent/company identifiers,
|
||||
and works without the UI. `Agent.avatarUrl` is the 512px resting portrait URL.
|
||||
Resolve relative URLs against the instance base URL for integrations.
|
||||
|
||||
Supported logical sizes: 16, 20, 24, 32, 40, 48, 64, 96, 128, 256, 512.
|
||||
Density is 1 or 2. Poses: rest, idle, listening, thinking, working, success,
|
||||
confused, sleepy, loading. Other inputs receive 400. Display size controls
|
||||
face detail separately from raster dimensions: 24px at 2× remains eyes-only.
|
||||
|
||||
A cold request samples ClipLab's deterministic scene, exports SVG using the
|
||||
same geometry and face code as the live renderer, and rasterizes it with sharp
|
||||
in a worker thread. No Chromium, GPU, pre-rendering command, or per-agent asset
|
||||
row is required. Two workers serve a bounded queue, coalesce identical requests
|
||||
within the process, time out failed rendering, and stop after an idle interval.
|
||||
|
||||
The configured local-disk/S3 provider stores results in
|
||||
`generated-agent-avatars/<version>/<palette>/<pose>-<size>-<scale>.png`.
|
||||
This preset-only cache intentionally uses the storage provider directly;
|
||||
company asset APIs retain their existing authorization boundary. Cache data
|
||||
can be removed and regenerates on demand. Independent replicas can render the
|
||||
same key safely; successful writes are complete objects. ETags hash PNG bytes.
|
||||
A small `.png.json` sidecar stores the content digest and length; it is published
|
||||
after the complete PNG. Warm requests stream stored PNG bytes without re-rendering
|
||||
or buffering the image in the API process. Responses are immutable for one year.
|
||||
Cold renders are limited per client IP (32 outstanding keys and 256 new keys
|
||||
per minute per process); excess misses return 429 with Retry-After and no-store.
|
||||
Warm cache hits and requests joining the same in-flight key bypass admission.
|
||||
The route uses Express trust-proxy configuration, never an untrusted forwarded header. Render/storage failures return 503 with
|
||||
Retry-After and no-store rather than caching a broken image.
|
||||
|
||||
cap-v1 is frozen: change the version when changing palette values, poses,
|
||||
rendering, or dependencies in a way that changes pixels. Keep old versions
|
||||
available. The SVG exporter approximates 3D gradients with a planar gradient;
|
||||
front-facing identity portraits minimize the difference from WebGL.
|
||||
|
||||
## Components
|
||||
|
||||
- `AgentAvatar`: image only; pass the agent record or appearance and a semantic
|
||||
size. No per-agent queries, live-renderer imports, or circle cropping.
|
||||
- `AgentIdentity`: agent avatar and name. Human identities keep `Identity`.
|
||||
- `AgentCharacter`: lazy live hero with state and optional tracking-region
|
||||
props, plus an explicit `trackingScope="page"` for onboarding and agent headers. One live renderer per view; other instances keep their still image.
|
||||
Reduced motion, offscreen content, renderer failure, and static states do
|
||||
not run the animation loop. Pointer tracking defaults to its region and is off for touch.
|
||||
|
||||
Onboarding stores the eventual palette in its existing draft and presents gray
|
||||
until verified connection/hiring succeeds. Reconnect never randomizes identity.
|
||||
Names, explicit status badges, and status text remain authoritative.
|
||||
|
||||
## Development and verification
|
||||
|
||||
ClipLab provenance/licenses are under `packages/shared/src/cliplab/`.
|
||||
Palette tokens originate in `ui/src/index.css`. After deliberately introducing
|
||||
new versioned artwork, `node scripts/sync-agent-palette-tokens.mjs` synchronizes
|
||||
the TS palette data; `--check` detects drift. This does not generate images.
|
||||
|
||||
Storybook: **Agents / Personas**. Run with
|
||||
`PAPERCLIP_STORYBOOK_API_URL=http://localhost:<isolated-port> pnpm storybook`.
|
||||
`pnpm build-storybook` automatically packages all finite avatar presets (17
|
||||
palettes plus muted gray, nine poses, eleven logical sizes, both densities).
|
||||
The build uses the same bounded Node worker pool, SVG renderer, and Sharp pipeline
|
||||
as the API. Storybook-only URL resolution points to relative PNG paths under the
|
||||
published build, including branch-prefixed deployments. Production Paperclip
|
||||
continues to use its on-demand API; no image generation runs during agent creation.
|
||||
The generated files are build output, never committed. A manifest records image
|
||||
hashes and pixel dimensions; deployment verification fetches every image and checks
|
||||
its content type, PNG signature, dimensions, and hash. Dev Storybook still uses the
|
||||
API proxy for cold-cache and regeneration testing.
|
||||
|
||||
Focused checks:
|
||||
|
||||
```sh
|
||||
pnpm exec vitest run packages/shared/src/agent-appearance.test.ts server/src/__tests__/agent-avatars.test.ts
|
||||
node scripts/sync-agent-palette-tokens.mjs --check
|
||||
pnpm check:token-gates
|
||||
pnpm build-storybook
|
||||
```
|
||||
|
||||
The 500-avatar story must create zero WebGL contexts and load no live runtime.
|
||||
Use fixed poses/times for screenshots and Linux for authoritative visual
|
||||
baselines. Verify cold and warm URLs, reduced motion, reconnect, and saved
|
||||
appearance after refresh alongside normal typecheck/test/build checks.
|
||||
|
||||
Linux visual/performance checks (against the self-contained built Storybook):
|
||||
|
||||
```sh
|
||||
pnpm exec playwright test --config tests/storybook-visual/agent-personas.config.ts
|
||||
```
|
||||
|
||||
Use the Playwright 1.62.1 Noble image for authoritative Linux baselines. The suite
|
||||
covers both themes, palette and size grids, every expression, fixed-pose SVG/WebGL
|
||||
pixel comparisons, repeated mounts, context loss, delayed/failed images, and the
|
||||
500-avatar no-WebGL/no-live-download/no-frame-loop contract. Baselines follow the
|
||||
existing Storybook visual artifact workflow; they are not application assets.
|
||||
|
||||
## Acceptance record — 2026-09-10
|
||||
|
||||
Implemented in `codex/agent-personas`, based on `5cb4f061d`, with the original
|
||||
checkout left unchanged. Verification used disposable embedded-Postgres data,
|
||||
a local MinIO container, and Playwright's Linux Noble image.
|
||||
|
||||
| Check | Result |
|
||||
| --- | --- |
|
||||
| Repository typecheck and build | Passed |
|
||||
| Token gates and palette-token synchronization | Passed |
|
||||
| Storybook production build | Passed |
|
||||
| Linux visual/performance suite | 30 passed; exact snapshot comparison passed |
|
||||
| Static/live agreement | Rest at 16/24/48/256 logical pixels, both densities; all eight animated expression snapshots at 128px/2× |
|
||||
| Avatar endpoint | Cold/warm requests, concurrency, deletion, retry, ETags and invalid parameters passed |
|
||||
| Real local-disk and S3-compatible storage | Passed; persisted warm cache reused by a fresh service instance |
|
||||
| Compiled API-only smoke | Passed with TypeScript stripping disabled, no UI and a native worker |
|
||||
| Persistence and contracts | Creation, saved draft, SQL backfill, approvals, duplication, portability and revision restoration passed |
|
||||
| UI/runtime lifecycle | Reduced motion, one live owner, scoped pointers, hidden/offscreen suspension, failures and disposal passed |
|
||||
| Hands-on app | Stable rename/reload, fresh duplicate assignment, paused static portrait, matching task/list/configuration identities passed |
|
||||
|
||||
The broad `pnpm test:run` verification was completed in its groups/shards after
|
||||
resource-contention retries. General server (8,347 tests), UI (5,619), CLI (484),
|
||||
DB (133), and adapter/plugin source suites (2,235) passed. Remaining route suites
|
||||
and the added persona/contract tests passed after the OpenAPI coverage update.
|
||||
The queued-comment suite exposed a pre-existing fixture-cleanup failure: run
|
||||
claims created dependent runtime rows, so swallowed foreign-key errors left the
|
||||
`QUE` company behind. Its isolated-database cleanup now truncates the company
|
||||
fixture graph with cascade; all 17 tests pass together. No production queued-comment
|
||||
behavior changed. Verification completed across the repository runner's groups
|
||||
and shards, with targeted reruns after fixes, rather than another monolithic run.
|
||||
|
||||
Actual provider sign-in was not completed: the disposable instance had no managed
|
||||
sandbox available for sign-in. Onboarding gray/loading/success transitions were
|
||||
covered in Storybook and the onboarding tests. The app is usable in local-trusted
|
||||
mode; this standalone worktree has no managed issue identity or login handoff.
|
||||
|
||||
The reviewed Linux candidate archive is generated at
|
||||
`tests/storybook-visual/baseline-review/snapshots.tgz`. It remains a local review
|
||||
artifact; the existing baseline manifest was not repointed to an unpublished URL.
|
||||
|
||||
## Placement and sharpness refinement — 2026-09-10
|
||||
|
||||
The overview/configuration header owns the live character beside the agent name
|
||||
and follows the pointer across the whole page; there is no second hero in the
|
||||
overview body. The new-agent dialog and setup page
|
||||
use a larger, padded character frame with page-wide mouse tracking. Other
|
||||
placements retain region-scoped tracking. Touch, reduced motion, hidden views,
|
||||
and unmount cleanup still disable tracking and frame scheduling.
|
||||
|
||||
Live canvases render at twice the display density (capped at 4×), with a 1024px
|
||||
face texture and padded framing for rotations and expression props. This affects
|
||||
only the live renderer: existing versioned PNG URLs retain their original pixels.
|
||||
The Snapshot Agreement story provides explicit 1×/2× controls and labels the
|
||||
actual PNG and WebGL pixel dimensions for a fair comparison.
|
||||
|
||||
`Agents / Personas / Full pages` includes the actual application shell and route
|
||||
components for all agents, agent overview, task detail, dashboard, the new-agent
|
||||
dialog, and the connection page. Fixtures stay in Storybook; these examples do
|
||||
not read or mutate the running company's data. Dashboard activity rows now use
|
||||
the same agent avatars as its active-agent and task placements.
|
||||
|
||||
Refinement verification: 38 Linux Playwright checks passed, including exact
|
||||
comparison with the reviewed snapshots for all six full-page stories and
|
||||
corner-pointer clipping checks at 1×/2× display density. The 500-avatar view
|
||||
still loads no live renderer or WebGL contexts. All 18 targeted appearance,
|
||||
component/runtime, and avatar endpoint tests passed, as did shared/UI typechecks,
|
||||
token gates, UI build, and Storybook build. Hands-on inspection confirmed the
|
||||
single animated header at `/PER/agents/persona-tester-renamed/overview` and the
|
||||
larger onboarding character in the real route components. The repository-wide
|
||||
checks above were not repeated for this UI refinement.
|
||||
Reference in new issue
Block a user