mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-13 03:38:38 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - An agent needs personal files across tasks and sessions. > - AGENTS.md is one file in that directory. Supporting files need the same persistence. > - The Instructions Editor and agent runs must share one current directory. > - Concurrent runs should apply only the files they change. The last sync of the same file wins. > - This pull request uses existing file transport and removes temporary copies after sync. > - Old instruction-only sessions keep their restore contract. New saves do not create revision history. ## Linked Issues or Issue Description Refs #14325. This replaces its revision-oriented design with persistent agent files. Keep #14325 unmerged. Transport prerequisite #14416 merged first at `d172197117a14b80a1eb2d2835a0e7cce2679656`. This PR now targets master and remains below 100 changed files. Related work: #4513 and #8798 cover instruction tooling. This change handles run synchronization, cross-task personal files, browser editing, and old-session restoration. ## What Changed - Keep one current directory per company and agent. Point AGENT_HOME at a temporary working copy for each active run. Keep task files and provider HOME separate. - Restore text, binary files, and nested folders through workspace transport. Exclude remote agent files from task Git snapshots with a self-ignoring file inside the reserved runtime directory; never write through repository-controlled Git metadata. - Collect after the provider and child processes have stopped. Keep resumable conversation state. - Apply changed and deleted files under the agent lock. The last sync wins for the same file. Unrelated concurrent changes survive. - Remove temporary copies after successful sync, rejected sync, and staging failure. Register ownership before copying so restart recovery can remove interrupted preparation. Retry transient synchronization up to three times. Preserve the original remote lease reference until deletion succeeds; restart cleanup never acquires a replacement sandbox. Do not create captured directories or a conflict-review queue for new runs. - Keep browser editing, stale-draft protection, and streaming binary downloads. Keep the instruction entry and text editor limited to 1 MiB. - Keep historical agent-folder sync failures on their affected runs instead of repeating them above current saved instructions. Preserve legacy candidate review and current browser-save errors. Avoid duplicate quota warnings while retaining separate sync failures when they describe a different problem. - Require target-scoped caller grants for peer instruction access, while preserving self edits, responsible-user checks, and protected-change consent. - Treat full storage as a nonblocking run warning, never an agent pause or run-admission failure. Restore already-over-quota saved folders so ordinary agent cleanup can recover; warn on each run until cleanup. The run detail view shows the warning. - Allow 256 MiB per file, 2 GiB per directory, and 100,000 entries. Hash large files as streams. Check editor-save quotas with metadata instead of hashing unrelated files. - Preserve old native inputs, instruction-only copies, paths, digests, and pending legacy candidates. Adopt old revision heads once. New writes do not append history rows. - Add idempotent migration 0287 and verify upgrades from the preview tables and receipts. - Add nine interactive stories under **Agents / Persistent files**, including automatic incoming edits, stale browser drafts, and storage-limit diagnostics. ## Verification - Merge candidate: `4f5390107ec6ffd80a76d1d2e85530e66f21d079`, after merging current master and the landed transport prerequisite. Integration required no manual conflict resolution; the feature remains 99 changed files. Full workspace typecheck, production build, token gates, and 715 focused tests passed on this merge candidate. Fresh Greptile review is 5/5 with no unresolved findings. All 55 checks passed, with four conditional skips, including the build, typecheck, browser E2E, and canary dry run. A single retry recovered four jobs interrupted by runner shutdowns; no source changes were required. - Historical-warning UI fix: all 6,834 UI tests across 640 files passed, including regression coverage for three old failures, legacy preserved edits, and warnings scoped to the affected run. Full workspace typecheck, production build, Storybook build, and token gates passed. Browser-verified Storybook playtests passed for Historical Failures After Successful Save, Storage Limit, and Full Storage Run Warning. - Review follow-ups at `4e20c9fb2`: all 18 focused tests passed, including external Git directories, linked worktrees, symlinks, hardlinks, and distinct I/O failures alongside storage warnings. Server and UI typechecks, token gates, and the production build passed. - Storage warning regressions at `0724f3012`: all 33 directory tests and all five heartbeat-list tests passed, with no skips in their successful runs. They cover repeated runs while full, an already-over-quota saved folder, cleanup, warnings retained after unrelated save failures, and bounded warnings in large result JSON. Server typecheck passed after the final warning fixes. - Full workspace typecheck, production build, and token gates passed during this follow-up. Product E2E harness: 631 tests passed across 52 files; harness typecheck passed. Earlier native session/context and directory/legacy collection suites passed 537 tests; Runner unit/transport suites passed 329 tests. - **Real E2E at `0724f3012` (before this follow-up):** legacy local Codex and native Daytona Codex each passed six tasks, one server restart, seven independent assertions, and cleanup verification. Both prove browser-to-agent edits, agent-to-browser edits, nested/binary restoration, per-file last-sync-wins, a successful run after an oversized save rejection, and cleanup clearing the warning. - Native local Codex also passed the six-task quota flow before the final warning-retention fixes. That pass began at `918d1ed02` while the bounded-result warning fix was being edited, so it is not claimed as exact-final-head evidence. Its final-head rerun failed during embedded PostgreSQL bootstrap before any provider run: the macOS host had 87,365 of 87,381 SysV semaphores occupied. No unrelated services or kernel limits were changed. - The final-source report intentionally records **2/3 cells passed**, preserving the blocked native-local attempt: `tests/runner-e2e/results/agent-files-quota-final-20260928-report/`. Earlier failed attempts and provenance notes remain under `tests/runner-e2e/results/agent-files-quota-final-20260928-input/` and the original campaign directories. - Daytona used immutable image `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:5643f0d801417cae3581833a1a3bc6715b325e028602738d2652c44cac5dc6bf` and its exact Linux runner binary. Controller source is `0724f3012`; image source is recorded separately. - Legacy-session compatibility and all three ACP Stop/resume browser regressions passed on the prior validated feature head `169fab46d5af21caa2269b4c1b29b69c933a6951`. They assert the same provider session is retained and interrupted writes are not replayed. Migration upgrade tests also passed earlier. - Nine interactive stories are under **Agents / Persistent files**, including **Full Storage Run Warning**. Its playtest and visual browser inspection passed; the warning states that runs continue and the editor remains available. - Prior-head checks on `4e20c9fb2`: 55 passed, two conditional jobs skipped, no failures or pending checks. All eight browser E2E shards and their aggregate passed. Fresh Greptile review is 5/5 with no findings; all review threads are resolved, the security scan passed, and GitHub reports no merge conflicts. - The broad local follow-up test run was interrupted after host semaphore exhaustion affected isolated PostgreSQL instances. It also encountered the existing macOS long-path fixture failure and two timeout failures. This is not a claim that the full local suite passed. Logs are retained; focused storage/warning tests passed. ## Risks - A later sync can overwrite an earlier edit to the same file, including a saved browser edit. There is no text merge or retained version. This is the intended last-sync-wins policy. - A save that exceeds a storage limit is rejected and its temporary copy is discarded. The run itself continues normally, and later runs restore the last saved files with a warning until cleanup. Transient sync failures get bounded retries. An I/O failure partway through a sync can leave some files updated; a failed receipt does not claim whole-folder success. - Larger folders increase copy time, network traffic, and temporary disk usage. Active runs still need working copies. Terminal runs do not accumulate archives. Operators must provision disk for agents and configured concurrency; these limits are not company-wide quotas. - A restored old native session remains instruction-only until a fresh session starts. Its original conflict fence and existing pending candidates remain compatible. - Provider processes close at the collection boundary. Conversation resume remains available, but warm process reuse is lost. - Backups must include the instance filesystem and database. External bundles keep their existing behavior until explicitly moved to managed storage. ## Model Used OpenAI Codex, GPT-6 family. The session does not expose a more specific model ID or context-window size. Reasoning, code execution, and browser tools assisted this change. Real provider E2E uses `gpt-5.6-sol`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Fry (Paperclip) <noreply@paperclip.ing>
306 lines
12 KiB
TypeScript
306 lines
12 KiB
TypeScript
import { agentAppearanceSchema } from "../agent-appearance.js";
|
|
import { aiConnectionBindingSchema } from "../ai-connections.js";
|
|
import { z } from "zod";
|
|
import {
|
|
AGENT_ICON_NAMES,
|
|
AGENT_ROLES,
|
|
AGENT_STATUSES,
|
|
INBOX_MINE_ISSUE_STATUS_FILTER,
|
|
} from "../constants.js";
|
|
import { agentAdapterTypeSchema } from "../adapter-type.js";
|
|
import { envConfigSchema } from "./secret.js";
|
|
import { trustAuthorizationPolicySchema, trustPresetSchema } from "./trust-policy.js";
|
|
import { agentDesiredSkillSelectionSchema } from "./adapter-skills.js";
|
|
import { objectWithoutDefaults } from "./partial.js";
|
|
|
|
export const agentPermissionsSchema = z.object({
|
|
// No schema default: the server derives the default (enabled unless the
|
|
// permissions record marks the agent low-trust) when the field is omitted.
|
|
canCreateAgents: z.boolean().optional(),
|
|
canCreateSkills: z.boolean().optional().default(true),
|
|
trustPreset: trustPresetSchema.optional(),
|
|
authorizationPolicy: trustAuthorizationPolicySchema.optional(),
|
|
}).catchall(z.unknown());
|
|
|
|
export const agentInstructionsBundleModeSchema = z.enum(["managed", "external"]);
|
|
|
|
export const updateAgentInstructionsBundleSchema = z.object({
|
|
mode: agentInstructionsBundleModeSchema.optional(),
|
|
rootPath: z.string().trim().min(1).nullable().optional(),
|
|
entryFile: z.string().trim().min(1).optional(),
|
|
clearLegacyPromptTemplate: z.boolean().optional().default(false),
|
|
});
|
|
|
|
export type UpdateAgentInstructionsBundle = z.infer<typeof updateAgentInstructionsBundleSchema>;
|
|
|
|
export const upsertAgentInstructionsFileSchema = z.object({
|
|
path: z.string().min(1).max(512),
|
|
content: z.string().max(1024 * 1024),
|
|
baseRevisionId: z.string().uuid().nullable().optional(),
|
|
baseHash: z.string().regex(/^[a-f0-9]{64}$/).nullable().optional(),
|
|
clearLegacyPromptTemplate: z.boolean().optional().default(false),
|
|
}).strict();
|
|
|
|
export const restoreAgentInstructionSchema = z.object({
|
|
path: z.string().min(1).max(512),
|
|
revisionId: z.string().uuid(),
|
|
baseRevisionId: z.string().uuid(),
|
|
}).strict();
|
|
|
|
export type UpsertAgentInstructionsFile = z.infer<typeof upsertAgentInstructionsFileSchema>;
|
|
|
|
const adapterConfigSchema = z.record(z.string(), z.unknown()).superRefine((value, ctx) => {
|
|
const envValue = value.env;
|
|
if (envValue === undefined) return;
|
|
const parsed = envConfigSchema.safeParse(envValue);
|
|
if (!parsed.success) {
|
|
ctx.addIssue({
|
|
code: z.ZodIssueCode.custom,
|
|
message: "adapterConfig.env must be a map of valid env bindings",
|
|
path: ["env"],
|
|
});
|
|
}
|
|
});
|
|
|
|
export const createAgentInstructionsBundleSchema = z.object({
|
|
entryFile: z.string().trim().min(1).optional(),
|
|
files: z.record(z.string(), z.string()).refine((files) => Object.keys(files).length > 0, {
|
|
message: "instructionsBundle.files must contain at least one file",
|
|
}),
|
|
});
|
|
|
|
export const agentRuntimeConfigSchema = z.object({
|
|
aiConnection: aiConnectionBindingSchema.optional(),
|
|
debug: z.object({
|
|
providerTrace: z.literal("raw").optional(),
|
|
}).strict().optional(),
|
|
}).catchall(z.unknown()).superRefine((value, ctx) => {
|
|
if (Object.prototype.hasOwnProperty.call(value, "modelProfiles")) {
|
|
ctx.addIssue({
|
|
code: z.ZodIssueCode.custom,
|
|
path: ["modelProfiles"],
|
|
message: "runtimeConfig.modelProfiles is no longer supported",
|
|
});
|
|
}
|
|
});
|
|
|
|
export const createAgentSchema = z.object({
|
|
name: z.string().min(1),
|
|
role: z.enum(AGENT_ROLES).optional().default("general"),
|
|
title: z.string().optional().nullable(),
|
|
icon: z.enum(AGENT_ICON_NAMES).optional().nullable(),
|
|
appearance: agentAppearanceSchema.optional(),
|
|
reportsTo: z.string().guid().optional().nullable(),
|
|
capabilities: z.string().optional().nullable(),
|
|
desiredSkills: z.array(agentDesiredSkillSelectionSchema).optional(),
|
|
adapterType: agentAdapterTypeSchema,
|
|
adapterConfig: adapterConfigSchema.optional().default({}),
|
|
instructionsBundle: createAgentInstructionsBundleSchema.optional(),
|
|
runtimeConfig: agentRuntimeConfigSchema.optional().default({}),
|
|
defaultEnvironmentId: z.string().guid().optional().nullable(),
|
|
budgetMonthlyCents: z.number().int().nonnegative().optional().default(0),
|
|
permissions: agentPermissionsSchema.optional(),
|
|
metadata: z.record(z.string(), z.unknown()).optional().nullable(),
|
|
// The optional stored-session claim from a completed Claude login session. It
|
|
// is the non-secret `storedSessionId`; it carries no token. The agent-create
|
|
// transaction consumes it as the one-time stored-session claim.
|
|
storedSessionId: z.string().min(1).max(256).optional(),
|
|
// The optional apply-existing flag. When true, the caller binds the fixed
|
|
// Claude OAuth token reference to the owner stored value with no new login
|
|
// round trip. The server permits the no-claim bind only for a user actor and
|
|
// only when that owner already has a stored value. It carries no token.
|
|
applyStoredClaudeLogin: z.boolean().optional(),
|
|
// Narrow intent flag set by the onboarding wizard when it hires the very first
|
|
// agent (the chief of staff). It is not an agent column: the server consumes
|
|
// it to seed the server-owned chief-of-staff persona over the agent's entry
|
|
// instruction file instead of the generic default, and honors it only for
|
|
// board-authored requests. Mirrors onboardingFirstTask on issue create.
|
|
onboardingFirstAgent: z.boolean().optional(),
|
|
});
|
|
|
|
export type CreateAgent = z.infer<typeof createAgentSchema>;
|
|
|
|
export const builtInAgentProvisionSchema = z.object({
|
|
adapterType: agentAdapterTypeSchema.optional(),
|
|
adapterConfig: adapterConfigSchema.optional(),
|
|
budgetMonthlyCents: z.number().int().nonnegative().optional(),
|
|
}).strict();
|
|
|
|
export type BuiltInAgentProvision = z.infer<typeof builtInAgentProvisionSchema>;
|
|
|
|
export const builtInAgentEmptyMutationSchema = z.object({}).strict().default({});
|
|
|
|
export type BuiltInAgentEmptyMutation = z.infer<typeof builtInAgentEmptyMutationSchema>;
|
|
|
|
export const builtInAgentResetSchema = z.object({
|
|
resources: z.array(z.enum(["agent", "instructions", "skill", "routine"])).optional(),
|
|
}).strict().default({});
|
|
|
|
export type BuiltInAgentReset = z.infer<typeof builtInAgentResetSchema>;
|
|
|
|
export const createAgentHireSchema = createAgentSchema.extend({
|
|
sourceIssueId: z.string().guid().optional().nullable(),
|
|
sourceIssueIds: z.array(z.string().guid()).optional(),
|
|
// Agent-authored hires may explicitly request the caller's native runner
|
|
// settings. The server consumes this intent; it is never an agent column.
|
|
inheritRuntimeFrom: z.literal("caller").optional(),
|
|
});
|
|
|
|
export type CreateAgentHire = z.infer<typeof createAgentHireSchema>;
|
|
|
|
export const updateAgentSchema = objectWithoutDefaults(
|
|
createAgentSchema.omit({ permissions: true, onboardingFirstAgent: true }),
|
|
)
|
|
.partial()
|
|
.extend({
|
|
permissions: z.never().optional(),
|
|
replaceAdapterConfig: z.boolean().optional(),
|
|
status: z.enum(AGENT_STATUSES).optional(),
|
|
spentMonthlyCents: z.number().int().nonnegative().optional(),
|
|
});
|
|
|
|
export type UpdateAgent = z.infer<typeof updateAgentSchema>;
|
|
|
|
export const updateAgentInstructionsPathSchema = z.object({
|
|
path: z.string().trim().min(1).nullable(),
|
|
adapterConfigKey: z.string().trim().min(1).optional(),
|
|
});
|
|
|
|
export type UpdateAgentInstructionsPath = z.infer<typeof updateAgentInstructionsPathSchema>;
|
|
|
|
export const taskBridgeAgentKeyScopeSchema = z.object({
|
|
kind: z.literal("task_bridge"),
|
|
projectId: z.string().guid().optional().nullable(),
|
|
projectIds: z.array(z.string().guid()).max(50).optional(),
|
|
parentIssueId: z.string().guid().optional().nullable(),
|
|
parentIssueIds: z.array(z.string().guid()).max(50).optional(),
|
|
allowedAssigneeAgentIds: z.array(z.string().guid()).max(50).optional(),
|
|
}).strict().superRefine((value, ctx) => {
|
|
const hasProjectBoundary = Boolean(value.projectId) || Boolean(value.projectIds?.length);
|
|
const hasParentBoundary = Boolean(value.parentIssueId) || Boolean(value.parentIssueIds?.length);
|
|
if (!hasProjectBoundary && !hasParentBoundary) {
|
|
ctx.addIssue({
|
|
code: z.ZodIssueCode.custom,
|
|
message: "task_bridge keys require at least one project or parent issue boundary",
|
|
path: ["projectId"],
|
|
});
|
|
}
|
|
});
|
|
|
|
export const standardAgentKeyScopeSchema = z.object({
|
|
kind: z.literal("standard"),
|
|
}).strict();
|
|
|
|
export const skillTestAgentKeyScopeSchema = z.object({
|
|
kind: z.literal("skill_test"),
|
|
issueId: z.string().guid(),
|
|
}).strict();
|
|
|
|
export const agentApiKeyScopeSchema = z.union([
|
|
standardAgentKeyScopeSchema,
|
|
taskBridgeAgentKeyScopeSchema,
|
|
skillTestAgentKeyScopeSchema,
|
|
]);
|
|
|
|
export type AgentApiKeyScope = z.infer<typeof agentApiKeyScopeSchema>;
|
|
export type TaskBridgeAgentKeyScope = z.infer<typeof taskBridgeAgentKeyScopeSchema>;
|
|
export type SkillTestAgentKeyScope = z.infer<typeof skillTestAgentKeyScopeSchema>;
|
|
|
|
export function normalizeAgentApiKeyScope(value: unknown): AgentApiKeyScope {
|
|
const parsed = agentApiKeyScopeSchema.safeParse(value);
|
|
return parsed.success ? parsed.data : { kind: "standard" };
|
|
}
|
|
|
|
export const createAgentKeySchema = z.object({
|
|
name: z.string().min(1).default("default"),
|
|
scope: agentApiKeyScopeSchema.optional().default({ kind: "standard" }),
|
|
});
|
|
|
|
export type CreateAgentKey = z.infer<typeof createAgentKeySchema>;
|
|
|
|
export const agentMineInboxQuerySchema = z.object({
|
|
userId: z.string().trim().min(1),
|
|
status: z.string().trim().min(1).optional().default(INBOX_MINE_ISSUE_STATUS_FILTER),
|
|
});
|
|
|
|
export type AgentMineInboxQuery = z.infer<typeof agentMineInboxQuerySchema>;
|
|
|
|
export const wakeAgentSchema = z.object({
|
|
source: z
|
|
.enum(["timer", "assignment", "on_demand", "automation"])
|
|
.optional()
|
|
.default("on_demand"),
|
|
triggerDetail: z.enum(["manual", "ping", "callback", "system"]).optional(),
|
|
reason: z.string().optional().nullable(),
|
|
/** Select an exact failed run; its chat request and actor are server-derived. */
|
|
failedRunId: z.string().uuid().optional(),
|
|
payload: z.record(z.string(), z.unknown()).optional().nullable(),
|
|
idempotencyKey: z.string().optional().nullable(),
|
|
forceFreshSession: z.preprocess(
|
|
(value) => (value === null ? undefined : value),
|
|
z.boolean().optional().default(false),
|
|
),
|
|
debug: z
|
|
.object({
|
|
providerTrace: z.literal("raw"),
|
|
})
|
|
.strict()
|
|
.optional(),
|
|
});
|
|
|
|
export type WakeAgent = z.infer<typeof wakeAgentSchema>;
|
|
|
|
export const resetAgentSessionSchema = z.object({
|
|
taskKey: z.string().min(1).optional().nullable(),
|
|
});
|
|
|
|
export type ResetAgentSession = z.infer<typeof resetAgentSessionSchema>;
|
|
|
|
export const testAdapterEnvironmentSchema = z.object({
|
|
aiConnection: aiConnectionBindingSchema.optional(),
|
|
/** Saved agent whose redacted environment entries are restored for this probe. */
|
|
agentId: z.string().guid().optional(),
|
|
/** One-shot provider keys for a probe. Never persist these in agent config. */
|
|
testCredentials: z.object({
|
|
ANTHROPIC_API_KEY: z.string().max(16384),
|
|
OPENAI_API_KEY: z.string().max(16384),
|
|
OPENROUTER_API_KEY: z.string().max(16384),
|
|
GEMINI_API_KEY: z.string().max(16384),
|
|
XAI_API_KEY: z.string().max(16384),
|
|
GROQ_API_KEY: z.string().max(16384),
|
|
OPENCODE_API_KEY: z.string().max(16384),
|
|
CURSOR_API_KEY: z.string().max(16384),
|
|
KIMI_MODEL_API_KEY: z.string().max(16384),
|
|
API_SERVER_KEY: z.string().max(16384),
|
|
ZAI_API_KEY: z.string().max(16384),
|
|
KIMI_API_KEY: z.string().max(16384),
|
|
MINIMAX_API_KEY: z.string().max(16384),
|
|
}).partial().strict().optional(),
|
|
adapterConfig: adapterConfigSchema.optional().default({}),
|
|
/**
|
|
* Optional environment to run the adapter test inside. When omitted, the
|
|
* test runs against the local Paperclip host. When provided and the
|
|
* environment is non-local (SSH/sandbox), the test probes are executed
|
|
* inside that environment so the result reflects real agent execution.
|
|
*/
|
|
environmentId: z.string().guid().optional().nullable(),
|
|
});
|
|
|
|
export type TestAdapterEnvironment = z.infer<typeof testAdapterEnvironmentSchema>;
|
|
|
|
export const updateAgentPermissionsSchema = z.object({
|
|
canCreateAgents: z.boolean(),
|
|
canCreateSkills: z.boolean().optional(),
|
|
canAssignTasks: z.boolean(),
|
|
trustPreset: trustPresetSchema.optional(),
|
|
authorizationPolicy: trustAuthorizationPolicySchema.optional(),
|
|
});
|
|
|
|
export type UpdateAgentPermissions = z.infer<typeof updateAgentPermissionsSchema>;
|
|
|
|
export const resolveAgentInstructionCandidateSchema = z.object({
|
|
baseRevisionId: z.string().uuid().nullable(),
|
|
content: z.string().max(1024 * 1024),
|
|
}).strict();
|
|
export type ResolveAgentInstructionCandidate = z.infer<typeof resolveAgentInstructionCandidateSchema>;
|