Chat API

The main AI chat endpoint for ryOS that powers the Chats application. This endpoint handles conversational AI interactions with the "Ryo" persona, supports multiple AI providers, and includes extensive tool calling capabilities for controlling the ryOS system.

This endpoint is implemented with the shared apiHandler utility and optional auth via request-auth.

Endpoint

Server conversation history

Authenticated chat and floating assistant conversations are stored separately. The server is the canonical history: a normal authenticated request appends one user action, loads prior messages from Redis, streams one response, then commits only the assembled assistant message. Clients do not resend or overwrite the stored transcript. The server assigns sequence numbers, maintains a monotonic revision, and keeps at most 200 messages within a 4 MiB history budget. Completed AI SDK message parts are retained, including text, tool state, sources, and private image references. A message may contain up to 128,000 text characters and 768 KiB of serialized parts.

Import and reset requests use operation IDs for idempotency. Import additionally requires the current conversation ID and revision 0; it is rejected once the server conversation contains messages or after that conversation has been reset. Reset requires the current conversation ID and returns a new ID. It also schedules memory extraction from the cleared server snapshot exactly once for that reset operation. Conversation records are removed by account deletion.

Realtime cross-device updates

Every canonical conversation write is announced on the owner's authorized private-ai-{username} realtime channel as an ai-conversation-updated event carrying { channel, conversationId, revision, reason, operationId } with reason one of turn-begin, turn-complete, greeting, import, or reset. Other signed-in devices listening on the channel re-hydrate the conversation immediately (the user's message appears at turn-begin, the assistant reply at turn-complete); the originating device recognizes its own operationId and skips the echo. Delivery is best-effort — the focus/visibility refresh remains the catch-up path.

Request

Headers

*Required for authenticated users. Anonymous users are allowed with reduced rate limits.

If one auth header is present without the other, the request is rejected (400).

Query Parameters

Body

interface ChatRequest {
  // AI model to use (optional, defaults to "gpt-5.5")
  model?: SupportedModel;
  
  // Optional proactive greeting mode (non-streaming JSON response)
  proactiveGreeting?: boolean;

  // System state context (optional but recommended)
  systemState?: SystemState;

  // Required for server-synced authenticated turns. Obtain these values from
  // GET /api/ai/conversations/:channel and use a new operationId per request.
  conversation?: {
    id: string;
    revision: number;
    operationId: string;
  };

  // Supplied by AI SDK clients when replacing an assistant response.
  trigger?: "submit-message" | "regenerate-message";
  messageId?: string;

  // Authenticated submit: only the current user message or assistant tool
  // continuation. Omit for regeneration.
  message?: UIMessage;

  // Anonymous requests only: the complete local history.
  messages?: UIMessage[];
}

interface UIMessage {
  role: "user" | "assistant" | "system";
  content: string | MessageContent[];
}

interface MessageContent {
  type: "text" | "image";
  text?: string;
  image?: string; // base64 or URL
}

interface SystemState {
  // User identity
  username?: string | null;
  userOS?: string; // e.g., "iOS", "Android", "macOS", "Windows", "Linux"
  locale?: string; // e.g., "en", "zh-TW", "zh-CN", "ja", "ko"
  
  // User's local time (from browser)
  userLocalTime?: {
    timeString: string;
    dateString: string;
    timeZone: string;
  };
  
  // Running applications context
  runningApps?: {
    foreground: AppInstance | null;
    background: AppInstance[];
  };
  
  // Internet Explorer state
  internetExplorer: {
    url: string;
    year: string;
    currentPageTitle: string | null;
    aiGeneratedMarkdown?: string | null;
  };
  
  // Video player state
  video: {
    currentVideo: TrackInfo | null;
    isPlaying: boolean;
  };
  
  // iPod state (optional)
  ipod?: {
    currentTrack: TrackInfo | null;
    isPlaying: boolean;
    currentLyrics?: {
      lines: Array<{ startTimeMs: string; words: string }>;
    } | null;
  };
  
  // Karaoke state (optional)
  karaoke?: {
    currentTrack: TrackInfo | null;
    isPlaying: boolean;
  };
  
  // TextEdit state (optional)
  textEdit?: {
    instances: Array<{
      instanceId: string;
      filePath: string | null;
      title: string;
      contentMarkdown?: string | null;
      hasUnsavedChanges: boolean;
    }>;
  };
  
  // Chat room context (for chat room @mentions)
  chatRoomContext?: {
    roomId: string;
    recentMessages: string;
    mentionedMessage: string;
  };
}

interface AppInstance {
  instanceId: string;
  appId: string;
  title?: string;
  appletPath?: string;
  appletId?: string;
}

interface TrackInfo {
  id: string;
  title: string;
  artist?: string;
}

Response

The endpoint returns a Server-Sent Events (SSE) stream using the Vercel AI SDK's UI message stream format.

If proactiveGreeting is true (authenticated only), the endpoint returns JSON instead of SSE. The greeting is server-owned: eligibility is decided against the canonical conversation (empty thread, or last message idle for 5+ minutes and not already a greeting, and no turn in flight), and the generated greeting is persisted as a real conversation message (proactive-<uuid> id) so it survives hydration and syncs across devices.

{
  "greeting": "hey, how's the cursor roadmap coming along?",
  "message": {
    "id": "proactive-6f0f…",
    "seq": 12,
    "role": "assistant",
    "parts": [{ "type": "text", "text": "hey, how's the cursor roadmap coming along?" }],
    "createdAt": "2026-07-07T05:00:00.000Z"
  },
  "conversation": { "id": "…", "revision": 4, "…": "…" }
}

When the greeting is skipped (not eligible, no memories, generation failure, or a conflicting write won the race), the response is:

{ "greeting": null, "reason": "conversation_active" }

Stream Format

The response uses Content-Type: text/event-stream and streams data in the following format:

data: {"type":"text-delta","textDelta":"Hello"}
data: {"type":"text-delta","textDelta":" there"}
data: {"type":"tool-call","toolCallId":"...","toolName":"launchApp","args":{...}}
data: {"type":"tool-result","toolCallId":"...","result":{...}}
data: {"type":"finish","finishReason":"stop"}

Error Responses

Authentication is optional. Partial, stale, or invalid credentials are treated as anonymous rather than blocking the chat request.

Rate Limit Error Response

interface RateLimitError {
  error: "rate_limit_exceeded";
  isAuthenticated: boolean;
  count: number;
  limit: number;
  message: string;
}

AI Models

Supported Models

Model Selection

// Priority: Query param > Body param > Default
const model = queryModel || bodyModel || "gpt-5.5";

Rate Limiting

Anonymous users are identified by IP address. The special user "ryo" (with valid token) has unlimited access.

Tool Calling

The Chat API supports extensive tool calling for controlling ryOS applications and system features.

Application Control

launchApp

{
  id: AppId;           // Required: App to launch
  url?: string;        // For internet-explorer: URL to load
  year?: string;       // For internet-explorer: Time-travel year
}

// AppId source of truth: src/config/appRegistryData.ts
type AppId =
  | "finder" | "soundboard" | "internet-explorer" | "chats"
  | "textedit" | "paint" | "photo-booth" | "minesweeper"
  | "videos" | "tv" | "ipod" | "karaoke" | "synth"
  | "terminal" | "applet-viewer" | "control-panels" | "admin"
  | "stickies" | "infinite-mac" | "pc" | "winamp"
  | "calendar" | "contacts" | "dashboard" | "maps"
  | "books" | "calculator";

closeApp

{
  id: AppId;  // App to close
}

Media Control

mediaControl

{
  target?: "music" | "karaoke" | "videos" | "tv"; // default "music"
  // Transport (all targets; TV supports toggle/play/pause only)
  action: "toggle" | "play" | "pause" | "playKnown" | "addAndPlay" | "next" | "previous"
  // TV channel management (target "tv" only)
        | "list" | "tune" | "createChannel" | "deleteChannel" | "addVideo" | "removeVideo";
  id?: string;              // YouTube ID or URL (for addAndPlay/playKnown)
  title?: string;           // Item title (for playKnown)
  artist?: string;          // Artist name (for playKnown)
  enableVideo?: boolean;    // Music only: enable video playback
  enableFullscreen?: boolean;   // Music/karaoke only
  enableTranslation?: string;   // Music/karaoke only: lyrics translation language
  // TV channel-action params
  channelId?: string; channelNumber?: number; prompt?: string;
  name?: string; videoId?: string; url?: string; removeVideoId?: string;
}

searchSongs

// Input
{
  query: string;       // Search query (1-200 chars)
  maxResults?: number; // 1-10, default 5
}

// Output
{
  results: Array<{
    videoId: string;
    title: string;
    channelTitle: string;
    publishedAt: string;
  }>;
  message: string;
  hint: string;
}

Virtual File System

list

{
  path: "/Applets" | "/Documents" | "/Applications" | "/Music" | "/Applets Store";
  query?: string;  // Search filter (for Applets Store)
  limit?: number;  // Max results 1-50 (for Applets Store)
}

open

{
  path: string;  // e.g., "/Applets/Calculator.app", "/Documents/notes.md"
}

read

{
  path: string;  // Path to file in /Applets, /Documents, or /Applets Store
}

write

{
  path: string;     // e.g., "/Documents/notes.md"
  content: string;  // Document content
  mode?: "overwrite" | "append" | "prepend";
}

edit

{
  path: string;       // File path
  old_string: string; // Text to replace (must be unique)
  new_string: string; // Replacement text
}

HTML/Applet Generation

generateHtml

// Input
{
  html: string;    // HTML content (body only)
  title?: string;  // Applet title
  icon?: string;   // Emoji icon
}

// Output (executed server-side)
{
  html: string;
  title: string;
  icon: string;
}

System Settings

settings

{
  language?: "en" | "zh-TW" | "zh-CN" | "ja" | "ko" | "fr" | "de" | "es" | "pt" | "it" | "ru";
  theme?: "system7" | "macosx" | "xp" | "win98";
  masterVolume?: number;     // 0-1
  speechEnabled?: boolean;   // Text-to-speech
  checkForUpdates?: boolean; // Trigger update check
}

Special Tools

aquarium

{}  // No parameters

Example Usage

Basic Chat Request

const history = await fetch('/api/ai/conversations/chat', {
  headers: {
    'Authorization': `Bearer ${authToken}`,
    'X-Username': 'myusername'
  }
}).then(response => response.json());

const response = await fetch('/api/chat', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${authToken}`,
    'X-Username': 'myusername'
  },
  body: JSON.stringify({
    conversation: {
      id: history.conversation.id,
      revision: history.conversation.revision,
      operationId: crypto.randomUUID()
    },
    trigger: 'submit-message',
    message: {
      id: crypto.randomUUID(),
      role: 'user',
      parts: [{ type: 'text', text: 'Hello, Ryo!' }]
    },
    model: 'sonnet-4.6',
    systemState: {
      username: 'myusername',
      locale: 'en',
      internetExplorer: { url: '', year: 'current', currentPageTitle: null },
      video: { currentVideo: null, isPlaying: false }
    }
  })
});

// Handle SSE stream
const reader = response.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  
  const chunk = decoder.decode(value);
  // Process SSE events...
}

Using with Vercel AI SDK

import { useChat } from 'ai/react';

function ChatComponent() {
  const { messages, input, handleInputChange, handleSubmit } = useChat({
    api: '/api/chat',
    headers: {
      'Authorization': `Bearer ${token}`,
      'X-Username': username
    },
    body: {
      model: 'gpt-5.5',
      systemState: { /* ... */ }
    }
  });

  return (
    <form onSubmit={handleSubmit}>
      <input value={input} onChange={handleInputChange} />
      <button type="submit">Send</button>
    </form>
  );
}

Playing a Song via Tool Calling

The AI will automatically use tools when appropriate:

// User message: "Play Never Gonna Give You Up"

// AI will:
// 1. Call list({ path: "/Music" }) to check library
// 2. If not found, call searchSongs({ query: "Never Gonna Give You Up" })
// 3. Call mediaControl({ action: "addAndPlay", id: "dQw4w9WgXcQ" })

Configuration

Model Configuration

const result = streamText({
  model: selectedModel,
  messages: enrichedMessages,
  tools: { /* ... */ },
  temperature: 0.7,
  maxOutputTokens: 48000,
  stopWhen: stepCountIs(10),  // Max 10 tool-calling steps
  experimental_transform: smoothStream({
    chunking: /[\u4E00-\u9FFF]|\S+\s+/,  // CJK-aware chunking
  })
});

Related Endpoints

System Prompt Structure

The Chat API uses a layered system prompt approach:

1. Static System Prompt (cached for performance):
   • Core priority instructions
   • Ryo persona instructions
   • Answer style guidelines
   • Chat instructions
   • Tool usage instructions
   • Code generation instructions
2. Dynamic System Prompt (per-request):
   • User context (username, OS, locale)
   • Time information (Ryo's time + user's local time)
   • Running applications
   • Media playback state
   • Browser state
   • TextEdit documents
   • Chat room context (if applicable)