Media API

The Media API provides endpoints for text-to-speech synthesis, audio transcription, and YouTube music search. /api/speech is used by in-OS Ryo chat; Maps, Books, TextEdit, Calculator, and the desktop assistant speak via browser speechSynthesis.

All endpoints run on the standalone Bun server and implement rate limiting to prevent abuse.

Text-to-Speech

Convert text to spoken audio using OpenAI or ElevenLabs voice synthesis.

Endpoint

Request

{
  "stability": 0.3,
  "similarity_boost": 0.8,
  "use_speaker_boost": true,
  "speed": 1.1
}

• mp3_44100_128 - MP3 at 44.1kHz, 128kbps

• mp3_22050_32 - MP3 at 22.05kHz, 32kbps
• pcm_16000 - PCM at 16kHz
• pcm_22050 - PCM at 22.05kHz
• pcm_24000 - PCM at 24kHz
• pcm_44100 - PCM at 44.1kHz
• ulaw_8000 - μ-law at 8kHz

Response

Returns audio stream with headers:

{
  "error": "Unauthorized - missing credentials"
}

{
  "error": "'text' is required"
}

{
  "error": "ryo_voice_forbidden",
  "message": "Ryo ElevenLabs voices are limited to Ryo chat speech. Set source to \"ryo-chat\" or use model \"openai\".",
  "source": null
}

{
  "error": "speech_permit_required",
  "message": "A speech permit is required for Ryo ElevenLabs voices."
}

Other permit failures use speech_permit_invalid, speech_permit_expired, speech_permit_exhausted, speech_permit_owner_mismatch, speech_permit_text_mismatch, or speech_permit_hash_mismatch.

Speech permits

{
  "permitId": "uuid",
  "contentHash": "hex sha-256",
  "expiresInSeconds": 300,
  "messageId": "assistant-message-id"
}

{
  "error": "rate_limit_exceeded",
  "scope": "burst" | "daily",
  "limit": 10,
  "windowSeconds": 60,
  "resetSeconds": 45,
  "identifier": "username or anon:ip"
}

Rate Limits

> Note: The authenticated ryo account bypasses speech and permit rate limiting. Other users and admins do not.

Example

curl -X POST https://your-domain.com/api/speech \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer AUTH_TOKEN" \
  -H "X-Username: your-username" \
  -d '{
    "text": "Hello, welcome to ryOS!",
    "model": "elevenlabs",
    "source": "ryo-chat",
    "permitId": "SPEECH_PERMIT_ID",
    "voice_id": "OHP6tMHkOsRKrsDdbPah"
  }' \
  --output speech.mp3

curl -X POST https://your-domain.com/api/speech \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer AUTH_TOKEN" \
  -H "X-Username: your-username" \
  -d '{
    "text": "Hello, welcome to ryOS!",
    "model": "openai",
    "voice": "nova",
    "speed": 1.0
  }' \
  --output speech.mp3

Audio Transcription

Transcribe audio files to text using OpenAI Whisper.

Endpoint

Request

Response

{
  "text": "Transcribed text content here"
}

{
  "error": "No audio file provided"
}

{
  "error": "Invalid file type. Must be an audio file."
}

{
  "error": "File exceeds maximum size of 2MB"
}

{
  "error": "rate_limit_exceeded",
  "scope": "burst" | "daily",
  "limit": 10,
  "windowSeconds": 60,
  "resetSeconds": 45,
  "identifier": "ip:xxx.xxx.xxx.xxx"
}

Rate Limits

Example

curl -X POST https://your-domain.com/api/audio-transcribe \
  -F "[email protected]"

{
  "text": "This is the transcribed text from the audio recording."
}

const formData = new FormData();
formData.append('audio', audioBlob, 'recording.webm');

const response = await fetch('/api/audio-transcribe', {
  method: 'POST',
  body: formData,
});

const { text } = await response.json();
console.log('Transcription:', text);

YouTube Search

Search YouTube for music videos. Results are filtered to the Music category.

Endpoint

Request

Response

{
  "results": [
    {
      "videoId": "dQw4w9WgXcQ",
      "title": "Rick Astley - Never Gonna Give You Up",
      "channelTitle": "Rick Astley",
      "thumbnail": "https://i.ytimg.com/vi/dQw4w9WgXcQ/mqdefault.jpg",
      "publishedAt": "2009-10-25T06:57:33Z"
    }
  ]
}

{
  "error": "Invalid request body"
}

{
  "error": "All YouTube API keys have exceeded their daily quota",
  "code": "youtube_quota_exhausted"
}

{
  "error": "YouTube search is unavailable",
  "code": "youtube_unavailable"
}

The raw Google error is only written to server logs; clients map code to a localized message.

{
  "error": "YouTube API is not configured",
    "hint": "Add YOUTUBE_API_KEY to your .env.local file and restart the API server"
}

{
  "error": "rate_limit_exceeded",
  "scope": "burst" | "daily"
}

Rate Limits

API Key Rotation

The endpoint supports multiple YouTube API keys for quota rotation:

• YOUTUBE_API_KEY - Primary key

• YOUTUBE_API_KEY_2 - Backup key
• YOUTUBE_API_KEY_3 - Second backup key

Keys are tried in order. A request moves on to the next key when Google reports a quota or rate-limit error (HTTP 429, or 403 with reason quotaExceeded, dailyLimitExceeded, rateLimitExceeded, etc.), when the key itself is rejected (invalid, referrer/IP-restricted, API not enabled in that project), or on a network error. Request errors that no key can fix (e.g. invalidParameter) stop immediately. Each failed attempt is logged with its key index, label, HTTP status, and Google reason, never the key. Quota is per Google Cloud project, so keys only add capacity if they belong to different projects.

Example

curl -X POST https://your-domain.com/api/youtube-search \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Taylor Swift Shake It Off",
    "maxResults": 5
  }'

{
  "results": [
    {
      "videoId": "nfWlot6h_JM",
      "title": "Taylor Swift - Shake It Off",
      "channelTitle": "TaylorSwiftVEVO",
      "thumbnail": "https://i.ytimg.com/vi/nfWlot6h_JM/mqdefault.jpg",
      "publishedAt": "2014-08-19T04:00:02Z"
    }
  ]
}

const response = await fetch('/api/youtube-search', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    query: 'Daft Punk Around the World',
    maxResults: 10,
  }),
});

const { results } = await response.json();
results.forEach(video => {
  console.log(`${video.title} - ${video.channelTitle}`);
});

Apple Music Artwork

Resolves Apple Music catalog artwork (and basic metadata) for an am: catalog song ID using Apple's public iTunes Lookup API (no Apple auth required). Apple Music songs cached in Redis often have no cover because client-side MusicKit artwork is never persisted server-side; this endpoint backfills real cover art (used by the admin UI). Results are cached in Redis.

Endpoint

Request

Only numeric catalog IDs can be resolved. Library IDs (i.<hash>), stations (station:…), and playlists (playlist:…) are not catalog songs and return 400.

Response

{
  "cover": "https://.../{w}x{h}bb.jpg",
  "artist": "Artist Name",
  "title": "Track Name",
  "album": "Album Name",
  "url": "https://music.apple.com/..."
}

When no artwork can be resolved, the same shape is returned with cover: null.

{
  "cover": null
}

{
  "error": "Expected an Apple Music catalog id (am:<number>)"
}

Caching

• Resolved artwork is cached for 30 days; "not found" results for 1 day.

• Successful responses send Cache-Control: public, max-age=86400; misses send public, max-age=3600.

Rate Limits

Environment Variables

Related

• Song API - Song library management for karaoke

• Chat API - AI chat with voice integration
• AI System - Overview of AI capabilities