Video Sources

The JavaScript SDK supports five video input sources. All source types use native LiveKit transport under the hood for reliable connectivity.

Source Types

Source	Config	Requires User Gesture	Notes
Camera	`{ type: 'camera', cameraFacing: 'user' \| 'environment' }`	Yes (permission prompt)	Default source. Uses `getUserMedia`.
Video file	`{ type: 'video', file: videoFile }`	No	From `<input type="file">`. Loops continuously.
Screen sharing	`{ type: 'screen' }`	Yes (picker prompt)	Desktop browsers only. Uses `getDisplayMedia`.
HLS	`{ type: 'hls', url: '...' }`	No	Adaptive bitrate via hls.js.
LiveKit	`{ type: 'livekit', url: 'wss://...', token: '...' }`	No	Bring your own LiveKit infrastructure.

Camera

The default source. Uses the browser getUserMedia API to capture from the device camera.

const vision = new RealtimeVision({
  apiKey: 'your-api-key',
  model: 'Qwen/Qwen3.5-9B',
  prompt: 'Read any visible text',
  source: { type: 'camera', cameraFacing: 'user' },
  onResult: (result) => {
    console.log(result.result)
  }
})

Set cameraFacing to 'environment' for the rear camera or 'user' for the front camera. The browser will ask the user for permission the first time.

Video File

Play a video file from an <input type="file"> element. The video loops continuously until you call stop().

const vision = new RealtimeVision({
  apiKey: 'your-api-key',
  model: 'Qwen/Qwen3.5-9B',
  prompt: 'Describe what you see',
  source: { type: 'video', file: videoFile },
  onResult: (result) => {
    console.log(result.result)
  }
})

Screen Sharing

Desktop browsers only. Captures the user's screen for real-time analysis. Useful for computer use agents and accessibility tools.

const vision = new RealtimeVision({
  apiKey: 'your-api-key',
  model: 'Qwen/Qwen3.5-9B',
  prompt: 'Read all visible text on the screen',
  source: { type: 'screen' },
  onResult: (result) => {
    console.log(result.result)
  }
})
 
await vision.start()  // Browser prompts user to select screen/window

Browser support: Requires the getDisplayMedia API -- Chrome 72+, Firefox 66+, Safari 13+, and Edge 79+.

When the user clicks "Stop Sharing" in the browser chrome, the stream automatically stops and triggers the onError callback.

HLS Stream

Stream from any HLS (HTTP Live Streaming) URL. Works with live broadcasts, IPTV feeds, or any HLS-compatible source.

const vision = new RealtimeVision({
  apiKey: 'your-api-key',
  model: 'Qwen/Qwen3.5-9B',
  prompt: 'Describe what you see',
  source: { type: 'hls', url: 'https://example.com/stream.m3u8' },
  onResult: (result) => {
    console.log(result.result)
  }
})

The SDK uses hls.js (opens in a new tab) for adaptive bitrate playback.

LiveKit (User-Managed)

For users who run their own LiveKit infrastructure. You provide your own room credentials, and the SDK connects to your LiveKit server.

const vision = new RealtimeVision({
  apiKey: 'your-api-key',
  model: 'Qwen/Qwen3.5-9B',
  prompt: 'Describe what you see',
  source: {
    type: 'livekit',
    url: 'wss://your-livekit-server.example.com',
    token: 'your-livekit-token'
  },
  onResult: (result) => {
    console.log(result.result)
  }
})

With a LiveKit source, you are responsible for publishing video to the room. The getMediaStream() method returns null for LiveKit sources.

Most users don't need this. The SDK already uses LiveKit natively for all other source types, with automatic reconnection and token refresh. See Transport & Connectivity for details.

Overview Configuration