Live object detection with the YOLO plugin

Detection as a plugin

The YOLO plugin turns any live room into a detection feed. It is a processor plugin with a Python worker: the framework spawns the worker when you enable the plugin, the worker runs YOLO (ultralytics) over the stream, and detections are drawn as bounding boxes on the player overlay — including on the public /play and /embed pages.

You write no glue code. The plugin declares needsWorker: true and a pure spawn(ctx), and the framework owns the process lifecycle: enabling starts it, disabling stops it, and start/stop/status/logs are exposed over the API.

Install and enable

curl -X POST $BASE/apps/live/plugins/yolo/install \
  -H "Authorization: Bearer $TOKEN"

curl -X PATCH $BASE/apps/live/plugins/yolo \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"enabled": true, "config": {"room": "studio"}}'

Every config field has a default, so a fresh install is valid the moment you point it at a room.

How it flows

1. The worker reads the app's live stream.
2. It runs the YOLO model per sampled frame and finds objects.
3. On a detection it POSTs the boxes to the plugin's live-data channel (authenticated with a per-start ingest token).
4. The player overlay polls that channel and draws the boxes over the video, client-side.

Unlike the face-blur plugin, YOLO only posts on a hit — no detection means no update, which keeps the channel quiet on empty scenes.

Check the worker

curl $BASE/apps/live/plugins/yolo/worker/status \
  -H "Authorization: Bearer $TOKEN"

curl "$BASE/apps/live/plugins/yolo/logs?limit=200" \
  -H "Authorization: Bearer $TOKEN"

The status endpoint tells you whether the worker is running; the logs endpoint tails its output for debugging a model or performance issue.

Performance notes

Detection is CPU-heavy. Keep the sampled frame rate modest and downscale large frames for detection — you rarely need full resolution to find a person or a car. The model file downloads on first run and is shared across every app on the node. For heavy workloads a GPU pays off, but the worker degrades gracefully on CPU-only hosts.

What to build with it

People-counting for a venue, vehicle detection on a parking camera, safety-zone alerts on a factory floor — anywhere you want the stream itself to become a signal. Pair the detections with a webhook consumer and YOLO becomes the eyes of an automation, not just an overlay.