Moderation / Live streams
Moderation

Live streams

Live stream moderation with the Detectivision API: moderate a camera or a stream while it runs, paid for the frames checked.

On this page

A live session checks frames continuously for as long as it runs. There are two kinds:

  • Stream link (source: "url"): you give the address of a stream (RTSP, RTMP, HLS, HTTP or SRT) and our servers watch it. You read the results by polling.
  • Camera (source: "camera"): you send frames yourself, for example from a browser camera or a device, and each frame's result comes back at once.

Sessions are priced per minute and you pay for the frames checked: a minute of credits is held ahead, and what the frames do not use goes back when the session ends. Flagged frames are kept as moments, with their picture, in your moderation history.

The session lifecycle#

  1. Start: POST /api/v1/live/sessions. A minute of credits is held.
  2. Run: poll GET /api/v1/live/sessions/{id} (stream link) or send frames to POST /api/v1/live/sessions/{id}/frames (camera). Each time the held minute runs out, the next one is held.
  3. Stop: DELETE /api/v1/live/sessions/{id}, or the session stops by itself (see below). The session keeps what its frames cost; the rest goes back to your balance.

Start a session#

curl https://api.detectivision.ai/api/v1/live/sessions \
  -H "Api-Key: $DETECTIVISION_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"source": "url", "url": "rtsp://cams.example.com/live", "fps": 1, "mode": "capture"}'
import os, requests

API = "https://api.detectivision.ai/api/v1/live/sessions"
H = {"Api-Key": os.environ["DETECTIVISION_API_KEY"]}

session = requests.post(API, headers=H, json={
    "source": "url",
    "url": "rtsp://cams.example.com/live",
    "fps": 1,
    "mode": "capture",
}).json()
print(session["id"], session["costPerMinute"])
const API = "https://api.detectivision.ai/api/v1/live/sessions";
const headers = { "Api-Key": process.env.DETECTIVISION_API_KEY, "Content-Type": "application/json" };

const session = await (await fetch(API, {
  method: "POST",
  headers,
  body: JSON.stringify({ source: "url", url: "rtsp://cams.example.com/live", fps: 1, mode: "capture" }),
})).json();
console.log(session.id, session.costPerMinute);
Field Values Default Meaning
source url, camera required Who supplies the frames.
url up to 2,000 characters required for url rtsp://, rtsps://, rtmp://, rtmps://, http(s):// (including HLS .m3u8) or srt://, on a public address.
fps above 0, at most 30 1 Frames checked per second. At high rates the models set the pace: with every flag on, about 25 frames a second are checked; choosing fewer flags (flags) goes faster.
mode detect, capture, censor capture detect: flags only. capture: also boxes. censor: boxes, and every moment also gets a censored copy (see Moments).
censorStyle blur, pixelate, box blur censor only: how the moments' censored copies hide what was found.
frameSide 240 to 1920 1280 Stream links: the long side, in pixels, of the pictures kept for moments and samples (640 SD, 1280 HD, 1920 Full HD). For HLS, the rendition closest to this size is watched.
momentEvery 1 to 600 3 At most one flagged frame is kept as a moment every this many seconds.
flags flag keys from GET /api/v1/flags every flag The flags to check on every frame of the session. The models that feed none of them are not run, so frames are checked faster. See Choosing flags.
tracking true, false false Follow each finding from frame to frame, so a blur never blinks off. Once an object has been seen in two frames in a row, a frame where a model misses it still carries it (tracked: true, with its track number) for two frames, and at least a second. Free. See Tracking.
metadata any JSON object, at most 16 KB none Kept with the session and on every moment it keeps; it comes back in answers and webhooks. See Metadata and actions.

The answer (201) is the session:

{
  "id": 31,
  "source": "url",
  "url": "rtsp://cams.example.com/live",
  "fps": 1,
  "mode": "capture",
  "frameSide": 1280,
  "momentEvery": 3,
  "status": "running",
  "reason": null,
  "framesChecked": 0,
  "framesFlagged": 0,
  "moments": 0,
  "creditsCharged": 2,
  "creditsUsed": 0,
  "costPerMinute": 2,
  "startedAt": 1790505000,
  "endedAt": null,
  "jobId": 2210
}

Poll the session about once a second, passing the last seq you have seen as after. Each call returns the session and the samples checked since then.

import time

after = 0
while True:
    data = requests.get(f"{API}/{session['id']}", headers=H, params={"after": after}).json()
    for entry in data["entries"]:
        after = max(after, entry["seq"])
        if entry["flagged"]:
            print(f"{entry['t']:.1f}s", [f["flag"] for f in entry["flags"]])
    if data["session"]["status"] != "running":
        print("ended:", data["session"]["reason"])
        break
    time.sleep(1)

An entry looks like this:

{
  "seq": 13,
  "t": 12.01,
  "at": 1790505013.4,
  "frame": true,
  "flagged": true,
  "flags": [{ "flag": "smoking", "confidence": 0.84, "detections": [{ "label": "smoking", "confidence": 0.84, "bbox": [410, 220, 520, 330] }] }],
  "unchecked": [],
  "moment": 2231
}
  • t is seconds since the stream started, at the Unix time.
  • frame: true means the sample's picture can be downloaded from GET /api/v1/live/sessions/{id}/frames/{seq} (JPEG). Only the last 90 or so samples keep their picture.
  • moment is set when the sample was kept as a moment.
  • bbox is in pixels of the stream (session.width × session.height while it runs).

You do not have to poll for the session to be billed and recorded: our servers keep watching the stream until you stop it. Polling is how you see results as they happen.

Camera: send frames#

For a camera session, send each frame as the raw body of POST /api/v1/live/sessions/{id}/frames, with its content type (image/jpeg, image/png or image/webp), at most 2 MB. The answer is that frame's result:

curl https://api.detectivision.ai/api/v1/live/sessions/32/frames \
  -H "Api-Key: $DETECTIVISION_API_KEY" \
  -H "Content-Type: image/jpeg" \
  --data-binary @frame.jpg
const res = await fetch(`${API}/${session.id}/frames`, {
  method: "POST",
  headers: { "Api-Key": process.env.DETECTIVISION_API_KEY, "Content-Type": "image/jpeg" },
  body: jpegBytes,
});
const frame = await res.json(); // { seq, t, flagged, flags, checked, unchecked, moment?, creditsUsed }
{
  "seq": 8,
  "t": 7.013,
  "flagged": true,
  "flags": [{ "flag": "smoking", "confidence": 0.84, "detections": [{ "label": "smoking", "confidence": 0.84, "bbox": [410, 220, 520, 330] }] }],
  "checked": ["nudity", "explicit_nudity", "sexual_objects", "smoking"],
  "unchecked": [],
  "creditsUsed": 1
}
  • Send at the session's fps. Frames that arrive faster than twice that rate are refused with 429.
  • Boxes are in pixels of the frame you sent. Scale frames down before sending (1280 px on the long side is plenty) to keep uploads fast.
  • creditsUsed is what the session has cost so far, this frame included.
  • A camera's next minute is held when a frame comes after the held one, so time without frames is not paid for. If no frame arrives for two minutes, the session stops.

Stop a session#

curl -X DELETE https://api.detectivision.ai/api/v1/live/sessions/31 \
  -H "Api-Key: $DETECTIVISION_API_KEY"

The answer is the session with status: "stopped" and reason: "Stopped by the owner.". It keeps what its frames cost (creditsCharged equals creditsUsed from now on); the rest of the minute it held goes back to your balance.

When a session stops by itself#

status reason When
stopped Out of live stream credits. The next minute could not be held.
stopped No frames for two minutes. A camera session received no frames for two minutes.
ended The stream ended. The stream ended.
ended The session reached the 4-hour limit; start a new one to continue. A stream link ran for 4 hours.
failed The stream could not be read. The stream could not be opened or read.
failed The session stopped on our side; start a new one. We could not keep watching the stream.

A stopped or finished session's reason is always Stopped by the owner. or one of the texts in this table.

Pricing#

A minute at the session's rate costs ceil(fps) × price of one check (costPerMinute): 1 credit per check in detect mode, 2 in capture and censor mode. So at fps: 1 a minute costs 1 (detect) or 2 (capture, censor); at fps: 30, 30 or 60.

You pay for the frames checked: each costs costPerMinute / (60 × fps), and a session's total is rounded up to whole credits. At fps: 30 in capture mode, 900 frames (about 30 seconds) cost 30 credits.

  • A minute is held when the session starts, and the next one each time the held minute runs out (a camera: when a frame comes after it). If a minute cannot be held, the session stops.
  • While it runs, creditsCharged is what has been taken from your balance (the minute held ahead included) and creditsUsed what the frames have cost.
  • When it ends, the session keeps creditsUsed and the rest goes back to your balance.

Live sessions use your live stream credits. See Credits and billing.

Moments#

When a frame raises a flag, it can be kept as a moment: an entry in your moderation history (dashboard) with the frame's picture and its flags, at most one every momentEvery seconds. Moments are free. The moment field on a frame or entry is its id. With tracking, a frame that only tracking covers is flagged but never becomes a moment, so a moment's picture always shows what a model saw.

In censor mode each moment also gets a censored copy: the frame with what was found blurred, pixelated or boxed, as censorStyle says. It is made just after the frame's answer, and you download it (JPEG) from GET /api/v1/moderation/{moment}/censored. Frames and entries still carry their boxes, so your own player can hide the same regions live.

Errors#

HTTP error message Cause
400 invalid_request Invalid live session request (with details) A field is missing or out of range.
400 invalid_request Give the stream URL. source: "url" without url.
400 invalid_input The stream URL was refused: use rtsp, rtmp, http(s) or srt with a public address. Unsupported scheme, or a private address.
400 invalid_input The frame is empty. Empty frame body.
400 invalid_flags Live streams do not check insult. Flags they check: … A flag live streams do not check.
400 invalid_metadata why it was refused metadata is not a JSON object of at most 16 KB.
402 insufficient_quota A live session costs 2 credits a minute; not enough live stream credits. Not enough credits for the first minute.
402 insufficient_quota Out of live stream credits. The session stopped for lack of credits (on a frame).
404 not_found No such live session. Unknown id, or another account's session.
404 not_found No such frame. / That frame is no longer kept. The sample's picture is not available.
409 conflict This session watches a stream URL; it takes no frames. Frames sent to a stream-link session.
409 conflict This session has stopped. Frames sent to a session that is not running.
413 payload_too_large A frame may be at most 2 MB. Frame too large.
415 unsupported_media_type Send frames as JPEG, PNG or WebP. Wrong content type.
429 too_many_requests Frames are coming faster than the session's rate. More than twice fps.
503 temporarily_unavailable Live stream moderation is temporarily unavailable. Try again in a minute.
503 temporarily_unavailable All live stream slots are taken; try again in a minute. Capacity is full right now.
503 not_available_yet why it cannot be served Live stream moderation cannot be served right now. Do not retry; Service status shows each type's state.

See also Errors for authentication errors.