REST API reference

Two API surfaces: the plugin controller (proxied behind Jellyfin auth at /Upscaler/*) and the AI service (direct, :5000).

Auth

• Plugin controller: Jellyfin's standard X-Emby-Token access-token header. Elevated endpoints additionally require admin role (RequiresElevation).
• AI service: shared-secret X-Api-Token header when API_TOKEN env var is set on the container. Plain HTTP with no token if unset (homelab-friendly default).

🔐 Securing the AI service (API token)

The AI service on :5000 does the GPU work and has no login of its own. If it's reachable beyond a fully trusted LAN, protect it with the X-Api-Token header. There are three distinct pieces - keep them apart and the rest is easy:

A request is accepted when its X-Api-Token matches either the env API_TOKEN or any active (non-expired) managed token - both compared in constant time (hmac.compare_digest).

Turn auth on - 3 steps

1. Generate a strong secret locally: openssl rand -base64 32 (≈256 bit). Don't reuse it anywhere else.
2. Set it as API_TOKEN on the container (compose / run below) and restart the container.
3. Paste the identical value into Plugin → Settings → AI Service API Token and save. ⚠️ If the two differ, every call returns 401/403 - the #1 setup mistake. Then hit Test Connection: green means the tokens match.

Generating named tokens in Jellyfin (API Tokens tab)

Once auth is on, the plugin's API Tokens tab (Jellyfin admin only) mints extra tokens without ever touching the env var:

1. Type a name (e.g. Living-room TV), pick an expiry (30 / 90 / 365 days, or never), click Create token.
2. The value is shown once - copy it immediately. Only a SHA-256 hash is stored; it can never be displayed again.
3. Revoke any token individually; the others and the env bootstrap stay valid. Expired tokens stop working automatically (no sweep needed).

docker-compose

services:
  ai-upscaler:
    image: kuscheltier/jellyfin-ai-upscaler:docker7   # or :docker7-intel / :docker7-amd / :docker7-cpu (CPU)
    environment:
      - API_TOKEN=change-me-to-a-long-random-string        # or: disable
    ports:
      - "5000:5000"
    volumes:
      - upscaler-models:/app/models
      - upscaler-config:/app/config        # persists managed API tokens across recreates
    restart: unless-stopped

docker run

docker run -d --name jellyfin-ai-upscaler \
  -e API_TOKEN=change-me-to-a-long-random-string \
  -p 5000:5000 \
  kuscheltier/jellyfin-ai-upscaler:docker7

Confirm it's active

Call GET /status on the service - "api_token_configured": true and "auth_enabled": true mean the token is enforced. The Operator Console at :5000 shows the same on its API tab. Then click Test Connection in the plugin: a green result means the plugin's token matches the container's.

Plugin controller endpoints (/Upscaler/*)

AI service endpoints (:5000)

System

Hardware

Models

Inference

Object masking

Since v1.8.3.24 this runs during playback. The player already captured frames for server-side upscaling; with masking enabled it sends them to Upscaler/detect-mask instead and draws the covered frame back onto the overlay. It replaces upscaling on that stream — two full inference passes per frame do not keep up with playback. Enable it on the Settings tab, or in the player menu under Auto → Cover objects.

Covers things you do not want on screen — the original request was to stop a dog reacting to dogs and cats on TV. Detection runs here rather than in ffmpeg, because jellyfin-ffmpeg is built without any DNN backend, so dnn_detect cannot run there at all; and accepting arbitrary -vf would expose ffmpeg's file-reading filters (movie=, subtitles=) to every authenticated user.

No detection model ships with the service. Every catalog entry carries a verified sha256 pin, and inventing one for an unhashed model would break the guarantee the importer exists to provide. Import your own — for example tiny-yolov3-11.onnx from the ONNX model zoo — through the normal upload path, which pins and verifies it.

Both detector families work and the service picks between them by reading the loaded model's own inputs and outputs, not by guessing: single-head exports (YOLOv5/v7/v8/v9, one tensor of raw anchors) and the NMS-head ONNX YOLOv3 exports (two inputs, three outputs, boxes already in source coordinates). A model it does not recognise is rejected at load time, because a misread tensor does not raise — it paints boxes over the wrong part of the picture.

TOKEN=your-api-token
BASE=http://localhost:5000

# 1. load a detector you imported earlier
curl -X POST "$BASE/models/load-detector" -H "X-Api-Token: $TOKEN" \
     -F model_name=tiny-yolov3 -F input_size=416

# 2. cover every animal in a frame with a solid box
curl -X POST "$BASE/detect-mask?classes=animals&mode=box&pad=12" \
     -H "X-Api-Token: $TOKEN" --data-binary @frame.jpg -o masked.jpg

# blur instead of a filled box, and only dogs
curl -X POST "$BASE/detect-mask?classes=dog&mode=blur" \
     -H "X-Api-Token: $TOKEN" --data-binary @frame.jpg -o masked.jpg

The response carries X-Detections with the number of regions covered. classes takes COCO names or the animals group; pad grows each box, which matters because a detector's box hugs the animal and the ears sticking out set a dog off just as well.

Example: load a model + upscale a frame

TOKEN=your-api-token
BASE=http://localhost:5000

# 1. Load model
curl -X POST -H "X-Api-Token: $TOKEN" $BASE/models/realesrgan-x4plus/load

# 2. Upscale a PNG (base64 in, base64 out)
IMG=$(base64 -w 0 input.png)
curl -X POST -H "X-Api-Token: $TOKEN" -H "Content-Type: application/json" \
  -d "{\"image\":\"$IMG\",\"model\":\"realesrgan-x4plus\",\"scale\":4}" \
  $BASE/upscale-frame | jq -r .image | base64 -d > output.png

Rate limiting & concurrency

The service has a single GPU serialiser - concurrent /upscale-frame requests queue behind one active inference. The MaxConcurrentStreams plugin config gates how many requests the plugin will have in-flight at once; tune this based on VRAM headroom.

ID validation regex

Model IDs and filter preset IDs are validated against ^[a-zA-Z0-9_-]+(?:\.[a-zA-Z0-9_-]+)*$ - dot-separated segments are allowed (for rife-v4.9 etc) but traversal patterns (.., leading/trailing dots) are blocked.