Model ecosystem

The AI registry.

Every model in OpenNVR is an adapter: its own container, behind one open contract, advertising the tasks it serves. An app asks for a task, never for an adapter by name — so a better plate reader is a drop-in, and a model you publish next year works with an app written today.

Everything below runs on your hardware by default. Under the stock local_only policy, an adapter that declares network egress is refused registration.

Describe what to watch for

A fixed detector knows its list. These four take a sentence. Each is an adapter below plus the app that calls it, on your hardware.

Search by meaning

Type "white van by the gate" and Footage Search ranks visits by what they look like, not only by the words a captioner used. CLIP puts frames and your sentence in the same space.

CLIP Embeddings · Footage Searchtask embed

Ask a frame a question

"Is the shutter down?" "How many bikes are outside?" The Camera Agent puts the question to a small vision-language model on the frame and answers in words.

Moondream, or any model behind Ollama · Camera Agenttask visual_qa

Detect a phrase

"Forklift." "Person wearing a hard hat." "Red truck." A phrase in, boxes and scores out, for anything outside the detector's eighty classes, with no model to train. Seconds a frame on CPU, so an app, a workflow step or a search asks it about one frame.

Open-Vocabulary Detection (OWL-ViT v2)task open_vocab_detection

Tell the agent what to watch

"Alarm if a car enters between 10 pm and 6 am." "Notify me when a person is on the yard camera after hours." The Camera Agent arms a monitor or an alarm on what the always-on detector can see, and says so when it cannot.

Camera Agent · monitors and alarmstask object_detection

A standing watch on a phrase, "alert me when a forklift is in the pedestrian aisle" running on every frame, is the Describe it step of Agentic Workflows, in design. Today the phrase detector is asked, not left running.

Ships with OpenNVR

15 adapters · GHCR images
vision

YOLOv8

Low-latency person and object detection on ONNX runtime — CPU and GPU. The default Tier 0 adapter.

task object_detection
vision

ByteTrack

Persistent multi-object track IDs across frames. Runs as a post-processor over an upstream detector.

task multi_object_tracking
vision

InsightFace

Face detection and recognition with 5-point landmarks and a REST-managed face database.

task face_recognition
vision

YOLO Pose

Body keypoints — shoulders, elbows, wrists, hips — so a rule can reason about what a person is doing, not just where they are. CPU-first, ~12 fps on 8 cores at 448px. What Guard Scan Compliance rides.

task pose_estimation
vision

fast-plate-ocr

License-plate text recognition on a cropped plate. Designed to chain downstream of a detector.

task license_plate_recognition
vision

BLIP

Vision-language scene captioning that runs locally — natural-language descriptions of a camera scene.

task scene_caption
vision

Moondream VLM

Ask a frame a question in plain language — "is the gate open?", "how many bikes?" — and caption scenes. The small vision-language model behind the agent's "what do you see?".

task visual_qa
vision

CLIP Embeddings

Semantic search. Frames and typed queries land in the same vector space, so a search for "truck" finds the visit a captioner called "a lorry". SigLIP 2 B/16 from adapter 0.1.8 (coco128 retrieval mAP 0.672 against CLIP's 0.571). Weights baked in, no network egress.

task embed
vision

Package Detection

Parcels on a doorstep — the one class COCO does not have. YOLOv8n fine-tuned on openly licensed doorstep sets, ~30 ms per frame on CPU. What the Package Delivery app calls on demand.

task package_detection
vision

Open-Vocabulary Detection

Detect whatever you can describe — "red truck", "forklift", "person wearing a hard hat". A phrase in, a box and a score out, for anything outside the detector's eighty classes. OWL-ViT v2; seconds a frame on CPU, so it is asked on demand, not run on every frame.

task open_vocab_detection
language

Gemma on the Qualcomm NPU (LiteRT-LM)

Captions and visual Q&A from Gemma 4 E2B on the Hexagon NPU of Qualcomm Dragonwing boards (Arduino Ventuno Q / IQ-8275) through LiteRT-LM. Measured: 1.6 to 1.8 s a photo where a CPU vision model took 16 s. The qualcomm flavour picks it; the model and runtime download once at first start.

task scene_caption
language

Text Embeddings

The caption arm of footage search: a sentence encoder (bge-small, 384-d) that turns a visit's caption and a search in words into vectors made to be compared with each other. 37.6 ms a caption on two ARM cores.

task text_embed
audio

Parakeet ASR

Speech to text with NVIDIA Parakeet TDT 0.6B v3 on ONNX Runtime, CPU first — the agent's default. On its own phrasings: WER 1.4% at 0.31 s an utterance on four cores, where whisper base.en scores 3.3% at 1.42 s. English and 25 European languages; weights baked in, no network egress.

task speech_to_text
audio

Whisper

Speech-to-text transcription via faster-whisper — CPU and GPU. The pick for translation and for Hindi.

task speech_to_text
audio

Piper

Fast, local neural text-to-speech for spoken alerts and the camera-agent voice loop.

task text_to_speech

In the repo, not yet in the index

1 experimental

Real adapters with tests and a Dockerfile, without a registry entry — so a stock catalog does not offer them yet. Build from source to try one.

Open for contribution

2 wanted
vision

PPE / Hard-Hat Detection

Construction-site safety compliance — a popular community target. Open for contribution.

task ppe_compliance
vision

Pose / Fall Detection

Pose-based fall detection for elder care. On the roadmap and open for contribution.

task fall_detection

Publish your model

Wrap it with the Apache-2.0 SDK — four names is the whole API for a model that takes an image — and it ships under any licence you choose, proprietary or classified included. A conformance kit checks the contract before anyone else does.

Getting listed is a pull request adding one entry to server/config/adapters_index.yml in the open-nvr repository — the file every deployment's catalog reads, and the one this page is checked against.