BenchmarksEdge AICamera AgentOpen Source

We benchmarked every small model a CPU camera agent needs. The bigger model lost.

By Varun Pratap Singh•
Bar chart of multi-step tasks passed out of 60 on a 4-core CPU: granite4:micro 44, granite4:tiny-h 35, granite4:micro-h 34, qwen3:1.7b 21, LFM2-1.2B-Tool 19, qwen3:4b 9

OpenNVR's camera agent answers questions about your cameras, arms alarms, searches footage and, soon, investigates events on its own. It has to do all of that on the box most of our users actually have: a mini-PC with four CPU cores and no GPU. For a year we picked its models the way most projects do, by reputation. This month we stopped, wrote a small harness for every role a model plays in the agent, and ran the candidates on one 4-core machine. Four defaults changed. A few results went against what we expected, and they are worth sharing whatever you are building.

The method, briefly

Each role got its own harness, and each harness is in the repository so you can rerun it on your hardware: tool calling and a multi-step "investigate" task for the language model, questions with known answers on labelled images for the vision model, word error rate on the agent's own phrasings for speech to text, real-time factor and intelligibility for text to speech, and text-to-image retrieval for search. Accuracy was measured once and checked against a second machine; every latency figure below comes from four CPU threads.

1. The language model: the bigger one lost

The agent's hardest job is an investigation: given an event ("person at the loading bay at 01:14"), use a handful of tools to look at the camera, the history and the site's rules, then decide. Twenty cases, three repeats each.

ModelPassedLooked before deciding
granite4:micro44 / 6056 / 60
granite4:tiny-h (7B MoE, 1B active)35 / 6049 / 60
qwen3:1.7b (our previous default)21 / 6036 / 60
LFM2-1.2B-Tool19 / 6037 / 60
qwen3:4b9 / 609 / 60

The 4-billion-parameter Qwen scored less than half of its 1.7-billion sibling. Reading the traces explains it: it reasoned at length about the event and almost never called a tool, so it decided on evidence it never gathered. IBM's Granite 4 micro does the opposite, about two tool calls per step before it answers, and that single habit more than doubled the pass rate. It costs roughly twice the seconds per turn on CPU. For an investigation that runs in the background, we took the trade.

One fix fell out of this. Told to "always look before describing", Granite looked even when the user said "thanks, that's all". Small talk now reaches the model with no tools at all, which is a change to the agent, not the model, and it helped every model we tested.

2. The vision model: a better answer, too late

For "is there a van at the gate?" and "how many people are in the yard?" we asked each model three questions on each of 128 labelled images: one about an object that is there, one about an object that is not (a "yes" is a hallucination, the failure that matters most on a camera), and a count.

ModelFalse yes, of 126People counted within oneCPU, per new frame
qwen3-vl:2b-instruct050 / 6118 s (follow-ups free)
gemma3:4b848 / 6117 s
moondream 0.5b2242 / 617.6 s

Qwen3-VL is the most accurate model in the set and never claimed to see something that was not there. On a GPU it answers in three seconds and is now our default there. On four CPU cores, encoding the frame alone takes eighteen seconds, and a better answer that arrives after the moment is a worse answer, so Moondream keeps CPU. One trap if you use Ollama: the plain qwen3-vl:2b tag is the thinking build and ignores "think: false"; the -instruct tag is the same model without it.

3. Speech to text: the clearest win

ModelWord error ratePer utterance
NVIDIA Parakeet TDT 0.6B v3 (onnx-asr, int8)1.4%0.31 s
faster-whisper base.en (int8)3.3%1.42 s
faster-whisper small.en3.7%2.67 s

Half the errors at a fifth of the time, on 29 of the agent's own phrasings. A second of speech recognition per turn had been the largest single cost in the voice loop; it is now a third of a second. Parakeet does not translate and has no Hindi, so Whisper stays for those sites.

4. Text to speech: the nicer voice couldn't keep up

Kokoro-82M sounds better than Piper, and transcribes back about as well. On four cores it also runs at 1.9 times real time, which means it cannot stream a reply ahead of playback. Piper runs at 0.23. Piper stays.

5. Search and detection

Searching footage by description ("red truck at the dock") depends on an embedding model. On text-to-image retrieval, SigLIP 2 B/16 reached a mean average precision of 0.672 against CLIP ViT-B/32's 0.571, at 57 ms per image instead of 26. It is now the default, and the platform re-embeds its own history after the upgrade. The always-on detector moved from YOLOv8n to YOLO26n: 62 ms a frame instead of 75, more accurate, the same output format.

What we took from it

  • Size is not the variable that matters for agents. Whether the model calls its tools before deciding mattered more than parameter count.
  • Latency is part of accuracy. On a camera, the best answer at eighteen seconds lost to a good one at eight.
  • Measure on the hardware your users have. Most of these rankings flip on a GPU, and our users mostly don't have one.

Every harness and every raw number is in the repository: the results in MODELS_AND_LATENCY.md and the tools beside it. If you rerun any of them on different hardware, we would genuinely like the numbers; a pull request with your row is welcome. And if you know a vision model under two billion parameters that encodes a frame in under five seconds on CPU without losing the no-false-positives property, that is the gap we could not close.