
I Tested Jev vs 12 Local Decision Models. Here's What I’d Use...
The AI Automators3 October 2026Watch on YouTube
Part of series
Ep. 4 · Jev AI + Claude: Live Demos
Praktische demonstraties van Jev AI gecombineerd met Claude voor realtime visuele en zakelijke automatisering.
View the seriesWhat you'll learn
- Thirteen decision models, including Jev and twelve local open models, were tested on over 7,000 decisions each.
- Jev performs best overall, but the gap with the local models is smaller than expected.
- The question type makes a difference, and a tiny model topped one of the tests.
- Local models are faster on short inputs, but that speed advantage changes once the input gets longer.
- Some models could not accept the questions at all, which affects which model you should choose.
Frequently asked questions
Which models were compared in the Jev arena?
How many decisions were tested per model?
Is Jev always the best choice?
When are local decision models faster than Jev?
Topics
Read next
OpenAI's Jev-kloon moet zwermende agents inperken
OpenAI introduceert een "Decisions API", een kloon van Jev, die volgens het bedrijf het belang van snelle en goedkope intelligentie bevestigt bij het beheersen van samenwerkende agents.
Nieuw AI-model van ChatGPT-bedenker Jev enthousiasmeert ontwikkelaars
Jev, een nieuw soort AI-model van een van de bedenkers van ChatGPT, biedt ontwikkelaars volgens de berichtgeving een goedkopere en snellere route naar software-intelligentie.
Nieuwe benchmark Taste-Bench meet slechte strategische keuzes van AI-agents
Uit de benchmark Taste-Bench blijkt dat zelfs het beste model, GPT-5.6 Sol, bij net geen 60 procent van eenvoudige ja/nee-vragen de juiste langetermijnkeuze maakt.
Waarom off-the-shelf AI vaak beter is dan custom modellen
Artikel onderzoekt waarom organisaties beter kiezen voor standaard AI-oplossingen dan vroegtijdig overstappen op gepersonaliseerde, zelfgebouwde modellen.
Promotievideo over automatisering met Jev AI en Claude
Een video belooft snellere en goedkopere workflows door Jev AI via OpenRouter met Claude te combineren.
Zelfcorrigerende AI-agents als bouwstijl in de Nederlandse startup-scene
Meerdere Nederlandse startups bouwen AI-systemen die zichzelf bijsturen op basis van feedback en meetdata, van dynamische prijsmodellen tot vroege detectie van industriële storingen. Een vergelijking laat zien hoe reinforcement en self-correction zich vertalen naar concrete producten.
Description from the channel
👉 Access our AI Builders course & join our community: https://www.theaiautomators.com/?utm_source=youtube&utm_medium=video&utm_campaign=tutorial&utm_content=jev-arena 🧪 Jev Arena: the app, the results from this video and the code to run the tests on your own hardware https://github.com/theaiautomators/jev-arena 🔗 Jev / TypeSafe TypeSafe: https://typesafe.ai/ Jev model docs: https://docs.typesafe.ai/models 🔗 Models in the arena Winnow 12B: https://huggingface.co/EldanRing/Winnow-12B Decider 4B: https://huggingface.co/Mapika/decider-4b Laya: https://huggingface.co/convaiinnovations/laya SemIf 4B (OpenJev): https://github.com/TheoLeeCJ/SemIf-OpenJev Nimble 9B: https://huggingface.co/bespokelabs/Bespoke-Nimble-9B CLM 8B: https://huggingface.co/Contrastive-LM/CLM-v0.1-8B Plumb 4B: https://huggingface.co/crh225/plumb-4b Qwen 3.5 4B: https://huggingface.co/Qwen/Qwen3.5-4B ModernBERT NLI: https://huggingface.co/MoritzLaurer/ModernBERT-large-zeroshot-v2.0 🔗 Benchmarks mentioned JevBench: https://github.com/fstandhartinger/jevbench DecisionBench: https://huggingface.co/spaces/Hanno-Labs/decision-bench-leaderboard Jev is the decision model everyone is talking about, but a wave of small open models like Winnow, Decider and Laya now promise the same kind of AI decisions on your own hardware, with inference kept inside your own infrastructure. So how close are they really, and which ones should you actually use? To find out, I built a Jev arena and ran 13 decision models through over 7,000 decisions each on my own workstation. I measured how often they got the decision right, how quickly they responded, whether the question type made a difference and what it takes to run them locally. Jev came out on top overall, but the gap was smaller than I expected. A tiny model topped one of the tests, some models couldn't accept the questions at all, and the speed advantage of running locally changed completely once the input got longer. If you're choosing between hosted Jev and a self-hosted decision model, this should tell you which ones are worth investigating and what to check before you commit. #AI #Jev #TypeSafe #DecisionModels #LocalAI #SelfHostedAI #OpenSourceAI #Winnow #Laya #Benchmarks #AIAutomation #AIAgents #AIBuilders