Benchmarks
Evals, open replicas, calibration studies, and comparison harnesses.
Measure twice, route once. Calibration papers, open replicas, and head-to-head rerank battles. Benchmark listings document datasets, baselines, and reproducible harnesses. Treat star counts as hints, not votes. Open replicas help you compare latency and calibration on your hardware before you pick a routing policy.
87 listings in this bucket · Learn Jev
BenchmarksPython1,491
DemoFeatured
Semantic ifs from open models, on a 3090 at home. Independent; not affiliated with Jev or TypeSafe.
by @hhkkmon
benchmarks
BenchmarksPython851
Featured
Train a small model that chooses among a changing list of text options, one probability per option in a single pass. Includes Doom, chess...
by @hhkkmon
benchmarksgame
BenchmarksPython97
An educational Jev-like visual inference experiment on Apple Silicon: shared context, direct candidate scoring, and local visual demos.
benchmarks
BenchmarksPython88
One-pass typed decisions with calibrated probabilities (System One style model), fine-tuned from Qwen3.5-2B.
benchmarkseval
BenchmarksHTML80
Personal-assistant agent built on Vercel's eve with 100 mocked tools, measuring how many steps it takes when Jev picks the tool versus th...
benchmarksagentseval
BenchmarksPython56
A small open decision model: state + typed questions -> calibrated probabilities. A Jev / System One re-creation on Qwen3.5.
benchmarkseval
BenchmarksTypeScript31
Demo
This is a LLM Gateway that mimics typesafe ai structured output. Like an imposter Jev.
by @iamMrDuncan
benchmarkseval
BenchmarksPython27
One-pass option scoring with a local Gemma 3 4B on Apple silicon via MLX, inspired by jevlike, with a Doom demo.
benchmarksgame
BenchmarksPython20
Jev-style parallel constrained decisions for any MLX model on Apple Silicon. Typed, schema-valid JSON in one forward pass.
by @beni_il_
benchmarks
BenchmarksPython19
Mini-Jev: what a Jev-style typed-decision interface looks like on a frozen Qwen3-4B, read the option letter's logits instead of generatin...
benchmarks
benchmarks
BenchmarksPython14
Unofficial study: Jev-style parallel typed decisions on stock 1.5B-8B models on an Apple Silicon laptop. Benchmarks, research notes, and ...
by @hhkkmon
benchmarkseval
BenchmarksPython14
Typed JSON inference with DiffusionGemma, with Every and Jev benchmark results.
benchmarkseval
BenchmarksPython11
A stronger one-pass scorer over a variable list of text options: hashed n-gram encoder, rival-aware attention, gated head, temperature sc...
benchmarkseval
BenchmarksPython8
Type-safe one-decision-per-token decoding engine for autoregressive LLMs, inspired by Jev.
benchmarks
BenchmarksPython7
Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.
benchmarkseval
BenchmarksJavaScript7
Demo
Jev vs Gemini 3.8 Flash: labelling 1,000 app reviews, 4.1× faster and 7× cheaper.
benchmarks
benchmarks
BenchmarksPython5
看看 Jev 能做什么：用中英文讲清热门应用、工作原理和各自优缺点。Explore Jev apps with plain-language examples, explanations, and comparisons.
benchmarks
BenchmarksJavaScript5
Demo
JEV-inspired parallel decisions for CUDA LLMs. One context, many decisions. vLLM API, game-agent examples, and reproducible benchmarks.
benchmarksagentseval
BenchmarksPython5
LegalForecast-MTD benchmark alpha and official evaluation workflows.
benchmarkseval
BenchmarksPython5
Local bilingual probability decisions from context, questions, and candidate answers. Independent research preview inspired by TypeSafe Jev.
benchmarks
BenchmarksPython4
Demo
Reproducible early-access evaluation of Jev on Korean understanding and medical text, with runtime and cost evidence.
benchmarkseval
BenchmarksTypeScript4
A word-level language model whose output layer is Jev: n-gram drafter, Noul chunk verification, bits-per-token eval.
benchmarkseval
BenchmarksPython4
Open replica of TypeSafe's Jev: typed calibrated decisions in one forward pass, on Gemma 4 E2B / Gemma 3 270M (Modal).
benchmarkseval
BenchmarksPython4
Inspired by TypeSafe Ai, Ask a local LLM typed questions, get calibrated probabilities instead of text. Structured output without generat...
benchmarkseval
BenchmarksPython3
Daf-jev: composable Python toolkit for TypeSafe's Jev (System One) decision API, question builders, confidence gates, evaluator, calibrat...
benchmarksmcpagents
BenchmarksPython3
Independent Jev 1.13.0 behavior study: report, controlled prompt experiments, raw results, and offline verification.
benchmarks
BenchmarksPython3
A chatbot from typed Jev decisions: hierarchical speculative decoding over System One probabilities.
benchmarks
BenchmarksPython3
Does a TypeSafe Jev rerank beat embedding search? Graded relevance eval (9,831 pairs, 164 zh/en queries) over the Agent Skills Hub catalo...
benchmarksragagents
BenchmarksPython3
A reproduction of Jev that turns any Qwen model into a fast decision model, serving the same /v1/systemone schema (Choice, Score, Noul) w...
by @yuzxfred
benchmarks
BenchmarksPython3
Batched single-token choice inference for open language models, compatible with TypeSafe.
benchmarks
BenchmarksPython3
Non-autoregressive decision engine on ModernBERT (151M) with calibrated uncertainty (RLCD), TypeSafe AI Jev benchmark audit, and in-brows...
benchmarksbrowser-useeval
BenchmarksPython2
Calibration and confidence-based routing measured on Banking77: 80.2% accuracy at $0.103 per 500 decisions.
benchmarksroutingeval
BenchmarksPython2
An experimental JEV-powered framework for forecasting short-term stock price direction from structured market data.
benchmarks
BenchmarksPython2
Benchmarks and a playground for TypeSafe's Jev (System One) model: chess, and who-is-the-player-talking-to for speech-to-text game NPCs.
benchmarksevalgame
BenchmarksPython2
Eight minimal working examples of TypeSafe's Jev (a System One model) applied to mechanical and electrical engineering: CAD/CAE/CAM routi...
benchmarksrouting
BenchmarksTypeScript2
Not every coding task needs your best model. Experimental Jev-powered model routing for Claude Code, V3 prototype runs today, V4 routes a...
benchmarksroutingagents
BenchmarksTypeScript2
Decision harness for TypeSafe Jev, confidence gates, shadow mode, recipes, and evals. Claude CLI 48.9s → Jev 1.3s on the same row-filter ...
benchmarksagentseval
BenchmarksPython2
Stop guessing confidence thresholds: calibrate, threshold, and drift-check typed decision models (TypeSafe Jev) against an LLM teacher.
benchmarkseval
BenchmarksTypeScript2
A chatbot built on a model that cannot generate text (TypeSafe AI's Jev, driven autoregressively).
benchmarks
BenchmarksPython2
Open alternative to Jev: typed, calibrated decisions from any open-weights LLM in one forward pass (HF + vLLM), with benchmarks.
benchmarkseval
BenchmarksPython2
Backtest Jev (TypeSafe) as a BUY/SELL/HOLD trader on NQ L10 order-book data.
benchmarkstrading
BenchmarksHTML1
A Jev-inspired decision interface for existing LLMs. Explicit choices, scores, calibration, and review thresholds.
benchmarkseval
BenchmarksPython1
Benchmarking Jev (Typesafe.ai) against a strong LLM on the Who&When Pro agent-failure-attribution benchmark (text subset).
benchmarksagentseval
BenchmarksRust1
Demo
High-throughput synthetic & pretraining dataset sifter powered by TypeSafe AI Jev (api.typesafe.ai). Stream, filter, and score Parquet & ...
benchmarks
BenchmarksPython1
Jev (TypeSafe) exploratory thread: claim audit, live demos, and runnable code.
benchmarks
BenchmarksJavaScript1
An observable raw-character chat experiment powered entirely by TypeSafe Jev Choice.
benchmarks
BenchmarksHTML1
A show-and-tell capability study for Jev, TypeSafe's System One decision model.
benchmarks
BenchmarksPython1
Can a decision model beat dedicated rerankers? TypeSafe Jev vs Cohere Rerank 4 vs ZeroEntropy zerank-2 vs a chat-model baseline: 14 datas...
benchmarksrag
BenchmarksHTML1
Reproducible Jev Ultrafast research-browser eval harness + field note (QC’d cases, suite runner, report generator). Not investment advice.
benchmarksbrowser-useeval
BenchmarksGo1
Blind security benchmarks for Jev, TypeSafe's System One model: prompt injection and vulnerable code detection, built on jev-go.
benchmarkseval
BenchmarksPython1
Jev (TypeSafe System One) × ASReview SYNERGY abstract screening demo, Choice/Noul vs gold labels.
benchmarks
BenchmarksPython1
Demo
An agent skill to discover TypeSafe Jev opportunities, design typed questions, and learn from recent community experiments.
benchmarksagents
BenchmarksHTML1
AI benchmark on Japan's 2026 Common Test: Jev vs luna-none vs luna-low (static dashboard).
benchmarkseval
BenchmarksPython1
Discriminative Monte Carlo Tree Search using TypeSafe Jev System One Primitives and Gemini.
benchmarksagents
BenchmarksPython1
Typed-decision benchmark from PadFlow (land development SaaS): schemas, anonymized labeled rows, and a runner for confidence-calibrated m...
benchmarkseval
benchmarkseval
BenchmarksPython
Using Jev to test how well it predicts financial markets(just like most llms as of september 2026, it doesnt do that good).
benchmarks
BenchmarksGo
Does Jev predict stock returns from news? It reads the news well; there is no tradeable alpha. Three arms separate reading from recall.
benchmarkstrading
BenchmarksPython
Demo
Jev (TypeSafe) vs. Gemini 3.8 Flash vs. GPT-5.6 Luna na anotação estruturada de sentenças do TJSP: qualidade, tempo e custo.
benchmarks
BenchmarksTeX
Position paper: the Hidden-Markov and fuzzy primitives missing from TypeSafe AI's Jev and System-One decision models. Two lemmas, one pri...
benchmarks
BenchmarksGo
Can Jev pick the winner of a real headline A/B test? 64.5% across 10,984 Upworthy randomized experiments, 74.7% when the difference was d...
benchmarks
BenchmarksTypeScript
Jev (TypeSafe) 性能評価プロジェクト, 日本郵便 KEN_ALL をマスタに、AI SDK 経由の Jev が住所のあいまい一致にどこまで使えるかを検証.
benchmarkssdk
BenchmarksTypeScript
TypeScript experiments, evaluations, and latency benchmarks for TypeSafe's Jev model.
benchmarkseval
BenchmarksPython
Jev (TypeSafe) vs Claude Haiku 4.5 on 2 000 phishing emails: accuracy, calibration, latency, cost. Reproducible benchmark.
benchmarksmoderationagents
BenchmarksPython
A small reproducible MuJoCo pilot comparing Jev, Claude Haiku, and reactive rules for pick-and-place.
benchmarksagents
BenchmarksTypeScript
Benchmarks Jev against other evaluation models in games with explicit states, legal actions, and measurable outcomes.
benchmarksevalgame
benchmarks
BenchmarksPython
Benchmarking TypeSafe's Jev decision model as a cost-efficient LLM router on RouterArena.
benchmarksroutingeval
BenchmarksPython
Measures how well TypeSafe's RLCD-Jev model spots real secret credentials in file snippets.
by @Teyhouse
benchmarks
BenchmarksJavaScript
A small second eval for shadcn-ui/lint that uses TypeSafe's Jev to judge the linter's own output.
benchmarkseval
BenchmarksJupyter Notebook
Zero-shot spam filtering with TypeSafe Jev Noul questions, compared with TF-IDF baselines.
benchmarksmoderationeval
BenchmarksPython
Application of TypeSafe Jev (noul judgment primitive) on the collusion.wiki corpus: agent vs human page authorship, head-to-head vs local...
benchmarksagents
BenchmarksJavaScript
Demo
Ongoing Japanese research deck on Jev and System One models, maintained as Markdown slides.
benchmarks
BenchmarksPython
Demo
Experiments with openjev, an open Jev-style option-logit runner, on local models.
benchmarks
Benchmarks
Demo
Hugging Face Space exploring open-source parallel constrained decoding as an alternative to Jev.
benchmarks
BenchmarksSwift
On-device iPhone visual decision tool using MLX and Qwen3-VL direct option logits.
benchmarks
BenchmarksPython
Evaluating TypeSafe's Jev as a fast monitor and action gate for agent sabotage in SHADE-Arena, compared with Gemini 2.5 Flash/Pro.
benchmarksagentseval
BenchmarksRust
Rust port of TypeSafe system-one-adapter (LLM-backed system_one evaluations).
benchmarkseval
BenchmarksPython
Open-source Jev-style System One decision model. Gemma 3 270M with a scoring head, fast, calibrated decisions in a single forward pass. N...
benchmarkseval
BenchmarksHTML
Charts: TypeSafe Jev evaluated on Thai standardized exams vs 110 other models.
benchmarkseval
BenchmarksPython
An evaluation of typesafe AI chess. As it turns out, the AI isn't doing really well even though chess is not a particularly open-ended ga...
benchmarksevalgame
BenchmarksJavaScript
Evaluating TypeSafe's System One primitives (Choice/Score/Noul), where a typed oracle beats an LLM call.
benchmarkseval