vLLM Jev
Native vLLM serving for open Jev-style checkpoints on Linux NVIDIA and Apple Silicon: POST /v1/systemone for Choice, Noul, and Score.
Overview
vLLM Jev (github.com/mode-io/vllm-jev, Apache-2.0) is native vLLM serving for compatible Jev-style decision checkpoints. You give it a question and candidate answers. It returns a label and a probability for each candidate. Linux with an NVIDIA GPU uses uv pip install . and vllm-jev serve, which prepares a Hugging Face checkpoint and starts native vLLM at http://127.0.0.1:8795. Apple Silicon (macOS 15 or newer) uses source scripts/install_mac.sh, then the same serve command. The README says Linux is native vLLM (scheduling, batching, compilation, KV cache, metrics). Apple Silicon text models run through MLX, or PyTorch MPS for Laya, and Valen multimodal decisions run through MLX. The plugin targets vLLM 0.29.0 and Python 3.12+. Serving does not train. The supported table lists ZefanCai/Open-Jev-2B (text, Linux and Mac), ZefanCai/Open-Jev-9B (text, Linux), convaiinnovations/laya, laya-multilingual, and laya-typed-decisions (text, Linux and Mac), IamBusy/OpenJev-0.6B (text, Linux and Mac), lostargon/Tiny-Jev (text, Linux and Mac), Valen-Team/Valen-Preview-0923 (text, images, and video, Linux and Mac), and yah01/vjev-vision plus vjev-vision-pilot (text and images, Linux). An experimental table adds Kev, Decider, Mica, This-That, and JevK5, with tested platforms called out per row. README contributing links Issues to github.com/Egbertjing/vllm-jev. Public counts were 198 stars and 7 forks when this listing was drafted.
Problem: Open Jev-style checkpoints already exist, and teams still serve them through each author's own HTTP wrapper, so batching, compilation, and a shared POST /v1/systemone contract get reinvented per model.
Built for: Linux NVIDIA and Apple Silicon operators who want one launcher for Open-Jev, Laya, Tiny-Jev, Valen, and a short list of experimental decision checkpoints, and who will read the README notes when labels move under concurrency.
First indexed on Jev Directory: 2026-10-02
Creator and team
- Name
- mode-io
- Handle
- @mode-io
“Give it a question and candidate answers; it returns a label and a probability for each candidate.”
How Jev is used
- Role in the product flow
- Local System One runtime: one vllm-jev serve process scores Choice, Noul, and Score on a supported open checkpoint
- Primitives
- ChoiceScoreNoul
- State in
- JSON state plus a questions map. Choice sends criteria labels. Noul is yes/no. Score is ordered levels. Valen accepts text, images, and short MP4 video. vjev accepts text and images. Text models take text only.
- Decision out
- answers.<id>.choice and probabilities for the labels you supplied, or the Noul and Score fields documented in the user guide. The server does not generate a free-form answer.
- On Linux, uv pip install . from the repo. On Apple Silicon, source scripts/install_mac.sh once (macOS 15 or newer)
- vllm-jev serve with a supported Hugging Face id, for example ZefanCai/Open-Jev-2B or Valen-Team/Valen-Preview-0923. First run downloads and prepares the checkpoint
- POST http://127.0.0.1:8795/v1/systemone with state and questions. CUDA_VISIBLE_DEVICES, --gpu-memory-utilization, and --port override the defaults
- Read answers.intent.choice and answers.intent.probabilities. Image and short-video requests use a vision model from the supported table
- Treat the experimental Kev, Decider, Mica, This-That, and JevK5 rows as tested adapters with the consistency limits in the README
vLLM Jev is the server, not a new weight family. githubnext-localjev keeps TypeSafe SDK calls on a Mac by asking oMLX DiffusionGemma for chat JSON, and the LocalJev README says those probabilities are not an OpenJev logit read. ollaya-dev-ollaya pulls small ONNX graphs and serves POST /v1/systemone on port 11435 for laya, kev, von, and decider. receptron-laya embeds Convai Laya ONNX inside Node. wfzyx-von is the English encoder checkpoint. liuziyu77-valen is the training and eval repo for Valen-Preview-0923. deepopen-com-deepopen is a pip package with a script-aware router over 3 Laya-based checkpoints. vLLM Jev takes those kinds of Hugging Face ids, verifies a native protocol, and serves Choice, Noul, and Score. The README also shows a live Open-Jev-2B comparison: the author HTTP server and vLLM Jev received the same 40 text-only Choice requests at once, one A800 each. Decision readouts in the About section are scalar candidate branches, marker scores, and Laya's trained decision head. A 2026-09-25 note says the fused Tiny-Jev head raised throughput 12.1% on a 16-option, 16-concurrent workload. Cite github.com/mode-io/vllm-jev README and docs/guide.md. The Choice prompt follows MIT-licensed Open-Jev, noted in THIRD_PARTY_NOTICES.md.
Sourced performance claims
- Valen video live run: 24 structured questions, 8 sampled frames, one A800 per runtime. PyTorch reference 5.31 s. vLLM Jev 1.22 s (4.3x). All 24 selected answers agreed. Model loading is excluded.Source: github.com/mode-io/vllm-jev README Demos, video understanding
- Open-Jev-2B on one A800, short input, concurrency 8: latency 4360.5 ms to 100.5 ms, throughput 1.83 to 78.83 req/s, 43.4x. Concurrency 1 on the same row: 546.6 ms to 73.0 ms, 7.5x. Open-Jev-9B short concurrency 8: 5337.6 ms to 205.2 ms, 26.0x.Source: github.com/mode-io/vllm-jev README Inference performance, NVIDIA GPUs
- Valen-Preview-0923 image row: median latency 231.9 ms to 77.3 ms (3.0x) on 500 image questions. Both paths reached 86.8% target-support accuracy. Choices agreed on 98.8%. Largest per-question probability difference 0.295. A separate 500-request HTTP run at concurrency 8 reached 40.6 req/s (193.4 ms median). HTTP time is excluded from the offline 500-question comparison.Source: github.com/mode-io/vllm-jev README benchmark notes, Valen
- Apple M5, 16 GB unified memory, MLX, sequential after warmup: OpenJev-0.6B text 67.9 ms (P95 120.2, 12.02 req/s) over 96 Choice requests. Valen image 205.9 ms (P95 249.7, 4.64 req/s) over 42 GameQA Maze questions. Valen matched the Linux reference answers on all 42 questions. Both answered 17 correctly. Probability values differ between backends. Mac runs one model request at a time.Source: github.com/mode-io/vllm-jev README Inference performance, Apple Silicon
- Laya English versus published laya-serve 0.3.20, same A800, 500 4-choice MMLU questions: concurrency 16 latency 404.5 ms to 77.5 ms (5.2x). Labels differed on 6 of 500. Largest option-probability difference 0.0222. README says Laya's direct predict_batch path can be faster when many states share one schema, and these HTTP figures do not measure that bulk path. JevK5 supports 2 to 16 options. In concurrent Linux batches, Kev-0.8B can change its selected label. This-That and Decider-2B can change the most likely Score level.Source: github.com/mode-io/vllm-jev README benchmark notes and experimental model limits
- Updates: 2026-09-25 fused Tiny-Jev head, 12.1% higher throughput on the measured 16-option, 16-concurrent workload, plus multimodal text and images. 2026-09-26 added all 3 Laya checkpoints, Tiny-Jev and Open-Jev-2B on Apple Silicon, and OpenJev-0.6B plus Valen on Mac. 2026-09-28 added experimental Valen video, including variable-frame-rate clips. 2026-09-29 added experimental Kev, Decider, Mica, This-That, and JevK5 adapters.Source: github.com/mode-io/vllm-jev README Updates
- Public GitHub repo mode-io/vllm-jev had 198 stars and 7 forks on 2026-10-02. License Apache-2.0. Created 2026-09-24. Last push 2026-09-29. Language Python.Source: GitHub API 2026-10-02
Features and stack
Features
- vllm-jev serve for a supported Hugging Face model id, default port 8795
- Linux native vLLM and an Apple Silicon preview via MLX or PyTorch MPS
- Choice, Noul, and Score over POST /v1/systemone
- Valen text, image, and short video. vjev text and images on Linux
- Open-Jev-2B and 9B, 3 Laya checkpoints, OpenJev-0.6B, Tiny-Jev
- Experimental adapters for Kev, Decider, Mica, This-That, and JevK5, with published consistency limits
Stack
- Python 3.12+
- vLLM 0.29.0
- Apache-2.0
- MLX and PyTorch MPS on Apple Silicon
- Hugging Face decision checkpoints
Pricing: Apache-2.0 code. Inference cost is your GPU or Apple Silicon machine. Upstream checkpoints keep their own licenses. The README says serving does not train.
Links
FAQ
- Is this LocalJev or ollaya?
- githubnext-localjev is a Mac bridge from System One-shaped HTTP to oMLX DiffusionGemma, and its README says the probabilities come from prompted JSON. ollaya-dev-ollaya is a Rust daemon that pulls ONNX graphs and listens on port 11435. vLLM Jev is a vLLM plugin: Linux NVIDIA uses native vLLM, Apple Silicon uses MLX or PyTorch MPS, and the default listen address in the README is http://127.0.0.1:8795.
- Which checkpoints does the README mark as supported?
- The supported table lists Open-Jev-2B, Open-Jev-9B (Linux only), Laya English, Laya multilingual, Laya typed-decisions, OpenJev-0.6B, Tiny-Jev, Valen-Preview-0923, and vjev-vision plus vjev-vision-pilot (Linux only). Kev, Decider, Mica, This-That, and JevK5 sit in a separate experimental table.
- Does the 4.3x Valen video figure include model loading?
- README Demos: model loading is excluded. The same Valen checkpoint answered 24 questions about one video, using 8 sampled frames and one A800 per runtime. PyTorch took 5.31 s. vLLM Jev took 1.22 s. All 24 selected answers agreed.
- Can experimental Kev or Decider labels change under load?
- README: JevK5 currently supports 2 to 16 options per question. In concurrent Linux batches, Kev-0.8B can change its selected label. This-That and Decider-2B can also change the most likely Score level. The README says those consistency limits remain under investigation.
Related learn guides
Original Jev guidance that pairs with this product pattern.
- System One model
The model family behind Jev: parallel typed questions, one forward pass, probabilities you can threshold.
- Where to run Jev
Compare official TypeSafe, Vercel AI Gateway, OpenRouter, Cloudflare Workers AI, and classifier.dev with a fact table.
Related products
Hand-picked neighbors with rich profiles or overlapping tags.