VideoJev
MCG-NJU System One video model: temporal grounding and multiple-choice QA by direct candidate scoring, no text generation. HF VideoJev-4B and 35B-A3B.
Overview
VideoJev (github.com/MCG-NJU/VideoJev, Apache 2.0, arXiv:2610.18615) from Yuhan Zhu, Xiangyu Zeng, and colleagues at MCG, Nanjing University, is a System One family for video. It adapts Qwen3.5-4B and Qwen3.6-35B-A3B multimodal backbones by replacing answer decoding with a shared head that scores temporal interval candidates and multiple-choice options against the same video evidence, with no text generation on the task outputs. Hugging Face ships BF16 VideoJev-4B and VideoJev-35B-A3B; the project page is mcg-nju.github.io/VideoJev and ModelScope hosts a VideoJev-4B studio demo. predict_video returns per-second sigmoid probabilities plus a contiguous interval for grounding (default threshold 0.25) and softmax probabilities with answer_index for QA. Training is two stages (head warmup, then joint) with vision frozen per README tables.
Problem: Video agents need when something happened and which option is right without paying for long autoregressive answers on every query over the same footage.
Built for: Researchers and engineers with NVIDIA GPUs who will run ZhuYuhan/VideoJev-4B or VideoJev-35B-A3B locally, read arXiv 2610.18615, and fine-tune on custom JSONL via XTuner.
First indexed on Jev Directory: 2026-10-18
Creator and team
- Name
- Yuhan Zhu
- Organization
- MCG, Nanjing University
- Handle
- @MCG-NJU
How Jev is used
- Role in the product flow
- Direct candidate scoring over video state: sigmoid membership on timestamp intervals for grounding and softmax Choice over answer strings for QA, with cached or joint multi-query inference
- Primitives
- ChoiceScore
- State in
- Video tensor or path plus task questions: grounding query strings or choice tasks with explicit options list per README predict_video example.
- Decision out
- Grounding: probabilities array and interval in seconds; QA: probabilities, zero-based answer_index, and answer text. Joint inference batches queries on one forward pass.
- Install Python 3.12, PyTorch 2.8, CUDA 12.8, FlashAttention 2 per requirements.txt and scripts/env.sh
- Load ZhuYuhan/VideoJev-4B or VideoJev-35B-A3B with AutoModel and AutoProcessor, trust_remote_code
- model.prepare_for_inference() then predict_video with fps, max_frames, and context limits
- Optional XTuner two-stage train: head_only then joint with init-head checkpoint
- Evaluate with scripts/evaluate_hf.py on official benchmarks under data/benchmarks
VideoJev is an open research video decision stack, not the hosted api.typesafe.ai product. liqiiiii-jeva and yinsongxu-llm2jev document text-first System One training and HTTP serve paths; VideoJev is the MCG paper line for long-video grounding and multiple-choice QA with public VLM checkpoints. liuziyu77-valen trains another multimodal decision head; VideoJev publishes TimeLens-trained weights and evaluation scripts instead. deepaleph-jevreel spends TypeSafe Jev on contact-sheet screening for agent search; VideoJev scores intervals inside the model. This directory does not claim TypeSafe affiliation for VideoJev. Cite github.com/MCG-NJU/VideoJev README, docs/DATA.md, docs/EVALUATION.md, and arXiv:2610.18615 only.
Sourced performance claims
- README Overview: temporal grounding and multiple-choice video QA through direct candidate scoring, without text generation.Source: github.com/MCG-NJU/VideoJev README
- README Results section labels leaderboard tables as of 2026-10-17 for temporal grounding and video question answering benchmarks.Source: github.com/MCG-NJU/VideoJev README Results
- arXiv 2610.18615 Table 1 narrative: VideoJev-4B reaches 65.3 mean grounding mIoU; VideoJev-35B-A3B reaches 67.7 mean mIoU and both sizes lead reported open-source grounding metrics in the paper tables with about 2.8 to 3.1x measured speedups versus generative Qwen backbones on Charades.Source: arXiv:2610.18615
- Public GitHub repo MCG-NJU/VideoJev had 165 stars and 7 forks as of October 18, 2026; created 2026-10-08.Source: GitHub API, 2026-10-18
Features and stack
Features
- VideoJev-4B on Qwen3.5-4B and VideoJev-35B-A3B on Qwen3.6-35B-A3B BF16 checkpoints
- predict_video grounding with configurable threshold and QA softmax over listed options
- Cached and joint multi-query inference documented in paper Figure 1 and README architecture figure
- XTuner training pipeline with head warmup then joint stage; vision encoder frozen
- ModelScope studio demo and Hugging Face ZhuYuhan/VideoJev-Demo sample video
- pytest suite under tests with VIDEOJEV_TEST_MODEL optional tokenizer checks
Stack
- Python 3.12
- PyTorch 2.8
- CUDA 12.8
- FlashAttention 2
- XTuner for training
- Hugging Face Hub
Pricing: Apache 2.0 code and public weights; you pay for GPUs, electricity, and Hugging Face bandwidth. No TypeSafe API required for local inference.
Links
FAQ
- Is VideoJev affiliated with TypeSafe?
- No. VideoJev is an independent MCG research release with its own GitHub repo, Hugging Face weights, and arXiv paper. It uses a System One style decision head on Qwen VLMs; it is not the hosted TypeSafe Jev API.
- Which checkpoint should I start with?
- README quick start uses ZhuYuhan/VideoJev-4B with predict_video at fps 2.0, max_frames 2048, context 8192. Swap the repo id to ZhuYuhan/VideoJev-35B-A3B for the larger MoE model.
- How does grounding output work?
- README inference output: per-second sigmoid probabilities, an interval in seconds, default threshold 0.25 on predict_video. QA uses softmax over provided options with answer_index zero-based.
- What hardware does training need?
- README training launch examples use torchrun with eight GPUs; 35B-A3B needs scripts/train_qwen36.py and grouped-GEMM extensions per environment notes. Evaluation QA uses up to 256K context per docs/EVALUATION.md.
- Where is the paper?
- README citation lists arXiv:2610.18615, VideoJev: System One Model for Video Understanding and Interaction, Zhu et al., 2026.
Related learn guides
Original Jev guidance that pairs with this product pattern.
- System One model
The model family behind Jev: parallel typed questions, one forward pass, probabilities you can threshold.
- Jev vs LLM classification
When to gate with System One probabilities instead of asking a chat model to label things.
Related products
Hand-picked neighbors with rich profiles or overlapping tags.