JEV Sees
Vision front end for hosted TypeSafe Jev: one frame, many objects, one Choice or Noul call over RGB, video, or RGB-D.
Overview
JEV Sees (github.com/CharlesFeng0314/JEV_sees, MIT, v0.1.0) is a visual front end for hosted TypeSafe Jev. The README line is one frame, many objects, one JEV call. Sees() is the image and video entry point. observe() tracks objects locally. state() builds the text Jev reads. Choice, Noul, Score, and TypeSafeClient are re-exported from the official typesafe-sdk, so the call is still client.system_one and the answers still live on response.choices, response.nouls, and response.scores. Python 3.10+. pip install -e . from a clone. TYPESAFE_API_KEY is required for system_one. observe() and state() stay local. RGB-only perception starts from Florence-2 dense-region captions (default microsoft/Florence-2-base-ft). Callers do not pass an object vocabulary. RGB-D mode lets depth clusters decide which physical objects exist, and overlapping Florence regions supply free-text names. Color stays evidence: a CV measurement, CLIP's color distribution, and Florence's region caption, so Jev can judge agreement instead of receiving one preselected color string. The README says an earlier YOLO latency table was removed and that a new image and video benchmark is required before publishing latency claims for this backend. Created 2026-09-30. Public counts were 113 stars and 6 forks when this listing was drafted.
Problem: Hosted Jev already returns Choice, Noul, and Score, and a camera frame is still stuck in front of a captioning model when the question was a closed judgment about the objects in view.
Built for: Python teams who want a local vision front end for hosted TypeSafe Jev: images, video, or RGB-D in, official Choice and Noul questions out, with one call covering many tracked objects.
First indexed on Jev Directory: 2026-10-02
Creator and team
- Name
- CharlesFeng0314
- Handle
- @CharlesFeng0314
“One frame. Many objects. One JEV call.”
How Jev is used
- Role in the product flow
- Local perception builds a scene state, then one hosted TypeSafe system_one call judges many named objects
- Primitives
- ChoiceScoreNoul
- State in
- sees.state(question) after observe() on an image, a video sample, or an RGB-D frame. Tracked objects carry object_id, label, confidence, bbox_xyxy, centroid, and color evidence. Video can add frame_index, media time, and the 2 most recent time-aware poses. RGB-D can add position_m and metric gaps when camera intrinsics are available.
- Decision out
- Official typed answers. A bus-color Choice returns response.choices['bus_color'].choice (the included image is expected to print blue). A per-pedestrian Noul returns response.nouls[object_id].noul. Video results are frame-primary in result.frames, with result.rows kept as a per-object peak summary.
- Clone the repo and python -m pip install -e . on Python 3.10 or newer
- Export TYPESAFE_API_KEY. Create the key at console.typesafe.ai. observe() and state() do not need it
- Sees(), observe a path, then TypeSafeClient.system_one with Choice, Noul, or Score objects you construct. JEV Sees does not infer a question type from prose
- For video, pass questions= as a callable that receives current tracks and returns official questions, as examples/traffic_relations.py does for one Noul per pedestrian
- For RGB-D, depth clusters decide which objects exist. Florence names overlapping regions. Weights download on first use and are not stored in the repo
JEV Sees builds the state. Hosted TypeSafe Jev makes the judgment. liuziyu77-valen trains a multimodal decision head (Valen-Preview-0923) and serves it yourself. tianyucodings-nanojev trains 0.6B parallel heads on Maze, Snake, and ViZDoom frames. deepopen-com-deepopen routes text across 3 local checkpoints. nokia-applied-research-anyjev fits calibration heads on text models you already host. This repo publishes no decision checkpoint. Florence-2 and CLIP load from Hugging Face or a local CLIP file, and system_one still goes to TypeSafe. The README also points at a sibling, JEV Control (github.com/CharlesFeng0314/JEV_control_your_roboarm), for turning judgments into robot-arm actions. That sibling is a separate repo. examples/bus_color.py is the smallest Choice. examples/traffic_relations.py builds one official Noul per current pedestrian and requires TYPESAFE_API_KEY. Cite github.com/CharlesFeng0314/JEV_sees README only.
Sourced performance claims
- README product line: one frame, many objects, one JEV call. The traffic demo caption says every visible pedestrian gets an accident-risk probability from the same JEV call. Question objects are official Noul or Choice values, one per object id.Source: github.com/CharlesFeng0314/JEV_sees README
- Perception: RGB-only mode starts from Florence-2 dense-region captions. Default checkpoint microsoft/Florence-2-base-ft, overridable with Sees(florence_model=...). RGB-D mode lets depth clusters decide which physical objects exist. With camera intrinsics, observations can include position_m. CLIP color evidence uses weights/clip/ViT-B-32.pt when JEV_SEES_ROBO_ROOT points at it, otherwise CLIP downloads on first use.Source: github.com/CharlesFeng0314/JEV_sees README Models and weights and RGB-D
- Scene memory: object ids stay stable when possible. Samples reuse pose_history and add frame_index plus media time. The 2 most recent time-aware poses of each visible object enter the Jev state. Color evidence keeps the CV measurement, the CLIP distribution, and Florence's region caption side by side.Source: github.com/CharlesFeng0314/JEV_sees README Perception, tracking, and scene memory
- Status v0.1.0, intentionally experimental. Python 3.10+. License MIT. The README says the earlier YOLO benchmark does not describe the Florence-2 pipeline and was removed. A new image and video benchmark is required before publishing latency claims for this backend.Source: github.com/CharlesFeng0314/JEV_sees README Performance and Status
- Public GitHub repo CharlesFeng0314/JEV_sees had 113 stars and 6 forks on 2026-10-02. Created 2026-09-30. Last push 2026-10-02. Language Python.Source: GitHub API 2026-10-02
Features and stack
Features
- Sees, observe, and state as the local visual layer
- Official Choice, Noul, Score, and TypeSafeClient re-exported for one import line
- One system_one call over many named objects in a frame
- Florence-2 region captions, CLIP color evidence, and optional RGB-D depth clusters
- Video questions as a callable, with frame-primary probabilities
- v0.1.0 experimental status and an explicit refusal to publish a latency number for the Florence-2 path
Stack
- Python 3.10+
- MIT
- typesafe-sdk TypeSafeClient
- Florence-2
- CLIP ViT-B-32
- RGB, video, and RGB-D
Pricing: MIT code. Perception weights download on first use (Florence-2 from Hugging Face, CLIP unless a local ViT-B-32 file is set). system_one calls spend your TypeSafe key. The README publishes no latency or price table for this backend.
Links
FAQ
- Is JEV Sees a multimodal model like Valen?
- liuziyu77-valen trains Valen-Preview-0923, a Qwen3.5 decision head over text, images, and video. JEV Sees does not ship a decision checkpoint. It tracks a scene locally and calls hosted TypeSafe Jev. Weights in this repo's README are Florence-2 and CLIP, used as perception, and they are downloaded rather than stored in the tree.
- How does one call cover many objects?
- Your code builds one official question per object id. The traffic example selects current pedestrians and creates a Noul for each. JEV Sees passes a single state from the frame. response.nouls[object_id] is that pedestrian's probability. The README says the library does not invent questions from the prompt.
- Why is there no latency number on this page?
- The Performance section says the earlier YOLO benchmark does not describe the Florence-2 pipeline and was removed on purpose. It asks for a new image and video benchmark before any latency claim for this backend. This listing does not fill that gap.
- Is this a browser or computer-use agent?
- It is a camera front end: RGB, video, and RGB-D. It does not pick DOM actions or desktop clicks. The README's sibling for action is JEV Control, a separate robot-arm repo. This page covers JEV Sees only.
Related learn guides
Original Jev guidance that pairs with this product pattern.
- System One model
The model family behind Jev: parallel typed questions, one forward pass, probabilities you can threshold.
- Jev vs LLM classification
When to gate with System One probabilities instead of asking a chat model to label things.
Related products
Hand-picked neighbors with rich profiles or overlapping tags.