SOLO
Prefix-cache-aware batch Jev decisions over Pandas, NumPy, or JSON rows: reorder fields for throughput, not NL-to-SQL.
Overview
SOLO (github.com/0814wdwd/solo_jev, MIT) is the System One Layout Optimizer: it runs a natural-language decision over every record, reorders rows and fields so prefix-caching backends reuse input computation, then returns decisions aligned with your original row order. The backend still receives the complete record each time. README reports up to 11.7x throughput over original order and 3.28x over NDV sorting on measured RTX 4090 synthetic workloads (see docs/benchmarks.md). The default packaged path serves AutoTrust JEV-9B through vLLM; docs/deployment.md and docs/vllm-jev.md cover hosting and the Open-Jev adapter. SOLO builds on the ICML 2026 PMLR paper Prefix-Cache-Aware Data Reordering for LLM-Augmented Database Analytics (proceedings.mlr.press/v306/li26gn.html).
Problem: Running the same semantic decision across thousands of table rows repeats huge shared context, so prefix caches help little when field order is wrong.
Built for: Data and ML engineers batching Jev or JEV-9B judgments over Pandas, NumPy, or JSON tables who want throughput without turning the problem into NL-to-SQL.
First indexed on Jev Directory: 2026-10-08
Creator and team
- Name
- 0814wdwd
- Handle
- @0814wdwd
How Jev is used
- Role in the product flow
- Batch per-row semantic decision with layout planning ahead of inference
- Primitives
- ChoiceNoulScore
- State in
- Each row as a complete record (Pandas DataFrame, NumPy, JSON/JSONL, or nested objects per docs/inputs.md) plus one shared natural-language question.
- Decision out
- Per-row decisions and probabilities in original index order after scan(); binary, categorical, or scored heads per JEV-9B metadata.
- Normalize tabular or JSON input to the engine contract
- Planner reorders rows and fields for prefix reuse (SOLO layout)
- DecisionEngine issues batched requests to local JEV-9B or your backend
- Results map back to the caller's row order
SOLO is not a SQL generator. realzachi-pg-jev adds a jev() predicate inside Postgres WHERE clauses; sheltercosmo-jev4pg plans SQL from English or Chinese questions plus 41 operators. SOLO keeps your relational rows and asks the same decision question across all of them, optimizing layout for decision-model throughput. Example README quickstart: refund policy text plus customer_request rows scanned with Does the request ask for a monetary refund? Use examples/offline_layout.py without a GPU to see planning only. Pair with hosted TypeSafe Jev by swapping the backend when you outgrow the bundled JEV-9B server. Cite the PMLR paper and SOLO version when publishing benchmarks.
Sourced performance claims
- README and docs/benchmarks.md: up to 11.7x throughput vs original order and 3.28x vs NDV sorting on one NVIDIA RTX 4090 with AutoTrust JEV-9B, BF16, vLLM 0.31.0, 128 records per synthetic case.Source: github.com/0814wdwd/solo_jev docs/benchmarks.md
- GitHub 0814wdwd/solo_jev had 105 stars on 2026-10-08.Source: GitHub API stargazers_count, 2026-10-08
Features and stack
Features
- DecisionEngine.scan for Pandas, JSON/JSONL, and NumPy
- Documented RTX 4090 reproducible demos in docs/benchmarks.md
- AutoTrust JEV-9B serving path and vLLM Open-Jev adapter docs
- Offline layout demo without network or GPU
- ICML 2026 citation block in README
Stack
- Python 3.10+
- solo_layout package
- vLLM
- AutoTrust JEV-9B
Pricing: Open source client; GPU hosting and TypeSafe or self-hosted inference costs are yours.
Links
FAQ
- Is SOLO text-to-SQL?
- No. It batch-applies one semantic decision per row. jev4pg and pg-jev solve different Postgres problems; see those listings.
- Do 11.7x speedups apply to every dataset?
- No. README positions them as reproducible synthetic workloads on one RTX 4090. Read docs/benchmarks.md before quoting in production decks.
- Which open model ships by default?
- README points at AutoTrust JEV-9B on Hugging Face, served through vLLM per docs/deployment.md.
Related learn guides
Original Jev guidance that pairs with this product pattern.
- System One model
The model family behind Jev: parallel typed questions, one forward pass, probabilities you can threshold.
- Jev use cases
The patterns builders actually search for: moderation, routing, triage, RAG verify, and agent gates.
Related products
Hand-picked neighbors with rich profiles or overlapping tags.