typesafe-computer-use
Computer use for about $0.
by @awlevin
Overview
typesafe-computer-use (github.com/awlevin/typesafe-computer-use) is Aaron Levin's Python clicker for real desktop automation. OCR and accessibility APIs turn the screen into numbered actions; each step sends one TypeSafe Jev Choice for operation and target; TYPE_TEXT, URL proposals, and final answers delegate to a compact writer model. Default runs are dry-run; pass --act to drive mouse and keyboard. README comparison tables claim about 155x lower per-step cost and 14x to 40x faster model latency versus Claude Opus 5 on a bare screenshot for the same decision (author measured; rerun locally).
Problem: Screenshot-first computer use agents burn dollars and seconds on steps that are really one click from a short list.
Built for: macOS builders who want a dry-run friendly clicker with TypeSafe Jev routing and a small writer only when typing matters.
First indexed on Jev Directory: 2026-09-17
Creator and team
- Name
- Aaron Levin
- Handle
- @awlevin
“Most steps do not need a plan. They need one choice from a short list, made quickly and cheaply, with a confidence number you can gate on.”
How Jev is used
- Role in the product flow
- Per-step macOS computer-use Choice over OCR and accessibility action lists
- Primitives
- ChoiceScore
- State in
- Structured screen capture: element indices, roles, text, and deterministic date parsing helpers described in README (not raw pixels for the classifier path).
- Decision out
- Next operation and target index with calibrated confidence; writer model only on TYPE_TEXT or answer paths.
- Capture screen via accessibility plus OCR into finite actions
- One Jev Choice selects operation and compatible target
- Execute click, scroll, or navigation directly when safe
- Invoke writer model for free-text fields or final natural-language answers
Aaron's loop is the anti-screenshot pattern: deterministic parsers build state, Jev picks among finite UI actions, and only the exceptions touch a chat model. That is why related browser-use listings feel slower by comparison: they still ship pixels to frontier models every step. Dry-run mode lets you watch decisions without touching the cursor; --act is explicitly beta because real mice are scary. The showcase clip on X walks through TechCrunch ticket checkout style goals; embed uses remote twimg with referrerPolicy no-referrer like other directory videos. Do not confuse this repo with Milind's CoreML madewithjev computer-use build; this listing tracks awlevin's GitHub and Aaron's post only.
Sourced performance claims
- README table cites about $0.0002 per decision versus about $0.032 for Opus 5 on the same screenshot step.Source: github.com/awlevin/typesafe-computer-use README
- README cites 0.13 to 0.38 s model latency versus 5.2 s for Opus 5 on the measured step.Source: github.com/awlevin/typesafe-computer-use README
- Public GitHub repo awlevin/typesafe-computer-use had about eight hundred twenty eight stars when this listing was drafted.Source: GitHub star count September 2026
Features and stack
Features
- clicker CLI with dry-run default and --act for live control
- TypeSafe Jev every step; Anthropic writer optional for text
- Documented cost and latency tables versus Opus screenshot baseline
- macOS first with experimental Windows notes in README
Stack
- Python
- macOS accessibility
- OCR
- TypeSafe Jev
- Anthropic writer models
Pricing: Open source; TypeSafe per-step fees plus optional writer model usage per README measurements.
Demo
Showcase clip with the same lightbox player as the homepage. Click to play.
typesafe-computer-use without screenshot theater
OCR builds the menu, Jev picks the click for fractions of a cent, and Haiku only types when the classifier admits defeat. Your Opus screenshot budget can stay asleep.
Links
FAQ
- Does Jev see screenshots?
- The classifier path uses structured OCR and accessibility state per README. Writer models may use vision for final answers when enabled.
- Is --act safe on day one?
- README warns the tool is beta and drives your real mouse. Start with dry runs until thresholds look sane.
- Where are the speedup numbers from?
- They come from Aaron's README comparison table on the same goal and screenshot. Reproduce with the documented harness before quoting in prod decks.
Related learn guides
Original Jev guidance that pairs with this product pattern.
- Jev use cases
The patterns builders actually search for: moderation, routing, triage, RAG verify, and agent gates.
- Jev vs LLM classification
When to gate with System One probabilities instead of asking a chat model to label things.
Related products
Hand-picked neighbors with rich profiles or overlapping tags.