Laya-MLX on Apple Silicon

The upstream Laya stack typically uses PyTorch and Transformers. Laya-MLX ports weights and the decision heads to MLX so developers on Mac can run inference without a CUDA box. Tokenization still uses Hugging Face tokenizers.

The project documents numerical parity checks against upstream on fixed fixtures.

md in the laya-mlx repo measures end-to-end short-input latency on Apple M3 Max (40-core GPU, 128 GiB RAM). 4 ms (laya-multilingual), with model load excluded. Ten-question batches and full-context runs are much slower. These numbers are one machine and one OS build.

They are not a guarantee on M2, M4, or laptops with less memory.

The same document shows PyTorch MPS FP32 somewhat slower than MLX FP16 on identical hashes. That is expected when FP16 is allowed. Throughput rows use repeated question templates. Distinct-question production mixes may differ.

Community posts (including our showcase blurb for Lonely__MH's snake demo) sometimes contrast local tens-of-ms inference with hosted Jev latency. That is apples to oranges. No TLS, no multi-tenant queue, different hardware class. Use Laya-MLX when Mac-local loops matter.

Do not claim a universal 50x speedup over your Jev region without measuring.

Clone github.com/mizorewww/laya-mlx, follow uv sync extras in the README, and pin upstream Laya revisions the benchmark scripts expect. PyPI lists laya-mlx for packaged installs. For Convai's official path on Linux/GPU, use the laya PyPI package instead.

jev.aitools.fyi embeds a public X clip where snake ticks use local Laya decisions. Useful for sensing decision frequency, not for SLA proof: /showcase?demo=laya-mlx-lonely-mh.