Laya: Open 33ms Decision Engine Predates Jev by a Year
Original: I built non-autoregressive decision models with RL a year ago
Why This Matters
Non-autoregressive decision models could replace LLMs in high-volume classification pipelines, cutting latency and cost significantly.
ConvAI Innovations founder Nandakishor Mukkunnoth published a non-autoregressive RL-based decision model on arXiv in March 2025, more than a year before TypeSafe AI launched Jev with similar claims. His open-source response, Laya, runs in 32.8ms on a single GPU — 6-8x faster than Jev — with Apache 2.0 weights and 100+ language support.
In March 2025, Mukkunnoth published arXiv:2503.23303, releasing open weights on Hugging Face and a PyPI package for a non-autoregressive decision model trained with PPO over sales conversation trajectories. A second paper followed in September 2025 (arXiv:2510.01237), formalizing schema-based decisions guided by reinforcement learning. Then in September 2026, well-funded startup TypeSafe AI — co-founded by Diogo Almeida, a co-inventor of ChatGPT — launched Jev, a non-autoregressive decision model using what it called RLCD (Reinforcement Learning for Calibrated Decisions), priced at $0.042 per million input tokens with ~150ms latency. No technical papers, no open weights, no open datasets. Rather than simply complain, Mukkunnoth rebuilt the concept from scratch as Laya: a bidirectional-encoder-based System 1 decision engine that outputs calibrated probability distributions over structured schemas in 32.8ms on a T4 GPU (7.2ms batched). The core argument is that generative LLMs are architectural overkill for routine classification tasks — routing, spam detection, jailbreak detection, urgency scoring — where waiting 500-2000ms and paying frontier inference rates makes no sense. Laya handles 100+ languages and ships fully open-source under Apache 2.0.