Hybrid LLM judge cuts Opus costs by 4x for identity resolution
Original: A Jev that can ask Opus for help gives Opus accuracy at 4x lower cost
Why This Matters
Shows a practical cost-accuracy trade-off pattern for LLM-as-judge systems at production scale.
Hightouch tested cheap decision models Jev and GPT-6 Luna as identity resolution judges, finding that a Jev+Claude Opus hybrid hits frontier accuracy at over 4x lower cost than running Opus alone.
Hightouch engineers Albert Jin, Neil Desai, and Delaney Mackenzie published results from an experiment asking whether lightweight decision models could replace expensive frontier LLMs when grading identity resolution — the process of stitching together user records across data sources.
Out of the box, both Jev and GPT-6 Luna performed reasonably well: 88.8–97.3% accuracy. Luna edged out Jev on standard cases, but that lead evaporated on harder edge cases. The key difference: Jev's confidence scores were better calibrated. When Jev got something wrong, it usually flagged low confidence — which made it predictable enough to route selectively to a stronger model.
That insight powered the hybrid design. Jev handles the bulk of evaluations cheaply; when its confidence falls below a set threshold, it escalates to Claude Opus. The result is Opus-level accuracy at more than 4x lower cost.
The team also tested GPT-6 Luna in the same hybrid setup, but Jev's superior calibration made it the better trigger model. The post notes that picking the right confidence threshold is non-trivial — too tight and costs climb back up; too loose and accuracy slips.