Can Frontier Models See Past a Decoy? Elorian Tests Visual Reasoning

Elorian has launched its first visual reasoning model, Glimpse, a system designed to interpret the physical world from images. To assess its performance, the company partnered with Mercor to create the DecoyCount benchmark, which tests whether models count objects accurately or rely on visual shortcuts. In internal trials, Glimpse achieved 50.48 % accuracy on 1,660 items, surpassing the leading models in the academic study, whose best result was 43.17 % by GPT‑6 Astra. Other systems such as Gemini‑3.8‑flash, Qwen3.8‑max, and Claude‑Fable‑5.1 scored below 33 %.

The report notes that the most advanced models fail to transfer their general reasoning abilities to aligned vision tasks, highlighting a gap between textual understanding and visual interpretation. DecoyCount addresses this by combining vision and reasoning tests, distinguishing perception errors (missing an object) from judgment errors (counting a reflection as an extra item). This distinction helps pinpoint the source of problems and guide targeted improvements.

The news is significant because it shows that, despite major advances in language models, effective integration of vision and reasoning remains an open challenge. Glimpse’s results suggest that new architectures focused on “visual thinking” could close this gap, enabling more reliable applications in robotics, automated inspection, and human‑machine interaction.

Read the original article on Latent Space

This summary is an informational synthesis produced by dataqbs.com. All rights to the original content belong to its author and the cited media outlet. We act solely as curators of technology news and claim no authorship.