Artificiality's First Exam: An LLM-Referenced C-test for Supraintelligence

The authors propose a new way to gauge the capabilities of artificial‑intelligence systems that surpass both human performance and existing models. Traditional benchmarks tend to plateau because they are calibrated against the performance of their designers, making them less useful for assessing potentially superior intelligences. To break this cycle, the study turns to inverse inductive inference problems, whose difficulty depends on the underlying mechanism that generates observations and the prior knowledge available. They employ Levin’s Kt measure, a computable variant of Kolmogorov complexity that combines the shortest description of the mechanism with the logarithm of the time required to execute it. Unlike earlier tests that relied on very simple universal Turing machines, here large language models (LLMs) serve as reference machines, already embedding a broad human cultural base and capable of receiving natural‑language instructions. The paper explains how the metric is formalized, how a lightweight process is created to generate and validate items, and how the influence of the reference machine’s power and the distinction between the test designer and the testee are analyzed. A first set of certified questions is presented, applied to twelve LLMs, whose performance curves decline as difficulty rises, reaching a high‑complexity zone that has not yet been overtaken by frontier intelligences. This research is significant because it offers both a theoretical and practical tool for tracking AI progress beyond current limits, enabling early detection of supra‑intelligent capabilities.

Read the original article on arXiv cs.AI

This summary is an informational synthesis produced by dataqbs.com. All rights to the original content belong to its author and the cited media outlet. We act solely as curators of technology news and claim no authorship.