arXiv (Cornell University)
Quantifying artificial intelligence through algorithmic generalization
November 2024 • Takuya Ito, Murray Campbell, Lior Horesh, Tim Klinger, Parikshit Ram
The rapid development of artificial intelligence (AI) systems has created an urgent need for their scientific quantification. While their fluency across a variety of domains is impressive, AI systems fall short on tests requiring algorithmic reasoning -- a glaring limitation given the necessity for interpretable and reliable technology. Despite a surge of reasoning benchmarks emerging from the academic community, no theoretical framework exists to quantify algorithmic reasoning in AI systems. Here, we adopt a fram…