Advances in artificial intelligence, particularly large language models (LLMs), have been driven by the "scaling law" paradigm: performance improves with more data, computation, and larger models. However, this approach reveals a critical vulnerability when confronting frontier domains where usable data is inherently scarce. In these environments, LLMs often fail at the complex, multi-step reasoning required for true innovation.
In a new Perspective article published in National Science Review, Professor Xiaotie Deng and Ph.D. Candidate Hanyu Li from Peking University argue that the path forward requires a fundamental shift in strategy. Instead of relying on scaling alone, they propose augmenting the powerful statistical intuition of LLMs with symbols derived from human wisdom.
"The challenges LLMs face in data-scarce environments don't mean the scaling paradigm has reached its ceiling," explains Dr. Deng. "Rather, it signals that we need to integrate a distinctly human capability: using symbols as a cognitive technology to map and navigate complexity."
The authors illustrate this principle with historical examples, from the Pirahã people, whose language lacks number words, limiting their ability to recall exact quantities, to the triumph of Leibniz's calculus notation over Newton's, which provided a more intuitive framework for thought. A well-designed symbol system, they argue, is not just a label but a powerful tool for reasoning.
This synergy is powerfully demonstrated by AlphaGeometry, an AI system that reached a gold-medal level in the International Mathematical Olympiad. The system combines an LLM, trained on a human-designed symbolic language for geometric constructions, with a deductive solver. The LLM makes an intuitive leap to propose a constructive step, which the symbolic engine then efficiently explores.
"This neuro-symbolic synthesis overcomes the limitations of both purely statistical and purely symbolic paradigms," says Hanyu Li. "If scaling laws have given models their powerful intuition, it is the art of the symbol that will provide the compass for genuine discovery."
The authors suggest this approach opens promising new research fields, including automated algorithm design with theoretical guarantees, combinatorial optimization, and optimized code generation for specific hardware. The central task ahead, they conclude, is the art of symbolization itself—crafting powerful abstractions to guide the next generation of AI.
Journal
National Science Review