AlphaGeometry: Olympic-level geometry AI system

Our AI systems go beyond state-of-the-art approaches to geometry problems and advance AI reasoning in mathematics

Reflecting the Olympic spirit of ancient Greece, the International Mathematics Olympiad is a modern-day arena for the world’s most talented high school mathematicians. The competition not only showcases young talent but has also emerged as a testing ground for advanced AI systems in mathematics and reasoning.

In a paper published today in Nature, we introduce AlphaGeometry, an AI system that solves complex geometry problems at a level approaching that of human Olympic gold medalists. This is a breakthrough in AI performance. In a benchmark test of 30 Olympic geometry problems, AlphaGeometry solved 25 problems within the standard Olympic time limit. For comparison, the previous state-of-the-art system solved 10 of these geometry problems, and the average human gold medalist solved 25.9 problems.

AI systems often struggle with complex problems in geometry and mathematics due to a lack of inference skills and training data. AlphaGeometry’s system combines the predictive power of neural language models with a rule-bound inference engine that works together to find solutions. And by developing a method to generate a huge pool of synthetic training data (100 million unique examples), we can train AlphaGeometry without human demonstration and avoid data bottlenecks. Masu.

Use AlphaGeometry to demonstrate AI’s improved ability to reason, discover and validate new knowledge. Solving Olympic-level geometry problems is an important milestone in developing deep mathematical reasoning for more advanced and general AI systems. We are open sourcing AlphaGeometry’s code and models in the hope that, in combination with other tools and approaches in synthetic data generation and training, it will help unlock new possibilities across mathematics, science, and AI. I am.

AlphaGeometry takes a neurosymbolic approach

AlphaGeometry is a neural symbolic system consisting of a neural language model and a symbolic deduction engine that work together to find proofs of complex geometric theorems. Similar to the idea of ​​”thinking fast and then slow,” one system provides quick, “intuitive” ideas, and the other system provides more careful, rational decision-making.

Language models are good at identifying common patterns and relationships in data, allowing them to quickly predict potentially useful constructs, but they are often difficult to rigorously infer or They lack the ability to explain their decisions. Symbolic deduction engines, on the other hand, are based on formal logic and use explicit rules to reach conclusions. Although these are reasonable and explainable, they can be “slow” and inflexible, especially when dealing with large and complex problems alone.

AlphaGeometry’s language model guides its symbolic deduction engine to possible solutions to geometry problems. Olympic geometry problems are based on diagrams that require new geometric components to be added before solving, such as points, lines, and circles. AlphaGeometry’s language model predicts which new constructs will be most useful to add out of an infinite number of possibilities. These clues help fill in the gaps, allowing the symbolic engine to make further inferences about the diagram and get closer to the solution.

Generates 100 million synthetic data examples

Geometry relies on an understanding of space, distance, shape, and relative position and is the basis of art, architecture, engineering, and many other fields. Humans can use pen and paper to learn geometry, examine diagrams, and use existing knowledge to uncover new and more sophisticated geometric properties and relationships. Our synthetic data generation approach emulates this knowledge building process at scale, allowing AlphaGeometry to be trained from scratch without human demonstration.

The system uses highly parallelized computing to generate 1 billion random diagrams of geometric objects, then calculates all relationships between points and lines within each diagram. I have brought it out thoroughly. AlphaGeometry found all the proofs in each diagram and then worked backwards to figure out any additional construction needed to arrive at those proofs. We call this process “symbolic deduction and traceback.”

That huge data pool was filtered to exclude similar examples, resulting in a final training dataset of 100 million unique examples of varying difficulty, of which 9 million additional The structure was distinctive. Because there are so many examples of how these building blocks have led to proofs, AlphaGeometry’s language model is able to make good suggestions for new building blocks when presented with Olympic geometry problems. Masu.

Pioneer of mathematical reasoning with AI

All Olympic problem solutions provided by AlphaGeometry have been checked and verified by computer. They also compared their results to previous AI methods and human performance at the Olympics. Additionally, math coach and former Olympic gold medalist Evan Chen reviewed some of AlphaGeometry’s solutions.

Chen said: “AlphaGeometry’s output is impressive because it is verifiable and clean. Previous AI solutions to proof-based competition problems have been hit-or-miss (the output is only occasionally correct and requires human checking). AlphaGeometry does not have this weakness; its solution has a machine-verifiable structure; nevertheless, its output is still human-readable. computer solving problems You can even imagine a program with many pages of boring algebraic calculations, but not with the classic geometry rules involving angles and similar triangles. I will use it.”

Each Olympics features six problems, only two of which typically focus on geometry, so AlphaGeometry can only be applied to a third of the problems at a given Olympics. Nevertheless, its geometry capabilities alone make it the world’s first AI model capable of exceeding IMO bronze medal standards in 2000 and 2015.

In geometry, our system is approaching IMO gold medalist standards, but we have our eyes on an even bigger prize: advances in inference for next-generation AI systems. Given the wide potential for training AI systems from scratch using large-scale synthetic data, this approach could shape how future AI systems discover new knowledge in mathematics and other fields. .

AlphaGeometry builds on the work of Google DeepMind and Google Research to pioneer mathematical reasoning with AI, from exploring the beauty of pure mathematics to solving mathematical and scientific problems using language models. I am. And most recently, we introduced FunSearch, which used large-scale language models to make the first discoveries in unsolved problems in the mathematical sciences.

Our long-term goal is to build AI systems that are generalizable across mathematics disciplines, extending the frontiers of human knowledge while developing the advanced problem solving and reasoning that general AI systems rely on. is.