We aim to build increasingly capable and general artificial intelligence (AI) systems, working to better understand the world and create AI tools. This allows useful knowledge to be transferred between many different types of tasks.
Using reinforcement learning, AI Systems Alphazero and Muzero achieved superhuman performance playing games. Since then, we’ve expanded our capabilities to help design better computer chips, along with data center and video compression optimizations. A specialized version of Alphazero, called Alphadev, has also discovered new algorithms for accelerating software at the foundations of our digital society.
Early results show the transformative potential of more general-purpose AI tools. Here, we explain how these advances are shaping the future of computing and are already helping billions of people and the planet.
Designing better computer chips
Specialized hardware is essential to ensure that today’s AI systems are resource efficient for large-scale users. But designing and creating new computer chips requires years of work.
Our researchers have developed an AI-based approach to designing more powerful and efficient circuits. By treating circuits like neural networks, we’ve found a way to accelerate chip design and take performance to new heights.
Neural networks are often designed to take user input and produce output such as images, text, or video. Within a neural network, edges connect to nodes in a graph-like structure.
To create the circuit design, the team proposed a circuit neural network. This is a new type of neural network where you will learn how to convert edges into wires and nodes, convert them into logic gates, and connect them together.
Animated illustration of a circuit neural network to learn circuit design. Decide which edges (wires) connect which nodes (logic gates) to improve the overall circuit design.
We optimized the learning circuit for computational speed, energy efficiency, and size while preserving its functionality. Using “simulated annealing”, a classic search technique that takes us a step further into the future, we tested various options to find the optimal configuration.
This technique won me the IWLS 2023 programming contest. It features the best solution in 82% of circuit design problems in the competition.
Our team has also improved the circuit design by treating it like a game of solving challenges, using Alphazero where we can see many steps into the future.
So far, research combining circuit neural networks with the reward features of reinforcement learning has shown very promising results for building even more advanced computer chips.
Optimize data center resources
Data centers manage everything from serving search results to processing datasets. Like a game of multidimensional Tetris, a system called Borg manages and optimizes workloads within Google’s vast data centers.
To schedule tasks, Borg relies on manually coded rules. However, at Google’s scale, hand-coded rules cannot cover the ever-changing and varied workload distribution. Therefore, they are designed as a one size fits all.
This is where machine learning techniques like Alphazero are especially useful. They can operate at scale and automatically create individual rules that are optimally tuned to different workload distributions.
During training, Alphazero learned to recognize patterns in the tasks entering the data center and also learned to anticipate the best ways to manage capacity and make decisions with the best long-term results.
When applied to Borg in experimental tests, AlphaZero was found to be able to reduce the proportion of unused hardware in data centers by up to 19%.
Animated visualization of neat and optimized data storage vs. messy and unoptimized storage.
Compress videos efficiently
Video streaming makes up a large portion of internet traffic. Therefore, finding ways to make streaming more efficient, no matter how small, will have a huge impact on the millions of people who watch video every day.
We worked with YouTube to compress and send videos using Muzero’s problem-solving capabilities. By reducing Bitrate by 4%, Muzero enhanced the overall YouTube experience without compromising visual quality.
We first applied Muzero to optimize the compression of individual video frames. We have now extended this work to make decisions about how frames are grouped and referenced during encoding, leading to even more bitrate savings.
The results of these first two steps show great potential for Muzero to become a more generalized tool, capable of finding optimal solutions throughout the video compression process.
Visualization showing how Muzero compresses video files. Define groups of photos with visual similarities for compression. A single keyframe is compressed. Muzero uses keyframes as references to compress other frames. This process repeats for the rest of the video until compression is complete.
Discovering faster algorithms
Alphadev, a version of Alphazero, made a new breakthrough in computer science. These basic processes are used trillions of times a day to sort, store, and retrieve data.
Alphadev’s sorting algorithm
Sorting algorithms help digital devices process and display information, from ranking online search results and social posts to user recommendations.
Alphadev has discovered an algorithm that increases the efficiency of sorting short sequences of elements by 70%, or about 1.7%, in sequences containing more than 250,000 elements compared to algorithms in the C++ library. This means that results generated from user queries can be sorted much faster. When used on a large scale, this saves a large amount of time and energy.
Alphadev’s hashing algorithm
Hashing algorithms are often used for data storage and retrieval, such as in customer databases. Typically, a key (such as the username “Jane Doe”) is used to generate a unique hash. This corresponds to the data value that you need to retrieve (for example, “Order number 164335-87”).
Like a librarian who uses a classification system to quickly find a particular book, using a hashing system the computer already knows what it is looking for and where it is located. When applied to the 9-16 byte range of data center hash functions, Alphadev’s algorithm improved efficiency by 30%.
Impact of these algorithms
Added sorting algorithm to LLVM standard C++ library. This replaced a subroutine that had been in use for over 10 years. Contributed Alphadev’s hashing algorithm to the Abseil library.
Since then, millions of developers and businesses have started using them in industries as diverse as cloud computing, online shopping, and supply chain management.
Versatile tools to power your digital future
Our AI tools are already saving billions of people time and energy. This is just the beginning. We envision a future where general-purpose AI tools help optimize the global computing ecosystem.
We’re not there yet – we still need faster, more efficient and sustainable digital infrastructure.
Many more theoretical and technological breakthroughs are needed to create fully generalized AI tools. But the possibilities of these tools – technology, science, medicine too – we’re excited about what’s on the horizon.
Learn more about Alphadev
