Advance best-in-class large-scale models, computing optimized RL agents, and more transparent, ethical and fair AI systems
The 36th International Conference on Neural Information Processing Systems (Neurips 2022) is a hybrid event based in New Orleans, USA, and will be held from November 9th to September 2022.
Neurips is the world’s largest conference in artificial intelligence (AI) and machine learning (ML), and we are proud to help promote the exchange of research advances in the AI ​​and ML communities.
The full team at DeepMind has published 47 papers, including 35 external collaborations in virtual panels and poster sessions. This is a brief introduction to some of the research we present.
Best in class large model
Large-scale models (LMS) – Generated AI systems trained with a huge amount of data – have delivered incredible performance in areas such as language, text, audio, image generation and more. Part of their success depends on their pure scale.
However, Chinchilla has created a 70 billion parameter language model, surpassing many larger models, including Gopher. We updated the scaling laws for large models to show that previously trained models are too large for the amount of training performed. This piece has already formed other models that follow these updated rules, creating slimmer and better models, and has won a conference-rewarded Main Track Paper Award.
Based on the chinchilla and multimodal model NFNETS and perceptors, Flamingo, a family of small number of learning visual language models, is also featured. Processing Image, Video and Text Data Flamingo represents the bridge between vision-only and language-only models. The single flamingo model sets up new and latest technology in small numbers of shot learning with a wide range of open-ended multimodal tasks.
Still, scale and architecture are not the only factors that are important for the power of transformer-based models. Data properties also play an important role. This is discussed in our presentation on data properties that facilitate contextual learning in transformer models.
Optimizing Reinforcement Learning
Reinforcement learning (RL) is a promising approach to creating generalized AI systems that can handle a wide range of complex tasks. It has brought breakthroughs in many domains from Go To Mathematics. I’m always looking for ways to make RL agents smarter and lean.
By significantly increasing the size of information available for search, we introduce a new approach that enhances the decision-making capabilities of RL agents in a computationally efficient way.
We also introduce a conceptually simple yet general approach for curiosity-driven exploration in visually complex environments: an RL agent called Byol-Explore. It achieves superhuman performance while being robust to noise and much simpler than previous work.
Advances in algorithms
From data compression to running simulations to predict weather, algorithms are the fundamental part of modern computing. Therefore, progressive improvements can have a significant impact on large-scale tasks and can help save energy, time and money.
Based on neural algorithm inferences, it shares a fundamentally new and highly scalable method for automated configuration of computer networks. Constraints.
During the same session, we also present a rigorous investigation of previous theoretical concepts of “algorithm alignment” to highlight the nuanced relationships between graph neural networks and dynamic programming, and combine them to optimize distributed performance. Emphasise the best way to do this.
Responsible pioneer
At the heart of Deepmind’s mission is our commitment to acting as responsible pioneers in the field of AI. We are committed to developing transparent, ethical and fair AI systems.
Describe and understand the behavior of complex AI systems is an integral part of creating a fair, transparent and accurate system. We provide a set of decidelatora that captures those ambitions and explain practical ways to satisfy them.
To act safely and ethically in the world, AI agents must be able to reason about harm and avoid harmful behavior. We present a collaboration on a new statistical measure known as counterfactual harm and show how to overcome the problems of standard approaches to avoid pursuing harmful policies.
Finally, we present a new paper that proposes ways to diagnose and mitigate impairments in model equity caused by distribution shifts, and demonstrates how important these issues are for the deployment of safe ML technologies in healthcare settings. I will.
Find out the full range of our work at Neurips 2022 here.
