“Classic diffusion models add small Gaussian noise (normal random variables with small amplitudes) in stages to convert the data distribution into a simple standard normal distribution. The model then trains the function and specifies the function to return the standard regular random variable to what follows the data distribution, and specifying “denoisy,” MA said.
However, according to the MA, the researchers do not need incremental updates to become small noises of Gaussian noise. Instead, they consider large jumps during steps following distributions beyond the normal distribution. These are distributions with long-term tails or distributions generated by subroutine algorithms. Using this technique, researchers reduced the number of mediation procedures, accelerated the algorithms for diffusion models, making them more widely applicable to a variety of tasks.
“We see that this generalization increases the efficiency of diffusion models. It could potentially lead to much wider use of diffusion models, such as language generation and, more interestingly, long-term inference and decision-making,” Ma said.
In addition to the MA, the research team includes Xupeng Huang, a current visiting student at HDSI. Tong Zhang from Uiuc. Difan Zou and Yi Zhang from HKU. Hanze Dong from Salesforce.
“The most exciting thing about this work is that we can utilize almost every intermediate transition step that can accelerate the algorithm and allow the algorithm to be applied more widely to various downstream tasks,” MA said. “We expect this work to apply to text and multimodal generation, long-term inference, tool use, problem-solving, and decision-making tasks to accelerate and improve the outcomes of such tasks.”
Learn more about research and education in San Diego, California: Artificial Intelligence
