3 Feb 2017 - Deep Unsupervised Learning using Nonequilibrium Thermodynamics

Computational Statistics Club is back after a brief hiatus, at its usual time (11:00am-12:00pm Friday, February 3, 2017) and place (Z6 fishbowl). Patrick Grinaway be presenting the paper below this week. Briefly, this paper adds to the growing set of methods that allow us to learn so-called deep generative models, with an interesting twist that also maintains tractability. If this idea sounds cool (or you just want to learn about it), join us at the CSC this week to discuss!

https://arxiv.org/abs/1503.03585

"Deep Unsupervised Learning using Nonequilibrium Thermodynamics"

Abstract: A central problem in machine learning involves modeling complex data-sets using highly flexible families of probability distributions in which learning, sampling, inference, and evaluation are still analytically or computationally tractable. Here, we develop an approach that simultaneously achieves both flexibility and tractability. The essential idea, inspired by non-equilibrium statistical physics, is to systematically and slowly destroy structure in a data distribution through an iterative forward diffusion process. We then learn a reverse diffusion process that restores structure in data, yielding a highly flexible and tractable generative model of the data. This approach allows us to rapidly learn, sample from, and evaluate probabilities in deep generative models with thousands of layers or time steps, as well as to compute conditional and posterior probabilities under the learned model. We additionally release an open source reference implementation of the algorithm.