skip to content

Langevin MCMC: theory and methods

Presented by: 
Eric François Moulines
Friday 7th July 2017 - 09:00 to 09:45
INI Seminar Room 1
Nicolas Brosse, Ecole Polytechnique, Paris
Alain Durmus, Telecom ParisTech and Ecole Normale Supérieure Paris-Saclay
Marcelo Pereira, Herriot-Watt University, Edinburgh

The complexity and sheer size of modern datasets, to whichever increasingly demanding questions are posed, give rise to major challenges. Traditional simulation methods often scale poorly with data size and model complexity and thus fail for the most complex of modern problems.
We are considering the problem of sampling from a log-concave distribution. Many problems in machine learning fall into this framework,
such as linear ill-posed inverse problems with sparsity-inducing priors, or large scale Bayesian binary regression.

The purpose of this lecture is to explain how we can use ideas which have proven very useful in machine learning community to
solve large-scale optimization problems to design efficient sampling algorithms.
Most of the efficient algorithms know so far may be seen as variants of the gradient descent algorithms,
most often coupled with « partial updates » (coordinates descent algorithms). This, of course, suggests studying methods derived from Euler discretization of the Langevin diffusion. Partial updates may in this context as « Gibbs steps »This algorithm may be generalized in the non-smooth case by « regularizing » the objective function. The Moreau-Yosida inf-convolution algorithm is an appropriate candidate in such case.

We will prove convergence results for these algorithms with explicit convergence bounds both in Wasserstein distance and in total variation. Numerical illustrations will be presented (on the computation of Bayes factor for model choice, Bayesian analysis of high-dimensional regression, aggregation of estimators) to illustrate our results.
The video for this talk should appear here if JavaScript is enabled.
If it doesn't, something may have gone wrong with our embedded player.
We'll get it fixed as soon as possible.
University of Cambridge Research Councils UK
    Clay Mathematics Institute London Mathematical Society NM Rothschild and Sons