Deep Neural Networks and Multigrid Methods

In this talk, I will first give an introduction to several models and algorithms from two different fields: (1) machine learning, including logistic regression, support vector machine and deep neural networks, and (2) numerical PDEs, including finite element and multigrid methods.  I will then explore mathematical relationships between these models and algorithms and demonstrate how such relationships can be used to understand, study and improve the model structures, mathematical properties and relevant training algorithms for deep neural networks. In particular, I will demonstrate how a new convolutional neural network known as MgNet, can be derived by making very minor modifications of a classic geometric multigrid method for the Poisson equation and then explore the theoretical and practical potentials of MgNet.