09:00 to 09:40 Registration 09:40 to 09:50 Welcome from Christie Marr (INI Deputy Director) 09:50 to 10:40 James Nagy (Emory University)Spectral Computed Tomography Co-authors: Martin Andersen (Technical University of Denmark), Yunyi Hu (Emory University)An active area of interest in tomographic imaging is the goal of quantitative imaging, where in addition to producing an image, information about the material composition of the object is recovered. In order to obtain material composition information, it is necessary to better model of the image formation (i.e., forward) problem and/or to collect additional independent measurements. In x-ray computed tomography (CT), better modeling of the physics can be done by using the more accurate polyenergetic representation of source x-ray beams, which requires solving a challenging nonlinear ill-posed inverse problem. In this talk we explore the mathematical and computational problem of polyenergetic CT when it is used in combination with new energy-windowed spectral CT detectors. We formulate this as a regularized nonlinear least squares problem, which we solve by a Gauss-Newton scheme. Because the approximate Hessian system in the Gauss-Newton scheme is very ill-conditioned, we propose a preconditioner that effectively clusters eigenvalues and, therefore, accelerates convergence when the conjugate gradient method is used to solve the linear subsystems. Numerical experiments illustrate the convergence, effectiveness, and significance of the proposed method. INI 1 10:40 to 11:10 Morning Coffee 11:10 to 12:00 Eldad Haber (University of British Columbia)tba INI 1 12:00 to 12:50 Christoph Brune (Universiteit Twente)Cancer ID - From Spectral Segmentation to Deep Learning One of the most important challenges in health is the fight against cancer. A desired goal is the early detection and guided therapy of cancer patients. A very promising approach is the detection and quantification of circulating tumor cells in blood, called liquid biopsy. However, this task is similar to looking for needles in a haystack, where the needles even have unclear shapes and materials. There is a strong need for reliable image segmentation, classification and a better understanding of the generative composition of tumor cells. For a robust and reproducible quantification of tumor cell features, automatic multi-scale segmentation is the key. In recent years, new theory and algorithms for nonlinear, non-local eigenvalue problems via spectral decomposition have been developed and shown to result in promising segmentation and classification results. We analyze different nonlinear segmentation approaches and evaluate how informative the resulting spectral responses are. The success of our analysis is supported by results of simulated cells and first European clinical studies. In the last part of this talk we switch the viewpoint and study first results for deep learning of tumor cells. Via generative models there is hope for understanding tumor cells much better, however many mathematical questions arise. This is a joint work with Leonie Zeune, Stephan van Gils, Guus van Dalum and Leon Terstappen. INI 1 12:50 to 14:00 Lunch @ Wolfson Court 14:00 to 14:50 Lars Ruthotto (Emory University)PDE-based Algorithms for Convolution Neural Network This talk presents a new framework for image classification that exploits the relationship between the training of deep Convolution Neural Networks (CNNs) to the problem of optimally controlling a system of nonlinear partial differential equations (PDEs). This new interpretation leads to a variational model for CNNs, which provides new theoretical insight into CNNs and new approaches for designing learning algorithms. We exemplify the myriad benefits of the continuous network in three ways. First, we show how to scale deep CNNs across image resolutions using multigrid methods. Second, we show how to scale the depth of deep CNNS in a shallow-to-deep manner to gradually increase the flexibility of the classifier. Third, we analyze the stability of CNNs and present stable variants that are also reversible (i.e., information can be propagated from input to output layer and vice versa), which in combination allows training arbitrarily deep networks with limited computational resources. This is joint work with Eldad Haber (UBC), Lili Meng (UBC), Bo Chang (UBC), Seong-Hwan Jun (UBC), Elliot Holtham (Xtract Technologies) INI 1 14:50 to 15:40 Gitta Kutyniok (Technische Universität Berlin)Optimal Approximation with Sparsely Connected Deep Neural Networks Despite the outstanding success of deep neural networks in real-world applications, most of the related research is empirically driven and a mathematical foundation is almost completely missing. One central task of a neural network is to approximate a function, which for instance encodes a classification task. In this talk, we will be concerned with the question, how well a function can be approximated by a neural network with sparse connectivity. Using methods from approximation theory and applied harmonic analysis, we will derive a fundamental lower bound on the sparsity of a neural network. By explicitly constructing neural networks based on certain representation systems, so-called $\alpha$-shearlets, we will then demonstrate that this lower bound can in fact be attained. Finally, we present numerical experiments, which surprisingly show that already the standard backpropagation algorithm generates deep neural networks obeying those optimal approximation rates. This is joint work with H. Bölcskei (ETH Zurich), P. Grohs (Uni Vienna), and P. Petersen (TU Berlin). INI 1 15:40 to 16:00 Eva-Maria Brinkmann (Westfalische Wilhelms-Universitat Munster)Enhancing fMRI Reconstruction by Means of the ICBTV-Regularisation Combined with Suitable Subsampling Strategies and Temporal Smoothing Based on the magnetic resonance imaging (MRI) technology, fMRI is a noninvasive functional neuroimaging method, which provides maps of the brain at different time steps, thus depicting brain activity by detecting changes in the blood flow and hence constituting an important tool in brain research. An fMRI screening typically consists of three stages: At first, there is a short low-resolution prescan to ensure the proper positioning of the proband or patient. Secondly, an anatomical high resolution MRI scan is executed and finally the actual fMRI scan is taking place, where a series of data is acquired via fast MRI scans at consecutive time steps thus illustrating the brain activity after a stimulus. In order to achieve an adequate temporal resolution in the fMRI data series, usually only a specific portion of the entire k-space is sampled. Based on the assumption that the full high-resolution MR image and the fast acquired actual fMRI frames share a similar edge set (and hence the sparsity pattern with respect to the gradient), we propose to use the Infimal Convolution of Bregman Distances of the TV functional (ICBTV), first introduced in \cite{Moeller_et_al}, to enhance the quality of the reconstructed fMRI data by using the full high-resolution MRI scan as a prior. Since in fMRI the hemodynamic response is commonly modelled by a smooth function, we moreover discuss the effect of suitable subsampling strategies in combination with temporal regularisation. This is joint work with Julian Rasch, Martin Burger (both WWU Münster) and with Ville Kolehmainen (University of Eastern Finland). [1] {Moeller_et_al} M. Moeller, E.-M. Brinkmann, M. Burger, and T. Seybold: Color Bregman TV. SIAM J. Imag. Sci. 7(4) (2014), pp. 2771-2806. INI 1 16:00 to 16:30 Afternoon Tea 16:30 to 17:20 Joan Bruna (New York University); (University of California, Berkeley)Geometry and Topology of Neural Network Optimization Co-author: Daniel Freeman (UC Berkeley) The loss surface of deep neural networks has recently attracted interest in the optimization and machine learning communities as a prime example of high-dimensional non-convex problem. Some insights were recently gained using spin glass models and mean-field approximations, but at the expense of simplifying the nonlinear nature of the model. In this work, we do not make any such assumption and study conditions on the data distribution and model architecture that prevent the existence of bad local minima. We first take a topological approach and characterize absence of bad local minima by studying the connectedness of the loss surface level sets. Our theoretical work quantifies and formalizes two important facts: (i) the landscape of deep linear networks has a radically different topology from that of deep half-rectified ones, and (ii) that the energy landscape in the non-linear case is fundamentally controlled by the interplay between the smoothness of the data distribution and model over-parametrization. Our main theoretical contribution is to prove that half-rectified single layer networks are asymptotically connected, and we provide explicit bounds that reveal the aforementioned interplay. The conditioning of gradient descent is the next challenge we address. We study this question through the geometry of the level sets, and we introduce an algorithm to efficiently estimate the regularity of such sets on large-scale networks. Our empirical results show that these level sets remain connected throughout all the learning phase, suggesting a near convex behavior, but they become exponentially more curvy as the energy level decays, in accordance to what is observed in practice with very low curvature attractors. Joint work with Daniel Freeman (UC Berkeley). INI 1 17:20 to 18:10 Justin Romberg (Georgia Institute of Technology)Structured solutions to nonlinear systems of equations We consider the question of estimating a solution to a system of equations that involve convex nonlinearities, a problem that is common in machine learning and signal processing. Because of these nonlinearities, conventional estimators based on empirical risk minimization generally involve solving a non-convex optimization program. We propose a method (called "anchored regression”) that is based on convex programming and amounts to maximizing a linear functional (perhaps augmented by a regularizer) over a convex set. The proposed convex program is formulated in the natural space of the problem, and avoids the introduction of auxiliary variables, making it computationally favorable. Working in the native space also provides us with the flexibility to incorporate structural priors (e.g., sparsity) on the solution. For our analysis, we model the equations as being drawn from a fixed set according to a probability law. Our main results provide guarantees on the accuracy of the estimator in terms of the number of equations weare solving, the amount of noise present, a measure of statistical complexity of the random equations, and thegeometry of the regularizer at the true solution. We also provide recipes for constructing the anchor vector (that determines the linear functional to maximize) directly from the observed data. We will discuss applications of this technique to nonlinear problems including phase retrieval, blind deconvolution, and inverting the action of a neural network. This is joint work with Sohail Bahmani. INI 1 18:10 to 19:10 Poster Session & Welcome Wine Reception at INI