Theory of deep learning

Workshop theme

This workshop will focus on the mathematical foundations of deep learning methodology, including approximation, estimation, optimization and robustness properties of these methods. Key questions include: the performance of estimators that eschew established statistical wisdom by fitting noisy training data exactly; the implicit regularization imposed by optimization algorithms; the impact of network features such as depth, nonlinearities, and parameterization symmetries on robustness and approximation properties; and the performance of gradient algorithms in the loss landscapes that arise in deep learning.

Due to the ongoing COVID-19 pandemic, all non-UK based participants should attend this event virtually.

The application is closed

Fees

Registration only:

Registration Package: £TBC
Student Registration Package: £TBC

The Registration Package includes admission to all seminars, lunches and refreshments on the days that lectures take place (Monday - Friday), wine reception and formal dinner, but does not include other meals or accommodation.

Formal Dinner Only

Formal Dinner: £50

Participants on the Accommodation Package or Registration Package, including organisers and speakers, are automatically included in this event. For all remaining participants who would like to attend, such as programme participants, the above charge will apply.

Accommodation

Unfortunately we do not have any accommodation to offer so all successful applicants will need to source their own accommodation.

Please see the Hotels Combined website for a list of local hotels and guesthouses.

Meals

Lunch Lunch timings and location will be confirmed with timetable.

Evening Meal Participants are free to make their own arrangements for dinner.

Formal Dinner The Formal Dinner location and date is to be confirmed.

Participants on the Accommodation Package or Registration Package, including organisers and speakers, are automatically included in this event.

Timetable

Monday 9th August 2021
12:30 to 13:30	Lunch	No Room Required
13:30 to 14:20	Registration (Physical Participants)	No Room Required
14:20 to 14:30	David Abrahams Isaac Newton Institute Chair: Gitta Kutyniok Ludwig Welcome Talk	Room 1
14:30 to 15:00	Emmanuel Abbe EPFL - Ecole Polytechnique Fédérale de Lausanne; Princeton University Chair: Peter Bartlett Fundamental Limits of Gradient-Based Learning on Neural Networks	Room 1
15:00 to 15:30	Dmitry Yarotsky Skolkovo Institute of Science and Technology (Skoltech) Chair: Peter Bartlett Universal Scaling Laws in the Gradient Descent Training of Neural Networks	Room 1
15:30 to 16:00	Afternoon Tea	No Room Required
16:00 to 16:30	Suriya Gunasekar Microsoft (USA) Chair: Peter Bartlett Function space view of Multi-Channel Linear Convolutional Networks with Bounded Weight Norm	Room 1
16:30 to 17:00	Lénaïc Chizat Université Paris Saclay Chair: Peter Bartlett Analysis of Gradient Descent on Wide Two-Layer ReLU Neural Networks	Room 1
17:00 to 18:00	Welcome Wine Reception at INI	No Room Required

Tuesday 10th August 2021
10:30 to 11:00	Rama Cont University of Oxford Chair: Lars Ruthotto Asymptotic analysis of deep residual networks	Room 1
11:00 to 11:30	Morning Coffee	No Room Required
11:30 to 12:00	Yuguang Wang Shanghai Jiao Tong University Chair: Lars Ruthotto How Framelets Enhance Graph Neural Networks	Room 1
12:00 to 12:30	Quanxiao Li National University of Singapore Chair: Lars Ruthotto Approximation theory for machine learning and dynamical systems	Room 1
12:30 to 13:30	Lunch / Poster Session	No Room Required
13:30 to 14:30	Poster Session	Room 1
14:30 to 15:00	Daniel Hsu Columbia University Chair: Lars Ruthotto On the Approximation Power of Two-Layer Networks of Random ReLUs	Room 1
15:00 to 15:30	Aarti Singh Carnegie Mellon University Chair: Lars Ruthotto Local Signal Adaptivity: Provable Feature Learning in Neural Networks Beyond Kernels	Room 1
15:30 to 16:00	Afternoon Tea	No Room Required
16:00 to 16:30	Misha Belkin University of California, San Diego Chair: Emmanuel Abbe Do loss functions matter?	Room 1
16:30 to 17:00	Rachel Ward University of Texas at Austin Chair: Emmanuel Abbe Generalization bounds for sparse random feature expansions	Room 1

Wednesday 11th August 2021
10:30 to 11:00	Joan Bruna New York University; University of California, Berkeley Chair: Gitta Kutyniok On the Sample Complexity of Learning Under Invariance and Geometric Stability	Room 1
11:00 to 11:15	Group Photo	No Room Required
11:15 to 11:30	Morning Coffee	No Room Required
11:30 to 12:00	Sophie Langer Technische Universität Darmstadt Chair: Gitta Kutyniok Is sparsity Really the Answer?	Room 1
12:00 to 12:30	Florent Krzakala EPFL - Ecole Polytechnique Fédérale de Lausanne Chair: Gitta Kutyniok Generalization in machine learning: lessons from simple high-dimensional models	Room 1
12:30 to 13:30	Lunch	No Room Required
13:30 to 15:30	Free Afternoon	No Room Required
15:30 to 16:00	Afternoon Tea	No Room Required
16:00 to 16:30	Nathan Srebro Toyota Technological Institute; University of Chicago Chair: Gitta Kutyniok What can we learn with Gradient Descent?	Room 1
16:30 to 17:00	Sebastien Bubeck Microsoft (USA) Chair: Gitta Kutyniok A universal law of robustness via isoperimetry	Room 1
19:30 to 22:00	Formal Dinner	No Room Required

Thursday 12th August 2021
12:30 to 14:00	Lunch	No Room Required
14:00 to 14:30	Lenka Zdeborová EPFL - Ecole Polytechnique Fédérale de Lausanne Chair: Carola-Bibiane Schönlieb Insights on neural networks from statistical physics solvable models	Room 1
14:30 to 15:00	Johannes Schmidt-hieber Universiteit Leiden; Universiteit Twente Chair: Carola-Bibiane Schönlieb Convergence rates of deep ReLU networks for multiclass classification	Room 1
15:00 to 15:30	Sara Anna van de Geer ETH Zürich Chair: Carola-Bibiane Schönlieb Interpolation in Regression and Classification	Room 1
15:30 to 16:00	Afternoon Tea	No Room Required
16:00 to 16:30	Ioannis Kevrekidis Princeton University Chair: Peter Bartlett Learning Dynamical Systems From Data: Things We Knew In The 1990s And Things We Learn Now	Room 1
16:30 to 17:00	Andrea Montanari Stanford University Chair: Peter Bartlett Minimum complexity interpolation in random features models	Room 1

Friday 13th August 2021
12:30 to 14:00	Lunch	No Room Required
14:00 to 14:30	Matthias Ehrhardt University of Bath Chair: Johannes Schmidt-hieber Structure preserving deep learning	Room 1
14:30 to 15:00	Franca Hoffmann Rheinische Friedrich-Wilhelms-Universität Bonn Chair: Johannes Schmidt-hieber Using Continuum Limits To Understand Data Clustering And Classification	Room 1

Organisers

Francis Bach INRIA Paris - Rocquencourt; ENS - Paris
Peter Bartlett University of California, Berkeley
Gitta Kutyniok Technische Universität Berlin
Lars Ruthotto Emory University

Speakers

Emmanuel Abbe EPFL - Ecole Polytechnique Fédérale de Lausanne; Princeton University
David Abrahams Isaac Newton Institute
Misha Belkin University of California, San Diego
Joan Bruna New York University; University of California, Berkeley
Sebastien Bubeck Microsoft (USA)
Lénaïc Chizat Université Paris Saclay
Rama Cont University of Oxford
Matthias Ehrhardt University of Bath
Suriya Gunasekar Microsoft (USA)
Franca Hoffmann Rheinische Friedrich-Wilhelms-Universität Bonn
Daniel Hsu Columbia University
Ioannis Kevrekidis Princeton University
Florent Krzakala EPFL - Ecole Polytechnique Fédérale de Lausanne
Sophie Langer Technische Universität Darmstadt
Quanxiao Li National University of Singapore
Andrea Montanari Stanford University
Johannes Schmidt-hieber Universiteit Leiden; Universiteit Twente
Aarti Singh Carnegie Mellon University
Nathan Srebro Toyota Technological Institute; University of Chicago
Sara Anna van de Geer ETH Zürich
Yuguang Wang Shanghai Jiao Tong University
Rachel Ward University of Texas at Austin
Dmitry Yarotsky Skolkovo Institute of Science and Technology (Skoltech)
Lenka Zdeborová EPFL - Ecole Polytechnique Fédérale de Lausanne

Show All

Participants

Emmanuel Abbe EPFL - Ecole Polytechnique Fédérale de Lausanne; Princeton University
Ben Adcock Simon Fraser University
Omer Deniz Akyildiz The Alan Turing Institute; University of Cambridge
Md Ferdous Alam Ohio State University
Kossi Amouzouvi Kwame Nkrumah University of Science and Technology (KNUST); African Institute for Mathematical Sciences (AIMS)
Héctor Andrade Loarca Ludwig-Maximilians-Universität München
Marco Armenta Université de Sherbrooke
Simon Arridge University College London
Samuel Asante Gyamerah Kwame Nkrumah University of Science and Technology (KNUST)
Angelica I. Aviles-Rivero University of Cambridge
Célia Ayad École Polytechnique
Bubacarr Bah African Institute for Mathematical Sciences (AIMS); Stellenbosch University
Noga Bar Tel Aviv University
Peter Bartlett University of California, Berkeley
Misha Belkin University of California, San Diego
Martin Benning Queen Mary University of London
Julius Berner University of Vienna
Marta Betcke University College London
Gastao Bettencourt Universidade da Beira Interior
Katharina Bieker Universität Paderborn
Joan Bruna New York University; University of California, Berkeley
Tatiana Alessandra Bubba University of Cambridge
Sebastien Bubeck Microsoft (USA)
Christopher Budd University of Bath
Martin Burger Universität Erlangen-Nürnberg
Christian Cabrera University of Cambridge
Robert Calderbank Duke University
mehmet Caner North Carolina State University
Marcello Carioni Cambridge Centre for Analysis
Elena Celledoni Norwegian University of Science and Technology
kk Chahal None / Other
Lénaïc Chizat Université Paris Saclay
Nadav Cohen Tel Aviv University
Elisabetta Cornacchia EPFL - Ecole Polytechnique Fédérale de Lausanne
Danish Danish ali None / Other
Ingrid Daubechies Duke University
Praveen Kumar Dhankar Rashtrasant Tukadoji Maharaj Nagpur University
Sören Dittmer University of Cambridge; University of Bremen
Edgar Dobriban University of Pennsylvania
Bianca Dumitrascu Cambridge Computer Laboratory
Weinan E Princeton University
Matthias Ehrhardt Bergische Universität Wuppertal
Matthias Ehrhardt University of Bath
Dennis Elbrächter University of Vienna
Yonina Eldar Weizmann Institute of Science
Bryn Elesedy University of Oxford
Christian Etmann University of Cambridge
Asja Fischer Ruhr-Universität Bochum
Andrew Fitzgibbon Microsoft (UK)
Nayara Fonseca Abdus Salam International Centre for Theoretical Physics
Matt Foster University College London
Marianne Freiberger University of Cambridge
Yaé Ulrich Gaba African Institute for Mathematical Sciences (AIMS)
Nicolas Garcia Trillos University of Wisconsin-Madison
Luca Eva Gazdag University of Oslo
Silvia Gazzola University of Bath
Cedric Gerbelot CNRS - Ecole Normale Superieure Paris
Jan Gerken Chalmers University of Technology
Michael Giegrich University of Oxford
Julian Gilbey University of Cambridge
Mark Girolami University of Cambridge; The Alan Turing Institute
Alex Goessmann Technische Universität Berlin
Tamara Grossmann University of Cambridge
Steffen Grunewalder Lancaster University - Mathematics and Statistic Dept.
Suriya Gunasekar Microsoft (USA)
Rambabu Gupta Munich University of Technology
Armando W. Gutiérrez INRIA Saclay - Île-de-France; Ecole Polytechnique Paris
Anders Hansen University of Cambridge
Matthias Hein Eberhard Karls Universität Tübingen
Michael Hintermüller Weierstraß-Institut für Angewandte Analysis und Stochastik; Humboldt-Universität zu Berlin
Franca Hoffmann Rheinische Friedrich-Wilhelms-Universität Bonn
Conor Houghton University of Bristol
Daniel Hsu Columbia University
Sian Jenkins None / Other
Bangti Jin University College London
Amin Karbasi Yale University
George Karniadakis Brown University
Kenji Kawaguchi National University of Singapore; Harvard University
Rihuan Ke University of Cambridge
Ioannis Kevrekidis Princeton University
Sammy Khalife Ecole Polytechnique Paris
Stefan Kolek Martinez de Azagra Ludwig-Maximilians-Universität München
Ralf Korn Technische Universität Kaiserslautern
Lisa Maria Kreusser University of Cambridge; Bath Institute for Mathematical Innovation
Florent Krzakala EPFL - Ecole Polytechnique Fédérale de Lausanne
Gitta Kutyniok Ludwig-Maximilians-Universität München
Sophie Langer Technische Universität Darmstadt
Jonas Latz University of Cambridge
Jason Lee University of Southern California
Yunseok Lee Ludwig-Maximilians-Universität München
Ron Levie Technische Universität Berlin
Zhu Li University College London
Quanxiao Li National University of Singapore
Diogo Lobo Universidade de Coimbra
Daniel Lozano-rojas University of Leicester
Johannes Lutzeyer Ecole Polytechnique Paris
Tengyu Ma Stanford University
Yi Ma University of California, Berkeley
Sohir Maskey Ludwig-Maximilians-Universität München
Sofya Maslovskaya Universität Paderborn
Gonzalo Mena University of Oxford
Hrushikesh Mhaskar Claremont Graduate University
Andrea Montanari Stanford University
Guido Montufar Max-Planck-Institut für Mathematik, Leipzig; University of California, Los Angeles
Alexander Mozeika King's College London
Subhadip Mukherjee University of Cambridge
Davide Murari NTNU
Nicole Mücke Technische Universität Berlin
Johannes Müller Max-Planck-Institut für Mathematik, Leipzig
Sebastian Neumayer Technische Universität Berlin
Richard Nickl University of Cambridge
Shilei Niu Xi'an Jiaotong- Liverpool University
Levon Nurbekyan University of California, Los Angeles
Adam Oberman McGill University
Christian Offen Universität Paderborn
Andrius Ovsianas University of Cambridge
Brynjulf Owren Norwegian University of Science and Technology
Philipp Petersen Universität Wien
Gabriel Peyre CNRS - Ecole Normale Superieure Paris
Giovanni Piccioli EPFL - Ecole Polytechnique Fédérale de Lausanne
Richard Pinch Institute of Mathematics and its Applications; Heilbronn Institute
Sebastian Pokutta Technische Universität Berlin; Konrad-Zuse-Zentrum für Informationstechnik Berlin
Clarice Poon University of Bath
Mariya Ptashnyk Heriot-Watt University
Arun Raja Institute for Infocomm Research, A*STAR Singapore
Maria Refinetti CNRS - Ecole Normale Superieure Paris
Christoph Reisinger University of Oxford
Renata Retkute University of Cambridge
Alejandro Ribeiro University of Pennsylvania
Konstantin Riedl Technische Universität München
Lars Ruthotto Emory University
Malena Sabaté Landman Cambridge Mathematics of Information in Healthcare
Diyora Salimova ETH Zürich
Wojciech Samek Technische Universität Berlin
Guillermo Sapiro Duke University
Johannes Schmidt-hieber Universiteit Leiden; Universiteit Twente
Philipp Scholl Ludwig-Maximilians-Universität München
Christoph Schwab ETH Zürich
Carola-Bibiane Schönlieb University of Cambridge; Cambridge Mathematics of Information in Healthcare
Hanie Sedghi Google Brain
Mariia Seleznova Technische Universität Berlin
Ohad Shamir Weizmann Institute of Science
Zouwei Shen National University of Singapore
Ferdia Sherry University of Cambridge
Chirag Varun Shukla Ludwig-Maximilians-Universität München
Aarti Singh Carnegie Mellon University
Konstantin Siroki University of Manchester
Konstantinos Spiliopoulos Boston University
Nathan Srebro Toyota Technological Institute; University of Chicago
Jan Stanczuk University of Cambridge
Ingo Steinwart Universität Stuttgart
Andrew Stuart CALTECH (California Institute of Technology)
Jeremias Sulam Johns Hopkins University
Yusuf Sulehman University of Manchester
Mahsa Taheri Ruhr-Universität Bochum
Jared Tanner University of Oxford
Rachel Thomas University of Cambridge
Matthew Thorpe University of Manchester
Tom Tirer Tel Aviv University
Hanna Tseran Max-Planck-Institut für Mathematik, Leipzig
Bart Van bloemen waanders Sandia National Laboratories
Sara Anna van de Geer ETH Zürich
Mihaela van der Schaar University of Cambridge
Eric Vanden-Eijnden Courant Institute of Mathematical Sciences; New York University
Maksim Velikanov Skolkovo Institute of Science and Technology (Skoltech)
Rene Vidal Johns Hopkins University; Amazon
Tiffany Vlaar University of Edinburgh
Yuguang Wang Shanghai Jiao Tong University
Hanchen Wang University of Cambridge
Tizian Wenzel Universität Stuttgart
Rebecca Willett University of Chicago
Guoqiang Wu Tsinghua University
Changmin Wu Ecole Polytechnique Paris
Jinghao Xue University College London
Dmitry Yarotsky Skolkovo Institute of Science and Technology (Skoltech)
Jong Chul Ye Korea Advanced Institute of Science and Technology (KAIST)
Haiyan Yu Chongqing University
Lenka Zdeborová EPFL - Ecole Polytechnique Fédérale de Lausanne
Ding-Xuan Zhou City University of Hong Kong
Ozan Öktem KTH - Royal Institute of Technology; Heriot-Watt University