Abstract: Deep neural networks have spectacular applications but remain mostly a mathematical mystery. An outstanding issue is to understand how they circumvent the curse of dimensionality to generate or classify data. Inspired by the renormalization group in physics, we explain how deep networks can separate phenomena which appear at different scales, and capture scale interactions. It provides high-dimensional model, which approximate the probability distribution of complex physical fields such as turbulences or structured images. For classification, learning becomes similar to a compressed sensing problem, where low-dimensional discriminative structures are identified with random projections. We introduce a multiscale random feature model of deep networks for classification, which is validated numerically.
Speaker: Stéphane Mallat was Professor at NYU in computer science, until 1994, then at Ecole Polytechnique in Paris and Department Chair. From 2001 to 2007 he was co-founder and CEO of a semiconductor start-up company. Since 2017, he holds the “Data Sciences” chair at the Collège de France. He is a member of the French Academy of sciences, of the Academy of Technologies, and a foreign member of the US National Academy of Engineering. Stéphane Mallat’s research interests include machine learning, signal processing and harmonic analysis. He developed the multiresolution wavelet theory and algorithms at the origin of the compression standard JPEG-2000, and sparse signal representations in dictionaries through matching pursuits. He currently works on mathematical models of deep neural networks, for data analysis and physics.