In this talk, we offer an entirely “white box’’ interpretation of deep (convolution) networks from the perspective of data compression (and group invariance). In particular, we show how modern deep layered architectures, linear (convolution) operators and nonlinear activations, and even all parameters can be derived from the principle of maximizing rate reduction (with group invariance). All layers, operators, and parameters of the network are explicitly constructed via forward propagation, instead of learned via back propagation. All components of so-obtained network, called ReduNet, have precise optimization, geometric, and statistical interpretation. There are also several nice surprises from this principled approach: it reveals a fundamental tradeoff between invariance and sparsity for class separability; it reveals a fundamental connection between deep networks and Fourier transform for group invariance – the computational advantage in the spectral domain (why spiking neurons?); this approach also clarifies the mathematical role of forward propagation (optimization) and backward propagation (variation). In particular, the so-obtained ReduNet is amenable to fine-tuning via both forward and backward (stochastic) propagation, both for optimizing the same objective. This is joint work with students Yaodong Yu, Ryan Chan, Haozhi Qi of Berkeley, Dr. Chong You now at Google Research, and Professor John Wright of Columbia University. Registration：https://harvard.zoom.us/webinar/register/3516062319845/WN_9Qb6IBIfQ0ujujfLvue2eA

Yi Ma received his B.S. degree in Automation and Applied Mathematics from Tsinghua University, China in 1995, an M.S. degree in EECS in 1997, an M.A. degree in Mathematics in 2000, and a Ph.D. in EECS in 2000 all from UC Berkeley. He was on the faculty of ECE Department of the University of Illinois at Urbana-Champaign from 2000 to 2011. He was the manager of the Visual Computing Group and a principal researcher of Microsoft Research in Asia from 2009 to 2013. He was then a founding professor and the executive dean of the School of Information Science and Technology of ShanghaiTech University from 2014 to 2017. He joins the faculty of EECS of UC Berkeley in 2018.