Approximation Theory of Deep Learning for Sequence Modelling

Abstract：

In this talk, we present some recent results on the approximation theory of deep learning architectures for sequence modelling. In particular, we formulate a basic mathematical framework, under which different popular architectures such as recurrent neural networks, dilated convolutional networks (e.g. WaveNet), encoder-decoder structures, and most recently - transformers - can be rigorously compared. These analyses reveal some interesting connections between approximation, memory, sparsity/low-rank, graphical structures that may guide the practical selection and design of these network architectures.

Tencent：

Link: https://meeting.tencent.com/dm/eFKeKFIJuToz

ID：227-124-893

Brief bio:

Qianxiao Li is an assistant professor in the Department of Mathematics, and a principal investigator in the Institute for Functional Intelligent Materials, National University of Singapore. He graduated with a BA in mathematics from the University of Cambridge and a PhD in applied mathematics from Princeton University. His research interests include the interplay of machine learning and dynamical systems, control theory, stochastic optimisation algorithms and data-driven methods for science and engineering.