Improving learning efficiency for deep neural networks

Speaker: Razvan Pascanu, DeepMind.

Abstract: In this talk I will look at learning efficiency for deep learning and attempt to motivate its importance and its place as one of the crucial open problems in the field. In particular I will focus on a series of works that attempt to look at this problem from different angles, from compression to the role of inductive biases and learning dynamics imposed by gradient based learning. These works will highlight how the problem can be looked at from this perspective showing the extent of the problem and hopefully encourage more research on understanding the causes of inefficiency in learning, and looking at solutions for it.

Presented at IEEE International Workshop on Machine Learning for Signal Processing, Thursday, September 24, 2020.