Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 14 – Transformers and Self-Attention
Stanford Online Stanford Online
571K subscribers
140,797 views
0

 Published On Mar 21, 2019

For more information about Stanford’s Artificial Intelligence professional and graduate programs, visit: https://stanford.io/3niIw41

Professor Christopher Manning, Stanford University, Ashish Vaswani & Anna Huang, Google
http://onlinehub.stanford.edu/

Professor Christopher Manning
Thomas M. Siebel Professor in Machine Learning, Professor of Linguistics and of Computer Science
Director, Stanford Artificial Intelligence Laboratory (SAIL)

To follow along with the course schedule and syllabus, visit: http://web.stanford.edu/class/cs224n/...

0:00 Introduction
2:07 Learning Representations of Variable Length Data
2:28 Recurrent Neural Networks
4:51 Convolutional Neural Networks?
14:06 Attention is Cheap!
16:05 Attention head: Who
16:26 Attention head: Did What?
16:35 Multihead Attention
17:34 Machine Translation: WMT-2014 BLEU
19:07 Frameworks
19:31 Importance of Residuals
23:26 Non-local Means
26:18 Image Transformer Layer
30:56 Raw representations in music and language
37:52 Attention: a weighted average
40:08 Closer look at relative attention
42:41 A Jazz sample from Music Transformer
44:42 Convolutions and Translational Equivariance
45:12 Relative positions Translational Equivariance
50:21 Sequential generation breaks modes.
50:32 Active Research Area

#naturallanguageprocessing #deeplearning

show more

Share/Embed