We simply explain and illustrate Mamba, State Space Models (SSMs) and Selective SSMs.
SSMs match performance of transformers, but are faster and more memory-efficient than them. This is crucial for long sequences!

AI Coffee Break Merch! 🛍️ https://aicoffeebreak.creator-spring.... Celebrating our merch launch, here is a limited time offer! 👉 Get 25% discount on AI Coffee Break Merch with the code MAMBABEAN.

Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏
Dres. Trost GbR, Siltax, Vignesh Valliappan, Michael

Outline:
00:00 Mamba to replace Transformers!?
02:04 State Space Models (SSMs) – high level
03:09 State Space Models (SSMs) – more detail
05:45 Discretization step in SSMs
08:14 SSMs are fast! Here is why.
09:55 SSM training: Convolution trick
12:01 Selective SSMs
15:44 MAMBA Architecture
17:57 Mamba results
20:15 Building on Mamba
21:00 Do RNNs have a comeback?
21:42 AICoffeeBreak Merch

📄 Gu, Albert, and Tri Dao. "Mamba: Linear-time sequence modeling with selective state spaces." arXiv preprint arXiv:2312.00752 (2023). https://arxiv.org/abs/2312.00752
📄 MoE-Mamba https://arxiv.org/abs/2401.04081
📄 Vision Mamba https://arxiv.org/abs/2401.09417
📄 MambaByte https://arxiv.org/abs/2401.13660
🕊️ Mamba rejected from ICLR:   / 1750526956452577486  
📖 Prefix sum (Scan) with Cuda: https://developer.nvidia.com/gpugems/...
📺 Transformer explained:    • The Transformer explained by Ms. Coff...  

Great resources to learn about Mamba:
📙 Mamba: https://jameschen.io/jekyll/update/20...
📕 The Annotated S4: https://srush.github.io/annotated-s4/
📘 Mamba The Easy Way: https://jackcook.com/2024/02/23/mamba...

▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕
Patreon:   / aicoffeebreak  
Ko-fi: https://ko-fi.com/aicoffeebreak
Join this channel to get access to perks:
   / @aicoffeebreak  
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

🔗 Links:
AICoffeeBreakQuiz:    / aicoffeebreak  
Twitter:   / aicoffeebreak  
Reddit:   / aicoffeebreak  
YouTube:    / aicoffeebreak  

#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research​

Scientific advising by Mara Popescu
Video editing: Nils Trost
Music 🎵 : Sunny Days – Anno Domini Beats