2023/05/18: Attetion, Google AI Infrastructure

1. How to Estimate the Number of Parameters in Transformer models

source: https://towardsdatascience.com/how-to-estimate-the-number-of-parameters-in-transformer-models-ca0f57d8dff0

2. An Intuition for Attention

source: https://jaykmody.com/blog/attention-intuition/

3. Google AI Infrastructure Supremacy: Systems Matter More Than Microarchitecture

source: https://www.semianalysis.com/p/google-ai-infrastructure-supremacy