Deep Learning Architectures & Systems

Let's continue with our survey of developments since GPT-3.

Useful starting points:

* [A Visual Tour of LLM of Modern LLM Architectures](https://www.youtube.com/watch?v=CepbWmGie0E) (youtube 38min)

* [A slop summary of Reiner Pope's interview ](https://solmaz.io/dwarkesh-reiner-pope-llm-training-and-serving)on how LL

Ms are actually trained and served. (Has some useful diagrams)

* [The Deepseek family of models](https://www.bentoml.com/blog/the-complete-guide-to-deepseek-models-from-v3-to-r1-and-beyond) are the most well documented of the lot for the last two years.

Informal, all levels welcome. Suggestions for future papers and topics welcome.