Ilya 30u30 Notes

Link to full paper list

Supposedly the papers ilya gave carmack to help him understand the industry, although who knows...

• Brief notes on Complextrophy, Attn is all you need, Effectiveness of RNNs [05.02-26.24] (1-3/30)
• Understanding LSTM Networks [05.29.24] (4/30)
• Deep Residual Learning [06.15.24] (5/30)
• GPipe [06.13.24] (6/30)
• ImageNet with Deep ConvNets [06.10.24] (7/30)
• Neural Machine Translation [06.17.24] (8/30)
• Neural Message Passing for Quantum Chemistry [06.17.24] (9/30)
• Pointer Networks [06.10.24] (10/30)
• Seq2Seq for Sets [06.10.24] (11/30)
• Scaling Laws for Neural LLMs [06.20.24] (12/30)
• Identity Mappings in Deep Residual Networks [06.20.24] (13/30)

Papers Remaining

• The Annotated Transformer
• Recurrent Neural Network Regularization
• Keeping Neural Networks Simple by Minimizing the Description Length of the Weights
• Multi-Scale Context Aggregation by Dilated Convolutions
• A Simple NN Module for Relational Reasoning
• Variational Lossy Autoencoder
• Relational RNNs
• Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton
• Neural Turing Machines
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• A Tutorial Introduction to the Minimum Description Length Principle
• Machine Super Intelligence Dissertation
• PAGE 434 onwards: Komogrov Complexity
• CS231n Convolutional Neural Networks for Visual Recognition

Remaining papers: 17/30