In order to study deep learning, we need to have an understanding of some basics of linear algebra. In particular, deep learning is typically understood and implemented with information stored and manipulated in vectors, matrices, and more generally tensors. We need to understand the basic operations on those objects and be comfortable with the notation typically used.
Topics to Look For¶
- Vectors and Matrices
- Vector addition and scalar multiplication
- Matrix multiplication
- Tensors
Resources¶
- BetterExplained: An Intuitive Guide to Linear Algebra and A Programmer’s Intuition for Matrix Multiplication
- These articles present linear algebra concepts in a more intuitive fashion, and they frame the concepts in a way that is very relevant to our uses of them.
-
Dive Into Deep Learning: Linear Algebra
- After reading the BetterExplained articles, use this for a more formal presentation of the same topics, and pay attention to the terminology and mathematical notation used. This textbook includes PyTorch code examples for most concepts. We’ll be using PyTorch later, and it doesn’t hurt to get the introduction to some of its syntax now, but you don’t need to worry about learning the syntax just yet.
Supplemental¶
[“Supplemental” resources are generally optional, provided to give another source that might help you understand something that wasn’t clear in the main resources.]
- 3Blue1Brown (a source of great, visual explanations of many concepts) covers linear algebra basics with a more visual/geometric approach that works well for a linear algebra course but is less relevant to our uses. It may help augment the above resources, though, so check out the first four videos (plus maybe chapter 8) from the Essence of linear algebra playlist.
- Khan Academy also takes an approach more suitable for a linear algebra course but may help you still. Their Linear algebra course covers most relevant topics (pick and choose by title) and many more.
-
Deep Learning: Linear Algebra, sections 2.0-2.3 (though feel free to read on if you want to see a lot of words that start with “eigen-“)
- Similar to the Dive Into Deep Learning textbook above, but this one is a bit more terse.