Preparation for Deep Learning Course

Videos

Switch between videos for different topics that give good intuition.

• Deep Learning for Computer Vision
• Math for Deep Learning - Andreas Geiger (MaDL)
• DL Stanford | Spring 2017
• For NLP: Stanford CS224n
• Neural Networks: Zero to Hero by Andrej Karpathy\
  

CS7643 DL lectures

• OMSCS DL lectures
• CS7643 GT on-campus | Fall 2021
  • lectures

Syllabus and other text

• CS7643 Spring 2024 Syllabus
• Paper(s) to gauge your math skill:
  • The Matrix Calculus You Need For Deep Learning
• Papers with Code - A nice read\
  

Textbooks

• Deep Learning by Ian Goodfellow
• Dive into Deep Learning
• Mathematics for Machine Learning
• New textbooks:
  • Understanding Deep Learning by Simon J.D. Prince
  • Deep Learning by Chris M. Bishop\
    

Math Background from a previous student

1. mostly derivatives; all the rules
2. multiplicative, sum, and those rules, working with cos/sin etc derivatives
3. the maths will help with programming assignment also, so for HW1 which im going to start
4. You will be asked to take partial derivatives of multi-variate functions with respect to a particular variable. You should know the derivatives of common functions (e^x, sigmoid, sin, cos, ln, etc.)
5. Jacobians and calculating particular values within a Jacobian
6. Derivatives of composed functions
7. Computing partial derivatives through a computation graph
8. This was the guidance that was given for quiz 1, i followed exactly and it was sufficient
9. i used khan academy to practice derivatives
10. and just random yotuube videos mostly
11. moving forward i will let you know how much more calculus is needed
12. to watch: Math for Deep Learning - Andreas Geiger\
    

Pinned message on Slack

My number 1 advice for preparing is to get familiar with the necessary Calculus. The first exam and assignment require knowing this. Some resources:

• The Matrix Calculus You Need For Deep Learning (if you just read this one paper, you should be fine)
• YouTube: The Matrix Calculus You Need For Deep Learning- Part 1
• YouTube: The Matrix Calculus You Need For Deep Learning- Part 2

A few other good ones recommended by students in the past:

1. Matrix calculus summary
2. Matrix Differentiation
3. Dive into Deep Learning: Appendix: Mathematics for Deep Learning
4. The Matrix Cookbook

If you have time, you can review Linear Algebra and Probability from the DL textbook:

• LA
• Probability

Also, we use Pytorch for the 2nd half of the class, but I was able to pick it up once we got there and don’t necessarily feel you need to learn it ahead of time.Other than that, have fun, it’s a great class!