Preparation for Deep Learning Course

Videos

Switch between videos for different topics that give good intuition.

• Deep Learning for Computer Vision - Highly Recommended
• Math for Deep Learning - Andreas Geiger (MaDL)
• DL Stanford | Spring 2017
• For NLP: Stanford CS224n
• Neural Networks: Zero to Hero by Andrej Karpathy
  
• CS7643 DL lectures
  • OMSCS DL lectures
  • CS7643 GT on-campus | Fall 2021
    • lectures

Syllabus, Papers and Textbooks

• Syllabus: CS7643 Spring 2024 Syllabus
• Paper(s) to gauge your math skill: The Matrix Calculus You Need For Deep Learning
• Papers with Code - A nice read to enhance understanding, use during the course.
  
• Textbooks
  • Deep Learning by Ian Goodfellow
  • Dive into Deep Learning
  • Mathematics for Machine Learning
  • New textbooks:
    • Understanding Deep Learning by Simon J.D. Prince
    • Deep Learning by Chris M. Bishop
      

Math Background & Preparation Advice

1. Derivatives:
   • Basic rules (sum, product, chain), trigonometric functions ((\sin, \cos)), and common functions ((e^x, \text{sigmoid}, \ln)).
   • Multi-variate functions: Partial derivatives with respect to specific variables.
2. Jacobians:
   • Understanding and calculating values within a Jacobian matrix.
3. Composed Functions:
   • Derivatives of composed functions and computation graphs.
4. Practice Resources:
   • Khan Academy and YouTube videos for derivative practice.
   • eg: Math for Deep Learning - Andreas Geiger (MaDL)

Tips

• First Quiz & Assignments: Focus on revising Calculus.
• PyTorch: Used in the 2nd half of the class; you can pick it up as you go.
• Guidance: Following this preparation was sufficient for Quiz 1.