Course Code: DA322M	Elements of Deep Learning	Credits: 3-0-0-6
Pre-requisite: None
Syllabus: Introduction: Engineered and learned features, discriminative models, decision surfaces, shallow and deep learning; Feature extraction: Correlation, cross-correlation, auto-correlation, convolution; Revisiting MLP: Multilayer perceptron, back-propagation learning; Activation functions; Loss functions; Optimization techniques: Stochastic gradient descent, batch optimization, momentum optimizer, RMSProp, Adam; Autoencoders; Convolutional Neural Network: Building blocks of CNN, vanishing and exploding gradient problems; Popular CNN architectures: LeNet, AlexNet, VGGNet, ResNet skip connections, inception blocks; Training issues: Early stopping, dropout, batch normalization, instance normalization, group normalization; Recurrent Neural Networks and variants; Applications of Deep Networks.
Textbooks: I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, MIT Press, 2016. M. A. Nielsen, Neural Networks and Deep Learning, Determination Press, 2015.
References: A. Zhang, Z. C. Lipton, M. Li, A. J. Smola, Dive into Deep Learning, 2021. Y. Bengio, Learning Deep Architectures for AI, Now Publishers Inc., 2009.