Course Description

Deep learning, which is considered as a subset of machine learning, is a core technology that has revolutionized entire Artificial Intelligence (AI) fields. It has achieved remarkable success in various applications, such as computer vision, natural language processing, speech recognition, robotics, biology, medicine, to name a few. This course will start from presenting the basic theory and practice of deep learning, and introduce rapidly changing techniques. This course also focuses on practical aspects of deep learning, which requires students to perform final course projects to gain a deeper knowledge through active exploration of various problems.

Schedule (Tentative)

Date	Topic	Course Materials / Extra Reading
Week 1	Introduction / Motivation / Course Logistics Linear Models / Multi-Layer Perceptron (MLP) [Slides]	[LeCun et al., Nature '15] CS229 Notes 1
Week 2	Linear Models / Multi-Layer Perceptron (MLP) (continued)
Week 3	Backpropagation / Automatic Differentiation [Slides]	Automatic differentiaion in machine learning: a survey CS231n Backpropagation
Week 4	Optimization / Gradient Descent [Slides]
Week 5	Convolutional Neural Networks 1 [Slides]	CS231n CNN CNN Explainer
Week 6	Convolutional Neural Networks 2 [Slides]
Week 7	Convolutional Neural Networks 2 (continued)
Week 8	CNN Applications (Object Detection and Segmentation)	CS231n Detection and Segmentation
Week 9	Project Proposals Recurrent Neural Networks [Slides]
Week 10	Transformers [Slides]	The Illustrated Transformer Attention is All You Need BERT GPT3
Week 11	Graph Neural Networks	CS224W Machine Learning with Graphs GNN1 GNN2 GNN Application Graph Convolutional Networks

Project

The class project is for you to gain in-depth knowledge of deep learning by making your hands dirty. You will be a part of 2~3 person group to conduct a project during the semester. You can choose whatever topics that excite you and the only restriction is that you should use deep learning to tackle the problems.

Grading

Class attendance and participation - [10%]
Homework - [20%]
Final project - [30%]
Final exam - [40%]

Prerequisites

Ideally, previous exposure to following courses make you get most ouf of this course.

• Introduction to machine learning (ICE 3045 or equivalent)
• Calculus and Linear Algebra (GEDB001, GEDB002, GEDB003 or equivalents)
• Probatility (ICE 2003 or equivalent)
• Programming (GEDT019, EEE2017 or equivalents, python language preferred)

Related Courses and Materials

• CS231n Convolutional Neural Networks for Visual Recognition, Stanford
• CS224n: Natural Language Processing with Deep Learning, Stanford
• Dive into Deep Learning
• DeepLearning.AI