I have taught courses on Deep Learning, Reinforcement Learning and Congestion Control Algorithms at Santa Clara University and at the Indian Institute of Technology, Gandhinagar.

Internet Congestion Control Course

Slides from my lectures for the Congestion Control course. Most of the material is taken from my book “Internet Congestion Control”, though I have added new material based on recent developments in this field in Lectures 7, 9, and 10.

  1. Lecture 1 - Introduction: Objectives of Congestion Control algorithms, Types of Congestion Control Algorithms, TCP Reno, Network Feedback, TCP Vegas.

  2. Lecture 2 - Congestion Control Models Part 1: TCP Throughput Analysis, Effect of Buffer Size, Window Size and Link Error Rate on throughput, Fluid Flow Model for Congestion Control, Throughput analysis for more general models.

  3. Lecture 3 - Congestion Control Models Part 2: Network wide models of Congestion Control using Optimization Theory, Stability Analysis of TCP using Classical Control Theory, The Proportional and Proportional + Integral Controllers, The Averaging Principle.

  4. Lecture 4 - Congestion Control in Broadband Wireless Systems: Split Connection TCP, TCP Westwood, Congestion Control over Lossy Links, Link Level Error Correction, the Bufferbloat Problem, CoDel Active Queue Management.

  5. Lecture 5 - Congestion Control in High Speed Networks Part 1: Design Issues in High Speed Networks, Analysis using Response Functions, RTT Fairness, High Speed TCP (HSTCP), TCP BIC.

  6. Lecture 6 - Congestion Control in High Speed Networks Part 2: TCP Cubic, Compound TCP, Yeah TCP, Compatibility with TCP Reno, Throughput analysis.

  7. Lecture 7 - Congestion Control in High Speed Networks Part 3: TCP BBR (Bottleneck BW and Rate Control), BDP Estimates, RTT Probing, Multiple flows case, Interaction with loss based protocols, XCP (Express Control Protocol), RCP (Rate Control Protocol)

  8. Lecture 8 - Congestion Control in Data Center Networks: Data Center Network Architecture and Traffic Patterns, Data Center TCP (DCTCP), D2TCP, Deadline Aware Congestion Control, Multipath TCP, the Incast Problem.

  9. Lecture 9 - Delay Bounding Congestion Control Algorithms TCP Vegas, TCP FAST, TCP COPA, TIMELY, Bounds on performance of delay bounding algorithms.

  10. Lecture 10 - Optimization based Congestion Algorithms: Global Optimization based algorithms, Remy and Indigo Congestion Control, Local Optimization based algorithms, Performance Oriented Congestion Control (PCC), PCC Allegro, PCC Vivace, Deep Reinforcement Learning based algorithms.

  11. Lecture 11 - Video Delivery over TCP: Video traffic characteristics, Adaptive Streaming, Video Rate Control, The DASH framework, Adaptive Bit Rate (ABR) Algorithms, Rate based and Buffer size based algorithms, ABR/TCP interaction.

Reinforcement Learning Course

Slides from my lectures for the Reinforcement Learning course. Professor Sanjiv Das was kind enough to compile class notes from these lectures, and they can be found here.

  1. Lecture 1 - Introduction: Introduction to Reinforcement Learning and discussion of important applications, An historical overview of the development of this topic.

  2. Lecture 2 - Markov Decision Processes: Markov Processes, Markov Reward Process, Value Function, Markov Decision Processes, Policies, Bellman Expectation Equation, Optimal Value Function, Optimal Policies, Bellman Optimality Equation.

  3. Lecture 3 - Model Based Control: Estimating the Value Function of a known MDP by Dynamic Programming, Policy Evaluation, Policy Iteration, Value Iteration.

  4. Lecture 4 - Model Free Prediction: Estimating the Value Function of an unknown MDP, Monte Carlo (MC) based Policy Evaluation, Temporal Difference (TD) Learning, Comparison of MC and TD Methods.

  5. Lecture 5 - Model Free Control: Optimizing the Value Function of an Unknown MDP, Epsilon Greedy Policies, On Policy Monte Carlo Control, On Policy Temporal Difference Control, SARSA Control, Off Policy Learning, Q-Learning.

  6. Lecture 6 - Deep Reinforcement Learning: Supervised Learning, Function Approximations using Deep Learning, Training Algorithms, Convolutional and Recurrent Neural Networks

  7. Lecture 7 - Deep Reinforcement Learning with Value Function Approximations: Large Scale Reinforcement Learning, Types of Value Function Approximations (VFA), VFA using Deep Learning Networks, Monte Carlo based VFA, Temporal Difference based VFA, Deep Q Networks (DQN), Advanced DQN Algorithms.

  8. Lecture 8 - Policy Gradient Methods: Policy based Reinforcement Learning, Policy Optimization, Policy Gradient, Monte Carlo based Policy Gradient (REINFORCE), Actor- Critic Algorithms.

  9. Lecture 9 - Monte Carlo Tree Search and The AlphaGo Zero Algorithm: Model based Reinforcement Learning, Learning Models from experience, Planning with a Model, Integrated Learning and Planning, Dyna-Q Algorithm, Monte Carlo Tree Search (MCTS) Algorithm, Playing Go using Monte Carlo Tree Search.

  10. Lecture 10 - Reinforcement Learning Based on Human Feedback: Large Language Models, Reward Models, Proximal Policy Optimization (PPO) Algorithm

Deep Learning Course

Here are the slides from my lectures for the Deep Learning course. My book on Deep Learning closely follows the contents of these lectures and can be found here.

  1. Lecture 1 - Introduction: Introduction to Deep Learning and discussion of important applications, Introduction to Types of Deep Learning Systems: Supervised Learning, Reinforcement Learning, Unsupervised Learning, Self-Supervised Learning, An historical overview of Deep Learning

  2. Lecture 2 - Mathematical Preliminaries: An overview of Probability Theory, Bayes Rule, Random Variables, Random Sequences, Markov Chains, Maximum Likelihood Estimation, Basics of Linear Algebra, Matrices, Tensors

  3. Lecture 3 - Linear Networks: The Classification and Regression Problems, Solving these using Supervised Learning, Binary Classification, Linear Models (Logistic Regression), Loss Functions, Introducing Gradient Descent, K-ary Classification, Using Keras to solve linear models

  4. Lecture 4 - Dense Feed Forward Networks: Interpreting the Linear Classifier, Dense Feedforward Networks, The Backprop Algorithm, Forward and Backward Passes, Gradient Flow Algebra, Derivation of the Backprop Algorithm, Dense Feedforward Networks using Keras

  5. Lecture 5 - Backprop: Gradient Flow Calculus, Forward and Backward Passes in Backprop, Derivation of Backprop

  6. Lecture 6 - Keras: Training Process, Some Common Training Datasets: MNIST, CIFAR-10, ILSVRC, IMDB etc, Getting Deeper into Keras, Ingesting Data into Keras Models: Image, Text and Tabular

  7. Lecture 7 - Training Part 1: Vanishing Gradient Problem, Activation and Loss Functions, Techniques to Improve Stochastic Gradient Descent, Illustration of algorithms using Keras, Instructions for doing Term Project

  8. Lecture 8 - Training Part 2: Weight Initialization, Data Pre-Processing, Batch Normalization, Model Under-fitting and Over-fitting problems, Illustration of Algorithms using Keras

  9. Lecture 9 - Training Part 3: Regularization Techniques – L2, L1 and Dropout, Dataset Augmentation, Hyper-Parameter Selection – Manual and Automated Tuning, Model Ensembles, Illustration of Algorithms using Keras

  10. Lecture 10 - Convolutional Networks Part 1: History and Applications of ConvNets, ConvNet Architecture, 2D Convolutions, 1D Convolutions, Sizing ConvNets, Modeling ConvNets with Keras

  11. Lecture 11 - Convolutional Networks Part 2: Pooling and Padding in ConvNets, Trends in ConvNet Design: Small Filters, Global Max Pooling, Depthwise Separable Convolutions, Some Historically significant ConvNet Architectures – LeNet5, AlexNet, ZFNet, VGGNet

  12. Lecture 12 - Convolutional Networks Part 3: ConvNet Architectures (cont): InceptionNet, ResNet. DenseNet, SqueezeNet, Transfer Learning using Keras, Text and Tabular Data Processing using 1D Convolutions

  13. Lecture 13 - Recurrent Networks Part 1: RNN Architectures – One to One, Many to One, Many to Many; Contrasting RNNs with ConvNets, Deep and Bi-Directional RNNs, Combination of RNNs and ConvNets

  14. Lecture 14 - Recurrent Networks Part 2: Difficulties in Training RNNs and how to solve them, Back Propagation through Time (BPTT) Algorithm, LSTMs, GRUs, Word Embeddings and the Word2Vec algorithm, Modeling RNNs with Keras

  15. Lecture 15 - Natural Language Processing Part 1: Application of RNNs to Natural Language Processing, Probabilistic Language Models, Beam Search, Softmax Temperature, Text Classification, Machine Translation, Attention Mechanism in RNNs

  16. Lecture 16 - Natural Language Processing Part 2: Image Captioning, Question Answering Systems, Reading Comprehension, Information Retrieval Systems, Speech Transcription

  17. Lecture 17 - Transformers Part 1: Self Attention, Transformer Architecture, Multiple Attention Heads, Encoding Position Information, Visualizing Attention Patterns

  18. Lecture 18 - Transformers Part 2: Language Models using Transformers, Encoders Decoders Using Transformers, BERT Bi-Directional Language Models, Image Processing Using Transformers

  19. Lecture 19 - Diffusion Models: Latent Variables, ELBO Bound, Forward and Backward Diffusion Processes, DDPM and DDIM Algorithms

  20. Lecture 20 - Reinforcement Learning Introduction, Components of a RL System: Agents, Rewards, Actions, Deep RL, Playing Pong with Policy Gradients, Playing Go with Supervised Learning and Policy Gradients, Imitation Learning

Exercises

I put together a list of problems in the Deep Learning area that I use for this course, it can be found here