TinyML and Efficient Deep Learning Computing
This course focuses on efficient machine learning and systems. This is a crucial area as deep neural networks demand extraordinary levels of computation, hindering its deployment on everyday devices and burdening the cloud infrastructure. This course introduces efficient AI computing techniques that enable powerful deep learning applications on resource-constrained devices. Topics include model compression, pruning, quantization, neural architecture search, distributed training, data/model parallelism, gradient compression, and on-device fine-tuning. It also introduces application-specific acceleration techniques for large language models and diffusion models. Students will get hands-on experience implementing model compression techniques and deploying large language models (Llama2-7B) on a laptop.
- Lecture Videos:https://live.efficientml.ai/
- Time:
Tuesday/Thursday 3:35-5:00pm Eastern Time
- Location:34-101
- Office Hour:
Thursday 5:00-6:00 pm Eastern Time, 38-344 Meeting Room
- Discussion:Piazza
- Homework Submission:Canvas
- Contact:
- For external inquiries, personal matters, or emergencies, you can email us at efficientml-staff [at] mit.edu.
- If you are interested in getting updates, please sign up here to join our mailing list to get notified!
- Prerequisites: 6.191 Computation Structures and 6.390 Intro to Machine Learning. Students who don't full-fill the prerequisites will be de-registered in the second week of class. If you believe you have equivalent prior experience (e.g., a computer architecture course taken during your undergraduate studies at another institution), you may petition for consideration. Please submit your Petition Form by Sept. 6, 2024, 11:59:59 PM EST.
Instructor
Teaching Assistants
Announcements
- 2024-08-30
The TinyML and Efficient Deep Learning Computing course will be returning in Fall, with recorded sessions on YouTube!
Schedule
Date
Lecture
Logistics
Introduction
Sep 5
Introduction
Basics of Deep Learning
Sep 10
Basics of Deep Learning
Lab 0 out
Chapter I: Efficient Inference
Sep 11
Chapter I: Efficient Inference
Pruning and Sparsity (Part I)
Sep 12
Pruning and Sparsity (Part I)
Pruning and Sparsity (Part II)
Sep 17
Pruning and Sparsity (Part II)
Lab 1 out
Quantization (Part I)
Sep 19
Quantization (Part I)
Lab 0 due
Quantization (Part II)
Sep 24
Quantization (Part II)
Neural Architecture Search (Part I)
Sep 26
Neural Architecture Search (Part I)
Lab 1 due, Lab 2 out
Neural Architecture Search (Part II)
Oct 1
Neural Architecture Search (Part II)
Knowledge Distillation
Oct 3
Knowledge Distillation
MCUNet: TinyML on Microcontrollers
Oct 8
MCUNet: TinyML on Microcontrollers
Lab 2 due, Lab 3 out
TinyEngine and Parallel Processing
Oct 10
TinyEngine and Parallel Processing
Student Holiday — No Class
Oct 15
Student Holiday — No Class
Chapter II: Domain-Specific Optimization
Oct 16
Chapter II: Domain-Specific Optimization
Transformer and LLM
Oct 17
Transformer and LLM
Efficient LLM Deployment
Oct 22
Efficient LLM Deployment
Lab 3 due, Lab 4 out
LLM Post Training
Oct 24
LLM Post Training
Project ideas out (on Canvas)
Long Context LLM
Oct 29
Long Context LLM
Vision Transformer
Oct 31
Vision Transformer
Lab 4 due, Lab 5 out
GAN, Video, and Point Cloud
Nov 5
GAN, Video, and Point Cloud
Diffusion Model
Nov 7
Diffusion Model
Chapter III: Efficient Training
Nov 11
Chapter III: Efficient Training
Distributed Training (Part I)
Nov 12
Distributed Training (Part I)
Lab 5 due
Distributed Training (Part II)
Nov 14
Distributed Training (Part II)
Project proposal due
On-Device Training and Transfer Learning
Nov 19
On-Device Training and Transfer Learning
Chapter IV: Advanced Topics
Nov 20
Chapter IV: Advanced Topics
Basics of Quantum Computing
Nov 21
Basics of Quantum Computing
Quantum Machine Learning
Nov 26
Quantum Machine Learning
Thanksgiving — No Class
Nov 28
Thanksgiving — No Class
Final Project Presentation
Dec 3
Final Project Presentation
Final Project Presentation
Dec 5
Final Project Presentation
Final Project Presentation + Course Summary
Dec 10
Final Project Presentation + Course Summary
Dec 14: Project report and course evaluation due
Course Videos
Logistics
Grading
The class requirements include five labs, and one final project. This is a PhD level course, and by the end of this class you should have a good understanding of efficient deep learning techniques, and be able to deploy large language models (LLMs) on your laptop.
The grading breakdown is as follows:
- 5 Labs (15% x 5)
- Final Project (25%)
- Proposal (5%)
- Presentation + Final Report (20%)
- Participation Bonus (4%)
Note that this class does not have any tests or exams.
Labs
There will be 5 labs over the course of the semester.
- Lab1: Pruning
- Lab2: Quantization
- Lab3: Neural architecture search
- Lab4: LLM compression
- Lab5: LLM deployment on laptop
Collaboration Policy
Labs must be done individually: each student must hand in their own answers. However, it is acceptable to collaborate when figuring out answers and to help each other solve the problems. We will be assuming that, as participants in a graduate course, you will be taking the responsibility to make sure you personally understand the solution arising from such collaboration. You also must indicate on each homework with whom you have collaborated.
Late Policy
You will be allowed 6 total homework late days without penalty for the entire semester. You may be late by up to 6 days on any homework assignment. Once those days are used, you will be penalized according to the following policy:
- Homework is worth full credit at the due time on the due date.
- The allowed late days are counted by day (i.e., each new late day starts at 11:59 pm ET).
- Once the allowed late days are exceeded, the penalty is 50% per late day counted by day.
- The homework is worth zero credit 2 days after exceeding the late day limit.
You must turn in at least 4 of the 5 assignments, even if for zero credit, in order to pass the course.
Regrade Policy
If you feel that we have made a mistake in grading your work, please submit a regrading request to TAs during the office hour and we will consider your request. Please note that regrading of a homework may cause your grade to go either up or down.
Final Project
The class project will be carried out in groups of 3 or 4 people, and has three main parts:
- proposal: choose from a list of suggested projects, or propose your own project
- poster presentation
- final report (4 pages, using the NeurIPS template)
Participation Bonus
We appreciate everyone being actively involved in the class! Around the middle and end of the semester, we will send out a survey to help us understand how the course is going, and how we can improve. Completing it is worth 4% in total.
