TinyML and Efficient AI Computing
This course focuses on efficient machine learning and systems. This is a crucial area as deep neural networks demand extraordinary levels of computation, hindering its deployment on everyday devices and burdening the cloud infrastructure. This course introduces efficient AI computing techniques that enable powerful deep learning applications on resource-constrained devices. Topics include model compression, pruning, quantization, neural architecture search, distributed training, model serving, model parallelism, gradient compression, and on-device fine-tuning. It also introduces application-specific acceleration techniques for large language models and diffusion models. Students will get hands-on experience implementing model compression techniques and deploying large language models on a laptop.
- Lecture Videos:https://live.efficientml.ai/
- Time:
Tuesday/Thursday 4:00-5:30 PM
- Location:54-100
- Office Hour:
Thursday 5:30-6:30 pm Eastern Time, 54-100
- Discussion:Piazza
- Homework Submission:Canvas
- Contact:
- Contact: MIT students who are enrolled in this class can email us at efficientml-staff [at] mit.edu.
- Prerequisites: Both 6.191 Computation Structures and 6.390 Intro to Machine Learning. We do not approve prerequisite waiver petitions.
- Cross-registration: We are unable to accommodate cross-registered students this semester.
- Please do not submit petitions or email the course staff requesting exceptions to these policies.
Instructors
Teaching Assistants
Announcements
Schedule
Date
Lecture
Logistics
Logistics
The class requirements include five labs, and one final project. This is a PhD level course, and by the end of this class you should have a good understanding of efficient deep learning techniques, and be able to deploy large language models (LLMs) on your laptop.
Note that this class does not have any tests or exams.
Labs
There will be 5 labs over the course of the semester.
- Lab1: GPU Basics
- Lab2: Quantization
- Lab3: Neural architecture search
- Lab4: Quantization
- Lab5: LLM deployment on laptop
Collaboration Policy
Labs must be done individually: each student must hand in their own answers. However, it is acceptable to collaborate when figuring out answers and to help each other solve the problems. We will be assuming that, as participants in a graduate course, you will be taking the responsibility to make sure you personally understand the solution arising from such collaboration. You also must indicate on each homework with whom you have collaborated.
Late Policy
You will be allowed 6 total homework late days without penalty for the entire semester. You may be late by up to 6 days on any homework assignment. Once those days are used, you will be penalized according to the following policy:
- Homework is worth full credit at the due time on the due date.
- The allowed late days are counted by day (i.e., each new late day starts at 11:59 pm ET).
- Once the allowed late days are exceeded, the penalty is 50% per late day counted by day.
- The homework is worth zero credit 2 days after exceeding the late day limit.
You must turn in at least 4 of the 5 assignments, even if for zero credit, in order to pass the course.
Final Project
The class project will be carried out in groups, and has three main parts:
- proposal: choose from a list of suggested projects, or propose your own project
- poster presentation
- final report (4 pages, using the NeurIPS template)
