Efficient AI Computing,
Transforming the Future.

Publications

VLASH: Real-Time VLAs via Future-State-Aware Asynchronous Inference

Jiaming Tang¹*, Yufei Sun¹ ³*, Yilong Zhao⁴, Shang Yang¹, Yujun Lin², Zhuoyang Zhang¹, James Hou¹ ⁶, Yao Lu², Zhijian Liu² ⁵, Song Han¹ ²
IROS 2026

ForeAct: Steering Your VLA with Efficient Visual Foresight Planning

Zhuoyang Zhang*, Shang Yang*, Qinghao Hu, Luke J. Huang, James Hou, Yufei Sun, Yao Lu, Song Han
CVPR 2026 Highlight

Locality-aware Parallel Decoding for Efficient Autoregressive Image Generation

Zhuoyang Zhang*, Luke J. Huang*, Chengyue Wu, Shang Yang, Yao Lu, Song Han
ICLR 2026 Oral

StreamingVLM: Real-Time Understanding for Infinite Video Streams

Ruyi Xu¹*, Guangxuan Xiao¹*, Yukang Chen¹, Liuning He¹, Yao Lu², Song Han¹,²
ICLR 2026

LongLive: Real-time Interactive Long Video Generation

Shuai Yang, Wei Huang, Ruihang Chu, Yicheng Xiao, Yuyang Zhao, Xianbang Wang, Muyang Li, Enze Xie, Yingcong Chen, Yao Lu, Song Han, Yukang Chen
ICLR 2026

QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMs

Wei Huang, Yi Ge, Shuai Yang, Yicheng Xiao, Huizi Mao, Yujun Lin, Hanrong Ye, Sifei Liu, Ka Chun Cheung, Hongxu Yin, Yao Lu, Xiaojuan Qi, Song Han, Yukang Chen
ICLR 2026

Fast-dLLM v2: Efficient Block-Diffusion Large Language Model

Chengyue Wu, Hao Zhang, Shuchen Xue, Shizhe Diao, Yonggan Fu, Zhijian Liu, Pavlo Molchanov, Ping Luo, Song Han, Enze Xie
ICLR 2026

Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding

Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, Enze Xie
ICLR 2026

SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer

Junsong Chen¹²*, Yuyang Zhao¹*, Jincheng Yu¹*, Ruihang Chu⁴, Junyu Chen¹, Shuai Yang¹, Xianbang Wang³, Yicheng Pan⁴, Daquan Zhou⁵, Huan Ling¹, Haozhe Liu⁶, Hongwei Yi¹, Hao Zhang¹, Muyang Li³, Yukang Chen¹, Han Cai¹, Sanja Fidler¹, Ping Luo², Song Han¹³, Enze Xie¹
ICLR 2026 Oral

Taming the Long-Tail: Efficient Reasoning RL Training with Adaptive Drafter

Qinghao Hu*, Shang Yang*, Junxian Guo, Xiaozhe Yao, Yujun Lin, Yuxian Gu, Han Cai, Chuang Gan, Ana Klimovic, Song Han
ASPLOS 2026

Long-RL: Scaling RL to Long Videos

Yukang Chen*, Wei Huang*, Baifeng Shi, Qinghao Hu, Hanrong Ye, Ligeng Zhu, Zhijian Liu, Pavlo Molchanov, Jan Kautz, Xiaojuan Qi, Sifei Liu, Hongxu Yin, Yao Lu, Song Han
NeurIPS 2025

Radial Attention: O(nlogn) Sparse Attention with Energy Decay for Long Video Generation

Xingyang Li*, Muyang Li*, Tianle Cai, Haocheng Xi, Shuo Yang, Yujun Lin, Lvmin Zhang, Songlin Yang, Jinbo Hu, Kelly Peng, Maneesh Agrawala, Ion Stoica, Kurt Keutzer, Song Han
NeurIPS 2025

SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation

Junsong Chen¹*, Shuchen Xue⁵*, Yuyang Zhao¹⁺, Jincheng Yu¹⁺, Sayak Paul⁴, Junyu Chen³, Han Cai¹, Song Han¹², Enze Xie¹
ICCV 2025

DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizer

Yecheng Wu, Junyu Chen, Zhuoyang Zhang, Enze Xie, Jincheng Yu, Junsong Chen, Jinyi Hu, Yao Lu, Song Han, Han Cai
ICCV 2025

DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Space

Junyu Chen†, Dongyun Zou, Wenkun He, Junsong Chen, Enze Xie, Song Han, Han Cai†
ICCV 2025

DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space

Wenkun He*, Yuchao Gu*, Junyu Chen*, Dongyun Zou, Yujun Lin, Zhekai Zhang, Haocheng Xi, Muyang Li, Ligeng Zhu, Jincheng Yu, Junsong Chen, Enze Xie, Song Han, Han Cai
ArXiv

DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencoder

Junyu Chen*, Wenkun He*, Yuchao Gu*, Yuyang Zhao, Jincheng Yu, Junsong Chen, Dongyun Zou, Yujun Lin, Zhekai Zhang, Muyang Li, Haocheng Xi, Ligeng Zhu, Enze Xie, Song Han, Han Cai
Arxiv

Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search

Yuxian Gu, Qinghao Hu, Shang Yang, Haocheng Xi, Junyu Chen, Song Han, Han Cai
NeurIPS 2025

XAttention: Block Sparse Attention with Antidiagonal Scoring

Ruyi Xu*, Guangxuan Xiao*, Haofeng Huang, Junxian Guo, Song Han
ICML 2025

SANA-1.5: Efficient Scaling of Training-Time and
Inference-Time Compute in Linear Diffusion Transformer

Enze Xie¹*, Junsong Chen¹*, Yuyang Zhao¹⁺, Jincheng Yu¹⁺, Ligeng Zhu¹⁺, Chengyue Wu⁶, Yujun Lin², Zhekai Zhang², Muyang Li², Junyu Chen³, Han Cai¹, Bingchen Liu⁴, Daquan Zhou⁵, Song Han¹²
ICML 2025

LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention

Shang Yang*¹, Junxian Guo*¹³, Haotian Tang¹, Qinghao Hu¹, Guangxuan Xiao¹, Jiaming Tang¹, Yujun Lin¹, Zhijian Liu², Yao Lu², Song Han¹²
MLSys 2025

QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

Yujun Lin*¹, Haotian Tang*¹, Shang Yang*¹, Zhekai Zhang¹, Guangxuan Xiao¹, Chuang Gan³⁴, Song Han¹²
MLSys 2025

SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models

Muyang Li*, Yujun Lin*, Zhekai Zhang*, Tianle Cai, Xiuyu Li, Junxian Guo, Enze Xie, Chenlin Meng, Jun-Yan Zhu, Song Han
ICLR 2025

COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training

Haocheng Xi¹, Han Cai², Ligeng Zhu², Yao Lu², Kurt Keutzer¹, Jianfei Chen⁴, Song Han²³
ICLR 2025

SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers

Enze Xie¹*, Junsong Chen¹*, Junyu Chen²³, Han Cai¹, Haotian Tang², Yujun Lin², Zhekai Zhang², Muyang Li², Ligeng Zhu¹, Yao Lu¹, Song Han¹²
ICLR 2025

DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads

Guangxuan Xiao¹, Jiaming Tang¹, Jingwei Zuo², Junxian Guo¹ ³, Shang Yang¹, Haotian Tang¹, Yao Fu⁴, Song Han¹ ⁵
ICLR 2025

Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models

Junyu Chen*, Han Cai*, Junsong Chen, Enze Xie, Shang Yang, Haotian Tang, Muyang Li, Yao Lu, Song Han
ICLR 2025

LongVILA: Scaling Long-Context Visual Language Models for Long Videos

Yukang Chen¹*, Fuzhao Xue¹*, Dacheng Li¹³*, Qinghao Hu²*, Ligeng Zhu¹, Xiuyu Li¹³, Yunhao Fang¹, Haotian Tang¹², Shang Yang¹², Zhijian Liu¹, Ethan He¹, Hongxu Yin¹, Pavlo Molchanov¹, Jan Kautz¹, Linxi Fan¹, Yuke Zhu¹⁴, Yao Lu¹, Song Han¹²
ICLR 2025

VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation

Yecheng Wu*, Zhuoyang Zhang*, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, Song Han, Yao Lu
ICLR 2025

HART: Efficient Visual Generation with Hybrid Autoregressive Transformer

Haotian Tang*, Yecheng Wu*, Shang Yang, Enze Xie, Junsong Chen, Junyu Chen, Zhuoyang Zhang, Han Cai, Yao Lu, Song Han
ICLR 2025

LEGO: Spatial Accelerator Generation and Optimization for Tensor Applications

Yujun Lin*, Zhekai Zhang*, Song Han
HPCA 2025

Sparse Refinement for Efficient High-Resolution Semantic Segmentation

Zhijian Liu*, Zhuoyang Zhang*, Samir Khaki, Shang Yang, Haotian Tang, Chenfeng Xu, Kurt Keutzer, Song Han
ECCV 2024

FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention

Guangxuan Xiao*¹, Tianwei Yin*¹, William T. Freeman¹, Frédo Durand¹, Song Han¹
International Journal of Computer Vision 2024

Atomique: A Quantum Compiler for Reconfigurable Neutral Atom Arrays

Hanrui Wang, Pengyu Liu, Daniel Bochen Tan, Yilian Liu, Jiaqi Gu, David Z. Pan, Jason Cong, Umut A. Acar, Song Han
ISCA 2024

Q-Pilot: Field Programmable Qubit Array Compilation with Flying Ancillas

Hanrui Wang, Daniel Bochen Tan, Pengyu Liu, Yilian Liu, Jiaqi Gu, Jason Cong, Song Han
DAC 2024

DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models

Muyang Li*, Tianle Cai*, Jiaxin Cao, Qinsheng Zhang, Han Cai, Junjie Bai, Yangqing Jia, Ming-Yu Liu, Kai Li, and Song Han
CVPR 2024

Condition-Aware Neural Network for Controlled Image Generation

Han Cai, Muyang Li, Zhuoyang Zhang, Qinsheng Zhang, Ming-Yu Liu, Song Han
CVPR 2024

VILA: On Pre-training for Visual Language Models

Ji Lin*, Hongxu Yin*, Wei Ping, Yao Lu, Pavlo Molchanov, Andrew Tao, Huizi Mao, Jan Kautz, Mohammad Shoeybi, Song Han
CVPR 2024

Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference

Jiaming Tang*, Yilong Zhao*, Kan Zhu, Guangxuan Xiao, Baris Kasikci, Song Han
ICML 2024

Efficient Streaming Language Models with Attention Sinks

Guangxuan Xiao¹, Yuandong Tian², Beidi Chen³, Song Han¹⁴, Mike Lewis²
ICLR 2024

AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

Ji Lin*, Jiaming Tang*, Haotian Tang*, Shang Yang*, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han
MLSys 2024

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models

Yukang Chen¹, Shengju Qian¹, Haotian Tang², Xin Lai¹, Zhijian Liu², Song Han², Jiaya Jia¹
ICLR 2024

Tiny Machine Learning Projects

Ji Lin, Ligeng Zhu, Wei-Ming Chen, Wei-Chen Wang, Han Cai, Guangxuan Xiao, Haotian Tang, Shang Yang, Yujun Lin, and Song Han
NeurIPS 2020/2021/2022, MICRO 2023, ICML 2023, MLSys 2024, IEEE CAS Magazine 2023

Tiny Machine Learning: Progress and Futures [Feature]

Ji Lin, Ligeng Zhu, Wei-Ming Chen, Wei-Chen Wang, and Song Han
IEEE CAS magazine

PockEngine: Sparse and Efficient Fine-tuning in a Pocket

Ligeng Zhu, Lanxiang Hu, Ji Lin, Wei-Chen Wang, Wei-Ming Chen, Chuang Gan, Song Han
MICRO 2023

EfficientViT: Multi-Scale Linear Attention for High-Resolution Dense Prediction

Han Cai, Junyan Li, Muyan Hu, Chuang Gan, Song Han
ICCV 2023

SpAtten-Chip: A Fully-Integrated Energy-Scalable Transformer Accelerator Supporting Adaptive Model Configuration and Word Elimination for Language Understanding on Edge Devices

Zexi Ji, Hanrui Wang, Miaorong Wang, Win-San Khwa, Meng-Fan Chang, Song Han, Anantha P Chandrakasan
ISLPED 2023

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

Guangxuan Xiao*¹, Ji Lin*¹, Mickael Seznec², Hao Wu², Julien Demouth², Song Han¹
ICML 2023

FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer

Zhijian Liu*, Xinyu Yang*, Haotian Tang, Shang Yang, Song Han
CVPR 2023

Retrospective: EIE: Efficient Inference Engine on Sparse and Compressed Neural Network

Song Han¹³, Xingyu Liu⁴, Huizi Mao³ , Jing Pu⁵ , Ardavan Pedram²⁶ , Mark A. Horowitz² , William J. Dally²³
ISCA 2023

BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation

Zhijian Liu*, Haotian Tang*, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela L. Rus, Song Han
ICRA 2023

QuEst: Graph Transformer for Quantum Circuit Reliability Estimation

Hanrui Wang, Pengyu Liu, Jinglei Cheng, Zhiding Liang, Jiaqi Gu, Zirui Li, Yongshan Ding, Weiwen Jiang, Yiyu Shi, Xuehai Qian, David Z Pan, Frederic T Chong, Song Han
ICCAD 2022

Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models

Muyang Li¹, Ji Lin¹, Chenlin Meng³, Stefano Ermon³, Song Han¹, and Jun-Yan Zhu²
NeurIPS 2022 & TPAMI

TorchSparse: Efficient Point Cloud Inference Engine

Haotian Tang*, Zhijian Liu*, Xiuyu Li*, Yujun Lin, Song Han
MLSys 2022

QOC: Quantum On-Chip Training with Parameter Shift and Gradient Pruning

Hanrui Wang¹, Zirui Li², Jiaqi Gu³, Yongshan Ding⁴, Yujun Lin¹, David Z. Pan³, Frederic T. Chong⁵, Song Han¹
DAC 2022

QuantumNAT: Quantum Noise-Aware Training with Noise Injection, Quantization and Normalization

Hanrui Wang¹, Jiaqi Gu², Yongshan Ding³, Zirui Li⁴, David Z. Pan³, Frederic T. Chong⁵, Song Han¹
DAC 2022

On-Device Training Under 256KB Memory

Ji Lin*, Ligeng Zhu*, Wei-Ming Chen, Wei-Chen Wang, Chuang Gan, Song Han
NeurIPS 2022

Lite Pose: Efficient Architecture Design for 2D Human Pose Estimation

Yihan Wang, Muyang Li, Han Cai, Wei-Ming Chen, Song Han
CVPR 2022

QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuits

Hanrui Wang¹, Yongshan Ding², Jiaqi Gu³, Zirui Li⁴, Yujun Lin¹, David Z. Pan³, Frederic T. Chong⁵, Song Han¹
HPCA 2022

Network Augmentation for Tiny Deep Learning

Han Cai, Chuang Gan, Ji Lin, Song Han
ICLR 2022

NAAS: Neural Accelerator Architecture Search

Yujun Lin *¹, Mengtian Yang *², Song Han¹
DAC 2021

Delayed Gradient Averaging: Tolerate the Communication Latency in Federated Learning

Ligeng Zhu¹, Hongzhou Lin², Yao Lu³, Yujun Lin¹, Song Han¹
NeurIPS 2021

MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning

Ji Lin, Wei-Ming Chen, Han Cai, Chuang Gan, Song Han
NeurIPS 2021

PointAcc: Efficient Point Cloud Accelerator

Yujun Lin, Zhekai Zhang, Haotian Tang, Hanrui Wang, Song Han
MICRO 2021

SemAlign: Annotation-Free Camera-LiDAR Calibration with Semantic Alignment Loss

Zhijian Liu*, Haotian Tang*, Sibo Zhu*, Song Han
IROS 2021

Anycost GANs for Interactive Image Synthesis and Editing

Ji Lin, Richard Zhang, Frieder Ganz, Song Han, Jun-Yan Zhu
CVPR 2021

SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning

Hanrui Wang, Zhekai Zhang, Song Han
HPCA 2021

Differentiable Augmentation for Data-Efficient GAN Training

Shengyu Zhao, Zhijian Liu, Ji Lin, Jun-Yan Zhu, Song Han
NeurIPS 2020

TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning

Han Cai, Chuang Gan, Ligeng Zhu, Song Han
NeurIPS 2020

Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution

Haotian Tang*, Zhijian Liu*, Shengyu Zhao, Yujun Lin, Ji Lin, Hanrui Wang, Song Han
ECCV 2020

GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learning

Hanrui Wang, Kuan Wang, Jiacheng Yang, Linxiao Shen, Nan Sun, Hae-Seung Lee, Song Han
DAC 2020

MCUNet: Tiny Deep Learning on IoT Devices

Ji Lin, Wei-Ming Chen, Yujun Lin, John Cohn, Chuang Gan, Song Han
NeurIPS 2020

APQ: Joint Search for Nerwork Architecture, Pruning and Quantization Policy

Tianzhe Wang, Kuan Wang, Han Cai, Ji Lin, Zhijian Liu, Hanrui Wang, Yujun Lin, Song Han
CVPR 2020

GAN Compression: Efficient Architectures for Interactive Conditional GANs

Muyang Li¹, Ji Lin², Yaoyao Ding³, Zhijian Liu², Jun-Yan Zhu¹ and Song Han²
CVPR 2020 & TPAMI

SpArch: Efficient Architecture for Sparse Matrix Multiplication

Zhekai Zhang*, Hanrui Wang*, Song Han, William J. Dally
HPCA 2020

Lite Transformer with Long-Short Range Attention

Zhanghao Wu*, Zhijian Liu*, Ji Lin, Yujun Lin, Song Han
ICLR 2020

Once-for-All: Train One Network and Specialize it for Efficient Deployment

Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, Song Han
ICLR 2020

HAT: Hardware-Aware Transformers for Efficient Natural Language Processing

Hanrui Wang¹, Zhanghao Wu¹, Zhijian Liu¹, Han Cai¹, Ligeng Zhu¹, Chuang Gan², Song Han¹
ACL 2020

Park: An Open Platform for Learning-Augmented Computer Systems

Hongzi Mao, Parimarjan Negi, Akshay Narayan, Hanrui Wang, Jiacheng Yang, Haonan Wang, Ryan Marcus, Mehrdad Khani Shirkoohi, Songtao He, Vikram Nathan, Frank Cangialosi, Shaileshh Venkatakrishnan, Wei-Hung Weng, Song Han, Tim Kraska, Mohammad Alizadeh
NeurIPS 2019

Point-Voxel CNN for Efficient 3D Deep Learning

Zhijian Liu*, Haotian Tang*, Yujun Lin, Song Han
NeurIPS 2019

TSM: Temporal Shift Module for Efficient Video Understanding

Ji Lin¹, Chuang Gan², Song Han¹
ICCV 2019

HAQ: Hardware-Aware Automated Quantization

Kuan Wang*, Zhijian Liu*, Yujun Lin*, Ji Lin, and Song Han
CVPR 2019

ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

Han Cai, Ligeng Zhu, Song Han
ICLR 2019

Deep Gradient Compression: Reducing the Communication Bandwidth in Distributed Training

Yujun Lin¹, Song Han², Huizi Mao², Yu Wang¹, William J. Dally²³
ICLR 2018

Learning to Design Circuits

Hanrui Wang, Jiacheng Yang, Hae-Seung Lee, Song Han
NIPS 2019 MLSys Workshop

AMC: AutoML for Model Compression and Acceleration on Mobile Devices

Yihui He*, Ji Lin*, Zhijian Liu, Hanrui Wang, Li-Jia Li, and Song Han
ECCV 2018

EIE: efficient inference engine on compressed deep neural network

Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark A. Horowitz, William J. Dally
ISCA 2016

Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding

Song Han, Huizi Mao, and William J. Dally
ICLR 2016

Learning both Weights and Connections for Efficient Neural Network

Song Han, Jeff Pool, John Tran, William Dally
NIPS 2015
USEFUL LINKS
Homepage of Prof. Song HanHomepage of MIT HAN LabMIT Accessibility
Copyright © MIT Han Lab.
Designed and Developed by Yujun Lin.