MIT Han Lab's EfficientViT-L2 Slashes Vision AI Costs at High Resolution

MIT Han Lab’s EfficientViT-L2 gives PyTorch developers an ImageNet-1k classifier and reusable feature backbone whose attention cost scales roughly linearly with token count. The author-trained efficientvit_l2.r288_in1k checkpoint is available through timm, uses a 288 × 288 evaluation recipe, and carries an Apache-2.0 license on its model card.

Developers can use the checkpoint for classification, pooled embeddings, or multi-scale feature extraction. Its main architectural advantage appears in dense tasks such as segmentation, detection, and image restoration, where conventional attention becomes expensive as resolution rises. The published dense-prediction results use task-specific EfficientViT models and heads, so they should not be attributed directly to this ImageNet checkpoint.

Linear attention trims the token matrix

Full softmax attention constructs an N × N interaction matrix for N image tokens. Its compute and memory therefore grow quadratically with token count. Increasing both image dimensions produces many more tokens, making attention costly well before convolutional layers reach the same bottleneck.

EfficientViT uses ReLU-based linear attention, which rearranges the computation to avoid materializing that full interaction matrix. With channel width held constant, the attention cost grows roughly linearly with token count. Depthwise convolutions and multi-scale aggregation add neighborhood information and capture patterns at several spatial scales. The design is described in the ICCV 2023 paper by Han Cai, Junyan Li, Muyan Hu, Chuang Gan, and Song Han.

Inside the L2 checkpoint

The author-trained L2 checkpoint targets ImageNet-1k classification at a default evaluation resolution of 288 × 288. Its reported size and compute requirements are: