MobileNetV4

Analysis

Why This Paper Matters

MobileNetV4 tackles a critical challenge in deploying deep learning on mobile and edge devices: achieving high accuracy while maintaining low latency across a wide range of hardware accelerators (CPUs, GPUs, DSPs, NPUs). Previous MobileNet versions were optimized primarily for specific hardware, but the mobile ecosystem is increasingly diverse. This paper introduces a universally efficient architecture that adapts to different compute and memory bottlenecks, making it practical for real-world deployment. The key innovations—the Universal Inverted Bottleneck (UIB) block, Mobile MQA attention, and an improved neural architecture search (NAS) recipe—collectively push the Pareto frontier of accuracy vs. latency.

How It Works

Hardware-Independent Pareto Efficiency

The authors use the roofline model to analyze latency bottlenecks across hardware. The ridge point (RP) is the ratio of peak MACs to peak memory bandwidth. By sweeping RP from 0 to 500 MACs/byte, they identify that low-RP hardware (e.g., CPUs) is compute-bound, while high-RP hardware (e.g., GPUs) is memory-bound. MobileNetV4 balances these by using large initial convolutional layers (expensive on low-RP but cheap on high-RP) and large final fully-connected layers (expensive on high-RP but cheap on low-RP), ensuring no single variant suffers both slowdowns.

Universal Inverted Bottleneck (UIB)

The UIB block extends the inverted bottleneck from MobileNetV2 by adding two optional depthwise convolutions: one before the expansion layer and one between expansion and projection. This yields four variants:

• Inverted Bottleneck (IB): spatial mixing on expanded features.
• ConvNext: cheaper spatial mixing with larger kernels before expansion.
• ExtraDW: new variant for inexpensive depth and receptive field increase.
• FFN: two pointwise convolutions with activation and normalization.

UIB provides flexibility in spatial/channel mixing, receptive field enlargement, and computational utilization. The presence and kernel size of the depthwise layers are searched via NAS.

Mobile MQA

Multi-Query Attention (MQA) uses one shared head for keys and values, reducing memory access and improving operational intensity. Mobile MQA incorporates Spatial Reduction Attention (SRA) using a 3x3 depthwise convolution with stride 2 to downsample keys and values while retaining high-resolution queries. This maintains token count and preserves attention quality while being efficient.

NAS Recipe

The NAS is performed in two stages:

1. Coarse-grained search: determine optimal filter sizes with fixed IB block (expansion factor 4, 3x3 depthwise).
2. Fine-grained search: search the configuration of UIB's two depthwise layers (presence and kernel size 3x3 or 5x5) with fixed expansion factor.

Design principles prioritize standard components (depthwise/pointwise convolutions, ReLU, BatchNorm) and avoid group convolutions, SE, GELU, and LayerNorm due to hardware incompatibility.

Results

ImageNet Classification

MobileNetV4 models were trained on ImageNet-1K and compared against leading efficient models (MobileOne, ConvNext, FastViT, EfficientViT). Key results:

• CPUs: MNv4 is ~2x faster than MobileNetV3 at same accuracy.
• EdgeTPUs: MNv4 doubles speed over MobileNetV3; MNv4-Conv-M is 50% faster than MobileOne-S4 and FastViT-S12 with 1.5% higher accuracy.
• S23 GPU and iPhone 13 CoreML: MNv4 models are on the Pareto front; EfficientViT has 2x latency on CoreML, FastViT has 5x latency on S23 GPU.
• DSPs: MNv4-Conv models are top performers.

COCO Object Detection

Using RetinaNet with FPN, MNv4-Conv-M achieves 32.6% AP, comparable to MobileNet Multi-AVG and MobileNetV2, but with 12% and 23% lower Pixel 6 CPU latency respectively. Adding Mobile MQA boosts AP by +1.6%.

Distillation

The enhanced distillation recipe dynamically mixes three datasets:

• D1: Inception Crop + RandAugment on ImageNet.
• D2: Inception Crop + extreme Mixup on ImageNet.
• D3: Class-balanced JFT-300M data with weaker augmentations.

Dynamic mixing (D1+D2) yields 84.4% top-1 accuracy (+0.3% over D2 alone). Adding JFT (D3) gives another 0.6% improvement. The full recipe achieves 85.9% for MNv4-Conv-L (2000 epochs) and 87.0% for MNv4-Hybrid with JFT pretraining.

Significance

MobileNetV4 provides a practical blueprint for designing efficient models that work well across diverse mobile hardware. The UIB block and Mobile MQA are modular innovations that can be adopted in other architectures. The NAS recipe and distillation techniques offer reproducible improvements. This work is likely to influence future mobile vision models and edge AI deployments, especially as hardware diversity increases. The paper also demonstrates that careful co-design of architecture, search, and training can yield substantial gains without exotic components.