SANA: O(n²)→O(n) Linear Attention Generates 1024² Images in 0.6 Seconds
How Linear Attention solved Self-Attention quadratic complexity. The secret behind 100x faster generation compared to DiT.
SANA: Ultra-Fast High-Resolution Image Generation with Linear Attention
TL;DR: SANA generates 1024×1024 images in just 0.6 seconds through Linear Attention and efficient token compression. It's a groundbreaking architecture that's 100x faster than DiT while maintaining equivalent quality.
1. Introduction: Overcoming the Speed-Quality Tradeoff
1.1 Speed Issues with Existing Diffusion Models
High-resolution image generation is computationally expensive:
The rest of this series is paid
- ✓ The whole “Diffusion models: DDPM to DiT” series: 6 posts in English, 3 in Korean
- ✓ 6 Jupyter notebooks
- ✓ One payment, no expiry. Opens when you sign in with the email you pay with.
The series pass needs no account to pay. The subscription needs a free account.