DiT: Replacing U-Net with Transformer Finally Made Scaling Laws Work (Sora Foundation)
U-Net shows diminishing returns when scaled up. DiT improves consistently with size. Complete analysis of the architecture behind Sora.
DiT: Diffusion Transformer - A New Paradigm Beyond U-Net
TL;DR: DiT replaces the Diffusion model's backbone from U-Net to Vision Transformer. Scaling laws apply, so performance consistently improves as the model grows larger. It's the foundation technology behind Sora.
1. Limitations of U-Net
1.1 Why U-Net?
The rest of this series is paid
- ✓ The whole “Diffusion models: DDPM to DiT” series: 6 posts in English, 3 in Korean
- ✓ 6 Jupyter notebooks
- ✓ One payment, no expiry. Opens when you sign in with the email you pay with.
The series pass needs no account to pay. The subscription needs a free account.