PreprintarXiv.org2025
Stronger Normalization-Free Transformers
Mingzhi Chen, Taiming Lu, Jiacheng Zhu, et al.
Introduces Derf, a normalization-free pointwise function that outperforms LayerNorm, RMSNorm, and DyT across vision, speech, and DNA tasks.
13Dec 11, 2025Transformers
arXiv