YuE

YuE is a family of open-source foundation models based on the LLaMA2 architecture, designed for long-form music generation—specifically the challenging lyrics-to-song problem. It scales to trillions of tokens and can generate up to five minutes of music while maintaining lyrical alignment, coherent musical structure, and engaging vocal melodies with appropriate accompaniment. Key technical innovations include track-decoupled next-token prediction to handle dense mixture signals, structural progressive conditioning for long-context lyrical alignment, and a multitask, multiphase pre-training recipe. YuE also introduces in-context learning for music generation, enabling versatile style transfer (e.g., converting Japanese city pop into an English rap while preserving accompaniment) and bidirectional generation. Extensive evaluation shows YuE matches or surpasses proprietary systems in musicality and vocal agility. Fine-tuning YuE enables additional controls and support for tail languages. Beyond generation, YuE's learned representations perform well on music understanding tasks, matching or exceeding state-of-the-art methods on the MARBLE benchmark.