Large-Scale Data Selection for Instruction Tuning
Unknown
Investigates automated data selection for instruction-tuning LLMs, finding many methods underperform random selection at scale, while a representation-based variant (RDS+) consistently outperforms others.