Improving Retrieval in LLMs through Synthetic Data
Zheyang Xiong, Vasileios Papageorgiou, Kangwook Lee, et al.
Proposes finetuning LLMs on synthetic key-value retrieval data to improve long-context retrieval and reasoning without harming general benchmarks.