Kimi k1.5

Analysis

Why This Paper Matters

Large language models have made impressive strides in reasoning, but most approaches either rely on expensive search methods (like Monte Carlo tree search) or produce excessively long outputs that are costly at inference time. Kimi k1.5 tackles both problems head-on: it shows that a straightforward reinforcement learning pipeline—without complex value functions or process reward models—can achieve state-of-the-art reasoning across text, vision, and code. More importantly, it introduces a family of "long2short" compression techniques that distill the reasoning power of long-context models into short, efficient outputs. This matters because it offers a practical path to deploy powerful reasoning models without blowing up latency or compute budgets.

How It Works

The training pipeline has four stages: pretraining, vanilla supervised fine-tuning (SFT), long-CoT SFT, and reinforcement learning. The paper focuses on the RL stage. First, the team curates a high-quality RL prompt set by tagging prompts for diversity, measuring difficulty via pass rate from a base model, and excluding questions prone to reward hacking (e.g., multiple-choice). Then, a small long-CoT warmup dataset is built using prompt engineering to encourage planning, evaluation, reflection, and exploration.

During RL training, two sampling strategies improve efficiency: curriculum sampling (starting with easier tasks) and prioritized sampling (focusing on problems with low success rates). The key innovation is the long2short compression, which addresses the "overthinking" problem—models naturally generate longer responses during RL, which improves accuracy but hurts efficiency. Four methods are proposed:

• Model merging: average weights of a long-CoT and a short model.
• Shortest rejection sampling: pick the shortest correct response from multiple samples.
• DPO: treat the shortest correct answer as positive, longer correct or wrong answers as negative.
• Length penalty: add a reward term that penalizes token length, applied after an initial phase without penalty to avoid slowing early training.

For coding, test cases are automatically generated using the CYaRon library and a base Kimi model, with validity checks via multiple ground-truth submissions. For math, a chain-of-thought reward model (trained on 800k examples) achieves 98.5% accuracy in manual spot checks, far outperforming a classic value-head reward model (84.4%).

Vision data is handled through three categories: real-world images (charts, science diagrams), synthetic visual reasoning data (spatial relationships, geometric patterns), and text-rendered data (screenshots, document photos) to ensure modality-consistent responses.

Results

The long-CoT model achieves state-of-the-art results across multiple modalities, matching OpenAI's o1 on long-context tasks and outperforming GPT-4o and Claude Sonnet 3.5 on short-context benchmarks. The short-CoT models, trained with long2short RL, show remarkable token efficiency: k1.5-short with RL scores 60.8% on AIME2024 using only 3,272 tokens on average, while k1.5-shortest achieves 88.2% on MATH500 with token usage comparable to other short models.

Scaling experiments confirm a strong correlation between context length and performance: longer contexts lead to better accuracy, especially on hard reasoning benchmarks. The 128k context version continues to improve where shorter models plateau.

Significance

Kimi k1.5 demonstrates that a clean RL framework—without complex search or auxiliary value models—can push multimodal reasoning to new heights. The long2short techniques are particularly valuable for practitioners: they show how to compress the reasoning power of a long-context model into a fast, efficient short-context model without sacrificing accuracy. This work also provides practical recipes for reward modeling (chain-of-thought RM), test case generation for code, and vision-language alignment. The main limitation is the lack of analysis on prompt set biases and the cost of generating high-quality warmup data. Overall, Kimi k1.5 offers a blueprint for building scalable, efficient reasoning systems that can be deployed in real-world applications.