Eagle 2.5

Analysis

Why This Paper Matters

Vision-language models (VLMs) have made remarkable strides in understanding static images and short video clips, but two critical challenges remain: processing long-form videos (e.g., 30+ minutes) and high-resolution images without losing contextual integrity or visual detail. Most existing VLMs either sacrifice temporal coverage for spatial resolution or vice versa, and they often require separate specialized models for each task. Eagle 2.5 addresses this gap by introducing a unified framework that dynamically allocates computational resources between spatial and temporal dimensions, enabling a single model to excel at both long video comprehension and high-resolution image understanding. This is particularly relevant for real-world applications like surveillance video analysis, medical imaging, and document processing where both long context and fine details matter.

How It Works

Eagle 2.5 builds on the LLaVA architecture, using an MLP projection layer to align SigLIP vision embeddings with a Qwen2.5 language backbone. To handle arbitrary image resolutions, it adopts an image tiling strategy inspired by LLaVA-1.5 and InternVL, but with two key innovations: Image Area Preservation (IAP) and Automatic Degrade Sampling (ADS).

Image Area Preservation (IAP) addresses a fundamental flaw in traditional tiling: rigid grids often distort image geometry by ignoring aspect ratios. IAP optimizes two objectives: maintaining at least 60% of the original image area in the tiled version, and aligning the tiling ratio with the original aspect ratio. The optimal tiling configuration is selected by maximizing a formula that penalizes configurations where the new area falls below 60% and rewards those where the tiling ratio matches the original aspect ratio.

Automatic Degrade Sampling (ADS) tackles the core challenge of allocating a fixed sequence length budget between visual and textual tokens. Given a training sample with maximum sequence length, ADS first computes the text token length, then derives a visual token budget. For images, it optimizes the maximal tile count per image to maximize spatial information; for temporal content (videos/documents), it optimizes the sampling count to maximize temporal coverage. This is formulated as a constrained optimization problem.

ADS operates in two phases: temporal degradation first fixes tile count to 1 and focuses on temporal sampling (targeting 2 FPS for videos), then tiling degradation dynamically adjusts the tile count from a set of possible configurations (12, 8, 6, 4, 2, 1) to maximize use of available context.

The training pipeline also includes a progressive mixed post-training schedule, where the model is sequentially exposed to context lengths of 32K, 64K, and 128K tokens. This gradual exposure avoids diluting focus on shorter contexts and allows the model to adapt systematically.

Results

Eagle 2.5-8B demonstrates strong performance across multiple video understanding benchmarks, outperforming similar-sized models and even surpassing larger models in some cases.

The model shows competitive performance across diverse image understanding benchmarks, demonstrating balanced capabilities across multimodal general perception and reasoning tasks.

Crucially, increasing long-context data under the training recipe does not harm and may slightly benefit short-context image benchmark performance, addressing a common concern that long-context training degrades short-context abilities.

Extensive image pre-training significantly enhances performance on short video benchmarks (MVBench) and a simpler long video benchmark (MLVU), but less so on a more challenging long video benchmark (Video-MME).

The information-first sampling strategy, particularly IAP, is crucial for maintaining performance on high-resolution image benchmarks (InfoVQA) and fine-grained video benchmarks (Perception-test).

Progressive mixed training (32K to 64K) outperforms direct 64K mixed training on video benchmarks, likely because it avoids diluting focus on shorter contexts and allows for a gradual learning process.

Significance

Eagle 2.5 presents a practical and effective framework for training generalist VLMs that can handle both long videos and high-resolution images without sacrificing performance on either. The key innovations—IAP, ADS, and progressive post-training—are model-agnostic and can be applied to other architectures. The release of Eagle-Video-110K dataset further supports research in long-video understanding. This work moves toward truly versatile multimodal AI systems that can process diverse real-world inputs, from hour-long surveillance footage to high-resolution medical scans, with a single unified model. Future work could extend these techniques to even longer contexts (e.g., >1 hour) and explore real-time streaming applications.