A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, et al.
Introduced the Transformer architecture, replacing recurrence and convolutions with self-attention mechanisms, achieving state-of-the-art results on machine translation and becoming the foundation for all modern large language models.
Jakubův, Jan, Chvalovský, Karel, Goertzel, Zarathustra, Kaliszyk, Cezary, et al.
This paper presents an AI/TP system that automatically proves about 60% of Mizar theorems in hammer setting and 75% with human-written premise hints.
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, et al.
SciPy 1.0 is a milestone release of the open-source scientific computing library for Python, consolidating over 600 contributors and millions of users.
Unknown
Develops a solvable model showing how feature learning improves neural scaling laws beyond the kernel limit.
Unknown
This paper proposes a solvable model that explains the empirical neural scaling laws observed in large language models.
Unknown
Proposes a smoothly broken power law functional form (Broken Neural Scaling Law) to accurately model and extrapolate scaling behaviors in deep learning.
Unknown
This paper revisits neural scaling laws for language and vision models, finding that scaling trends hold across modalities but with different exponents and saturation points.
Unknown
This paper demonstrates that data pruning can improve neural scaling laws, reducing the resource costs of deep learning.
Unknown
This paper introduces a dynamical model that reproduces key observations about neural scaling laws, explaining why performance scales with training time and model size.
Unknown
This paper explains neural scaling laws by deriving them from realistic data assumptions within learning theory.
Yizhao Gao, Shu-Yu Guo, Shijie Cao, et al.
SeerAttention-R adapts sparse attention for long-context reasoning, achieving strong AIME benchmark results with 4K token budgets and large sparse blocks.
Unknown
Double Sparsity reduces KV cache access in LLMs via post-training sparse attention combining token and channel sparsity.
Unknown
Proposes Dynamic Sparse Attention (DSA) to efficiently exploit dynamic sparse patterns in attention for scalable transformer acceleration.
Unknown
Seerattention introduces a block-sparse attention kernel that achieves a 7.3× speedup at 90% sparsity on 128k sequences, enabling efficient long-context LLMs.
Unknown
This paper introduces a novel sparse attention mechanism that improves efficiency and performance for long-range transformers.