A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
Yung-Sung Chuang, Linlu Qiu, Cheng-Yu Hsieh, et al.
Proposes Lookback Lens, a simple hallucination detector using attention weight ratios, effective across tasks and models, and reduces hallucinations via classifier-guided decoding.
Romain Cosentino, Sarath Shekkizhar
This paper links LLM reasoning ability to the density of self-attention graphs, showing higher intrinsic dimension in MLP inputs implies greater expressive capacity.
Rachit Bansal, Aston Zhang, Rishabh Tiwari, et al.
Proposes test-time training via targeted gradient updates on context to overcome static self-attention limitations in long-context LLMs, outperforming inference-time scaling.
S. Mostafa Mousavi, William L. Ellsworth, Weiqiang Zhu, et al.
A global deep-learning model using hierarchical attention for simultaneous earthquake detection and phase picking, outperforming prior methods and detecting twice as many events with fewer stations.
Zifan Zheng, Yezhaohui Wang, Yuxin Huang, et al.
This review demystifies LLM reasoning by categorizing attention head functions within a four-stage framework inspired by human thought.
Shantanu Acharya, Fei Jia, Boris Ginsburg
Star Attention enables efficient LLM inference over long sequences via a two-phase block-sparse approximation that reduces memory and time by up to 11x while preserving 97-100% accuracy.
Weigao Sun, Jiaxi Hu, Yucheng Zhou, et al.
A systematic survey of efficient LLM architectures addressing transformer limitations, covering linear/sparse models, efficient attention, MoE, hybrids, and diffusion LLMs.
Unknown
ERNIE-Layout enhances document understanding by reorganizing tokens using layout knowledge and applying spatial-aware disentangled attention in a multi-modal transformer.
Unknown
Introduces Bi-directional attention complementation mechanism (BiACM) for cross-modal interaction of text and layout in document AI.
Unknown
DocFormer introduces an encoder-only transformer with a CNN backbone that fuses visual, textual, and spatial features via a novel multi-modal self-attention layer for document understanding.
Unknown
MobileNetV4 introduces a universally efficient architecture with the UIB block, Mobile MQA attention, and an optimized NAS recipe for high accuracy and efficiency across mobile devices.
Unknown
Introduced the Transformer model with multihead attention for language translation, replacing recurrent architectures.