TAT-QA

Free

a large-scale question-answering benchmark focused on real-world financial data, integrating both tabular and textual information.

FreeFree tier

Inputs: textOutputs: text

Type

Open Source

Visit Website GitHub (38 stars)

About TAT-QA

TAT-QA (Tabular And Textual dataset for Question Answering) is a large-scale benchmark designed for question answering over hybrid contexts combining semi-structured tables and natural language paragraphs, specifically in the financial domain. The dataset contains 16,552 questions derived from 2,757 real-world financial report contexts, each comprising a table and at least two related paragraphs. Questions are human-generated by annotators with financial expertise and require diverse numerical reasoning capabilities such as addition, subtraction, multiplication, division, counting, comparison, sorting, and their compositions. Answer formats include single spans, multiple spans, and free-form text. The dataset also provides ground-truth derivations and scales (e.g., million, percent) for transparency. TAT-QA aims to advance research in complex, realistic QA and numerical reasoning over structured and unstructured data. A leaderboard tracks model performance using Exact Match and F1 scores, with top models like TAT-LLM achieving 81.4% EM. The dataset is freely available for download and is accompanied by a published ACL 2021 paper.

Key Features

Hybrid context comprising a semi-structured table and at least two relevant paragraphs

Human-generated questions by annotators with rich financial knowledge

Diverse answer forms: single span, multiple spans, free-form

Requires numerical reasoning: addition, subtraction, multiplication, division, counting, comparison, sorting, and compositions

Ground-truth derivations and scale provided for each answer

Large scale: 16,552 questions associated with 2,757 hybrid contexts from real-world financial reports

Actively maintained leaderboard with model performance metrics

Pros & Cons

Pros

Real-world financial data ensures practical relevance and authenticity
Hybrid context (table + text) mimics realistic real-world scenarios
Rich annotations including derivations and scale facilitate interpretable evaluation
Large dataset with diverse question types supports robust model training and evaluation
Actively maintained leaderboard enables fair comparison of models

Cons

Limited to English-language financial reports, reducing cross-lingual applicability
Requires understanding of financial terminology and domain knowledge
Only provides dataset and benchmarks, not pre-trained models or inference tools

Best For

Research on numerical reasoning over tabular and textual dataFinancial document question answering and analysisBenchmarking QA models on hybrid contextsDeveloping systems for automated financial report interpretationEvaluating model capabilities in complex reasoning with real-world data

FAQ

What does TAT-QA stand for?

TAT-QA stands for Tabular And Textual dataset for Question Answering.

How many questions and contexts are in TAT-QA?

TAT-QA contains 16,552 questions associated with 2,757 hybrid contexts from real-world financial reports.

What types of reasoning are required to answer TAT-QA questions?

Questions require numerical reasoning including addition, subtraction, multiplication, division, counting, comparison, sorting, and compositions of these operations.

Is the TAT-QA dataset freely available?

Yes, the dataset is freely available for download from the official website in JSON format.

How are answers annotated in TAT-QA?

Answers include the ground-truth value, a derivation showing the reasoning steps, the answer type (span, spans, arithmetic, counting), the source (table, text, or both), and the scale (e.g., million, percent) if applicable.