A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
Zhenting Wang, Qi Chang, Hemani Patel, et al.
Mcp-bench introduces a benchmark for evaluating tool-using LLM agents on complex real-world tasks via MCP servers, providing a standardized framework for assessing agent capabilities.
Unknown
This paper argues that effective real-world LLM agents require optimizing productivity, proactivity, and personalization, and proposes a training approach to achieve this.
Unknown
This paper proposes a multi-agent evaluation framework that aligns LLM-agent-based automated evaluation with multi-dimensional human evaluation for real-world text generation.
Unknown
Wildclawbench introduces a benchmark for evaluating frontier AI models on long-horizon, native-runtime agent tasks, showing that current models still struggle with such evaluations.
Unknown
This paper proposes a holistic agent leaderboard to standardize AI agent evaluation and address common pitfalls, shifting focus from benchmark-acing to real-world utility.
Unknown
This paper re-examines model editing under realistic deployment conditions, arguing that current evaluation methods overestimate real-world performance.
Unknown
This paper critically evaluates speculative decoding's real-world effectiveness, revealing that prior benchmarks overstate its speedups due to idealized conditions.
Kento Kawaharazuka, T. Matsushima, Andrew Gambardella, et al.
This paper reviews the application of foundation models in real-world robotics, categorizing approaches and discussing challenges and future directions.
Unknown
This paper introduces a method for learning symbolic world models from real-world observations to enable long-horizon planning in unseen environments.
Unknown
OSWorld-MCP is the first comprehensive and fair benchmark for assessing computer-use agents' tool invocation, GUI operation, and decision-making abilities in real-world scenarios.
Unknown
This paper provides a comprehensive guide to building agentic AI systems, covering frameworks, prompt engineering, and multi-agent orchestration for real-world applications.
Unknown
Introduces Vitabench, a benchmark for evaluating LLM agents on versatile interactive tasks in real-world applications, capturing complexity of handling extensive information.