Enhance AI Agent Performance: Deep Dive into Evals and Error Analysis (Part 2)

Discover practical strategies to debug and optimize agentic AI systems through systematic error analysis. This part 2 explores a real-world code agent case study, revealing common pitfalls and fixes to boost reliability.

Why Error Analysis is Crucial for Agentic Systems

Building AI agents that tackle complex, multi-step tasks—like writing code, conducting research, or automating workflows—sounds exciting, but it's fraught with challenges. These agents aren't just simple chatbots; they plan, use tools, execute actions, and iterate, often in non-deterministic ways. That's why basic evals fall short. In part 1, we covered setting up robust evals. Now, let's zoom into error analysis: the detective work that uncovers why your agent fails and how to fix it.

Imagine deploying an agent to generate Python code from natural language descriptions. It works great on easy tasks but bombs on trickier ones. Without dissecting failures, you're shooting in the dark. Error analysis gives you a structured lens to categorize mistakes, spot patterns, and iterate effectively. It's not just about metrics; it's about understanding the 'why' behind poor performance.

A Framework for Diagnosing Agent Failures

To make this actionable, we need a taxonomy of errors tailored to agentic workflows. Common categories include:

• Hallucinations: The agent invents facts or code that doesn't exist.
• Planning Errors: Flawed strategies or incomplete task decomposition.
• Tool Selection Issues: Picking the wrong tool or misusing it.
• Execution Failures: Code or actions that crash or produce wrong outputs.
• Iteration Problems: Failing to recover from errors or loop indefinitely.

This framework isn't rigid—adapt it to your domain. The goal? Quantify error types across your eval dataset to prioritize fixes. For instance, if 40% of failures stem from planning, that's your hot spot.

Let's apply this in a real case study: an agent that translates user specs into working Python scripts. We'll walk through its design, evals, failure modes, and upgrades.

Case Study: Building and Debugging a Code Generation Agent

The Agent's Workflow

Our agent uses a ReAct-style loop: Reason, Act, Observe. It starts with a user prompt like "Write a function to compute Fibonacci numbers iteratively." Here's the high-level setup:

1. Planner: Breaks the task into steps (e.g., define function signature, implement logic, add tests).
2. Executor: Writes and runs code via a Python interpreter tool.
3. Critic: Reviews outputs and decides next actions.

We implemented this using the Pasha SDK, with notebooks like agentic_search.ipynb showcasing similar patterns. Models like GPT-4o or Claude 3.5 Sonnet power the reasoning.

# Simplified agent loop pseudocode
while not done:
    observation = env.observe()
    action = model.reason("Given history and observation: " + observation)
    if action.type == 'tool':
        result = execute_tool(action.tool, action.args)
        history.append(result)
    elif action.type == 'finish':
        return action.output

Setting Up Evals

We curated 50 diverse prompts, from simple loops to data processing pipelines. Success metric: Does the final code pass all unit tests? (Automated via pytest). Run 5 trials per prompt to account for stochasticity—average pass@5 > 80% is our target.

Initial results? Only 60% success. Time to analyze.

Uncovering Failure Patterns

We logged full traces (prompts, actions, observations) and manually labeled 100 failing traces. Here's what emerged:

1. Hallucinations (25% of failures)

The agent fabricated non-existent libraries or syntax. Example prompt: "Parse JSON and sort by date."

Failure Trace:

• Agent writes: import jsonz (hallucinated lib).
• Execution: ModuleNotFoundError.

Root Cause: Weak instruction-following on standard libs.

Fix: Add RAG with a vector store of Python docs. Prompt: "Only use built-in libs unless specified. Consult docs: [retrieved snippet]." Boosted accuracy by 15%.

2. Planning Deficiencies (30%)

Agents jumped straight to coding without outlining steps. For "Implement a web scraper:",

Failure: Wrote full script without edge cases (e.g., no error handling for 404s).

Fix: Enforce structured planning:

**Plan:**
1. Import libs
2. Fetch URL
3. Parse HTML
4. Extract data
5. Handle errors
6. Test

Use chain-of-thought priming: "First, list 5-7 steps. Then execute one by one."

3. Tool Misuse (20%)

Picked interpreter but passed invalid args, like unescaped strings.

Example:

exec("print('Hello")  # Missing quote

Fix: Sandboxed executor with input validation. Better tool schemas in prompts: "Args must be valid Python literals."

4. Execution Crashes (15%)

Infinite loops or timeouts, e.g., recursive Fib without memoization.

Fix: Timeouts (30s per exec), loop detectors in observations.

5. Poor Iteration (10%)

After errors, agent repeated mistakes instead of debugging.

Fix: Add self-reflection: "Analyze the error: [stack trace]. What change?"

Quantitative Impact

Post-fixes, pass@5 jumped to 85%. Error distribution shifted: planning down to 10%, hallucinations to 5%.

Scaling Error Analysis with Automation

Manual labeling doesn't scale. Enter LLM judges! Tools like FastChat's LLM Judge classify traces automatically.

Prompt for Judge:

Classify this trace into: hallucination, planning_error, etc.
Justification: ...
Category: [one word]

Agreement with humans: 85%+. Run on 1,000 traces to find rare bugs.

Pro Tip: Version your agent configs in Git. Track metrics over commits.

Advanced Techniques for Robust Agents

• Dynamic Toolkits: Let agents request new tools mid-task.
• Multi-Agent Debate: Pit two agents against each other for better plans.
• Synthetic Data: Generate edge cases with GPT-4: "Create 20 failure-prone code prompts."

Real-world app: At a startup, we used this for ETL agents—cut debugging time 50%.

Key Takeaways and Next Steps

Error analysis turns agent dev from art to science. Start with your evals dataset, label failures, iterate on top issues. Check the Pasha SDK examples for ready code.

This isn't one-and-done—agents evolve, so re-eval regularly. Part 3? Scaling to production.

(Word count: ~1050)