A growing number of observability engineers now believe that the reasoning ability of large language models is no longer the primary bottleneck in AI-assisted root cause analysis. Instead, the more difficult challenge sits in the pipeline that determines what data reaches the model.
For teams integrating LLMs into incident response, the practical implication is that effort spent preparing context may yield better results than simply upgrading to a larger model.
Two Approaches to AI RCA
Most AI root cause analysis efforts fall into one of two categories. Agent-based designs give the model tools and allow it to investigate on its own, choosing which telemetry to fetch as it reasons through the problem. Deterministic designs correlate signals upfront and hand the model a single prepared context.
Coroot's work reflects a broader industry shift toward the second camp. Dynatrace's Davis AI similarly relies on deterministic, topology-based causal analysis. It traverses a real-time dependency map to pinpoint a root cause rather than letting an LLM run in an open-ended agent loop.
The two approaches break down in different ways, making it difficult to determine whether a failed diagnosis resulted from weak reasoning or from a harness that fed the model the wrong evidence.
Separating Reasoning from Context
Recent research from observability vendor Coroot attempts to separate these two variables. Engineer Nikolay Sivko split RCA with an LLM into two distinct jobs: reasoning over the data in front of it, and the harness that decides what data reaches the model and in what shape.
Sivko argued that asking "can AI do RCA?" is the wrong question, because the two jobs need to be measured independently.
Coroot's pipeline correlates signals into findings and passes them to the model in one focused context with no agent loop. This setup allows a wrong answer to be blamed on the model rather than on missing evidence.
To test this, Sivko built one scenario: a Chaos Mesh NetworkChaos experiment that injected delay between a catalogue service and its Postgres database. This slowed queries and pushed the front-end into 502 errors. The context deliberately included misleading signals, including query timings inflated by network round-trip time.
He then ran the same prompt, around 9,800 tokens, against eleven models. Each model was asked for the root cause, the cause-and-effect chain, and the immediate fix.
Model Performance Results
Stay ahead of the AI curve
The most important updates, news, and content — delivered weekly.
No spam. Unsubscribe anytime.
Closed frontier models performed well. Claude Opus 4.8, GPT-5.5, and Gemini 3.1 Pro all passed, correctly naming the experiment and flagging the need to delete both it and its schedule.
Larger open-weight models mostly kept pace. Among smaller models, Gemma 4 31B was the only self-hostable model to identify the root cause. The larger Qwen3.6 35B and Qwen3 Coder Next both missed it.
The Debate Continues
The findings do not settle the broader debate. Agent-based approaches maintain a genuine advantage. A model that fetches its own data can pull signals that a fixed pipeline never anticipated, which matters for novel incidents outside a predefined correlation set.
However, agents pay for that flexibility in operability. Accounts from ZenML and Incident.io describe multi-agent LLM investigations as notoriously hard to debug in production. A failed run leaves no clean stack trace, only unpredictable prompt interactions and emergent coordination between agents.
That fragility pushes many practitioners in the other direction. Engineers discussing the trade-off on Reddit report scrapping fully agentic designs in favor of mostly deterministic workflows with a narrow LLM step, citing better reliability and lower token cost.
Deterministic pipelines trade some flexibility for repeatability and cleaner evaluation. This is much of why the industry increasingly treats the harness, not the model, as the hardest part of AI RCA.
Cost and Context Engineering
On cost, Sivko noted that a single short call runs only a few cents even on frontier models, because the correlation work happens before the model is called.
He judged that the reasoning part of AI RCA "is basically solved" and that the real work is now "preparing the right, compact context for the model before you call it." This conclusion points the next round of engineering effort at the harness rather than the model.
That framing echoes a wider push around context engineering. Guidance from Anthropic and LangChain, along with observability vendor Mezmo, converges on the same idea: curating the smallest set of high-signal, compact context is now a core discipline for making LLM-based reasoning and observability reliable.