Σ-Mem: An Online Reliability Memory for LLM-based Multi-Agent Systems

Analysis

Why This Paper Matters

This paper addresses a critical gap in LLM-based multi-agent systems: existing memory systems focus on storing interaction content but fail to model which agents can be trusted and under what conditions. In multi-agent settings, a central model often cannot directly verify peer responses, especially when they are plausible or correlated. Σ-Mem introduces a reliability memory that records historical competence evidence, enabling the system to adaptively coordinate based on proven reliability.

The significance lies in its online, stable adaptation mechanism. By leveraging Weyl's inequality, the spectral change from each update is bounded, ensuring stability without retraining underlying models. This makes Σ-Mem practical for dynamic environments where agent reliability shifts over time. The ability to generalize to unseen peers and domains further enhances its utility, making it a foundational component for future multi-agent systems.

Technical Contributions

• Reliability Memory Design: Σ-Mem maintains real symmetric state matrices for individual peer competence and peer relationships, updated from post-decision correctness feedback. This provides a structured way to accumulate evidence over time.
• Stable Online Updates: The use of Weyl's inequality guarantees that each event-level update causes a bounded spectral change, ensuring stable adaptation without catastrophic forgetting or instability.
• General Interface: The memory supports multiple downstream tasks—residual steering of a central model, response-free peer routing, and reliability-weighted voting—demonstrating its versatility.
• Generalization: The memory generalizes to unseen peers and task domains, indicating that it learns transferable reliability patterns.

Results

The paper reports experiments across five Qwen-family models. Key results include:

• Σ-Mem adapts to counterfactual reliability shifts, showing it can track changes in agent trustworthiness.
• Direct memory readouts outperform majority voting and the best fixed peer on the full out-of-distribution (OOD) evaluation set.
• Performance improves consistently as more correctness feedback becomes available, indicating progressive accumulation of actionable reliability information.

These results highlight the practical benefits of reliability memory in improving coordination accuracy and robustness.

Significance

Σ-Mem introduces a new dimension to memory in LLM-based multi-agent systems: reliability. This moves beyond content recall to trust modeling, which is crucial for real-world deployments where agents may have varying competence. The stable online adaptation and generalization capabilities make it a scalable solution. This work could inspire further research into reliability-aware coordination, potentially leading to more trustworthy and effective collaborative AI systems.