GitHub

Free

Natural Language-Based Societies of Mind for collaborative AI reasoning

FreeFree tier

Type

Open Source

Visit Website GitHub (40 stars)

About GitHub

Natural Language-Based Societies of Mind (NLSOM) is an open-source framework that implements Minsky's 'Society of Mind' concept by creating communities of agents—including LLMs, neural network experts, APIs, and role-players—that communicate via natural language. It uses a collaborative 'Mindstorm' process where agents interview each other to solve complex tasks, improving multimodal zero-shot reasoning beyond single LLMs. The framework supports modular addition of agents, automatic community recommendations based on user goals, and reward mechanisms to evaluate agent contributions. NLSOM received the Best Paper Award at NeurIPS 2023 Ro-FoMo Workshop and has been accepted by CVMJ 2025.

Key Features

Natural language communication among heterogeneous agents (LLMs, NN experts, APIs, role-players)

Collaborative 'Mindstorm' process with mutual interviews for task solving

Modular and extensible agent architecture for easy addition of new components

Automatic recommendation of relevant agent communities and tools based on user goals

Reward mechanisms to evaluate and reward agent contributions to tasks

Supports multimodal zero-shot reasoning by combining diverse models

Open-source implementation with community-driven development

Pros & Cons

Pros

Overcomes limitations of single LLMs by leveraging diverse agent capabilities
Modular and extensible design allows easy integration of new agents and tools
Inspired by established cognitive theories (Minsky, Schmidhuber) for sound conceptual foundation
Award-winning methodology (Best Paper Award at NeurIPS 2023 workshop)
Accepted by a peer-reviewed journal (CVMJ 2025), indicating academic validity

Cons

Early-stage implementation (position paper); may not be production-ready
Requires multiple LLM/NN experts, which can be computationally expensive
Documentation notes repo was completed months before release, suggesting potential development delays
Dependence on external APIs and models may introduce reliability and latency issues

Best For

Multimodal zero-shot reasoning tasks requiring diverse expertiseComplex problem-solving through collaboration of specialized AI agentsAutomated recommendation of AI plugins and models for non-technical usersResearch and experimentation in multi-agent systems inspired by cognitive scienceBenchmarking agent collaboration and reward mechanisms

FAQ

What does NLSOM stand for?

Natural Language-Based Societies of Mind.

How does NLSOM differ from systems like HuggingGPT?

Unlike HuggingGPT which relies on a single model for a specific function, NLSOM employs communities of agents sharing a common function (e.g., 'search') where each agent has unique strengths, and they collaborate via mutual interviews in a Mindstorm process.

What types of agents can be part of an NLSOM society?

Agents can be LLMs, neural network-based experts, APIs, or role-players, all communicating through natural language.

Is NLSOM a research project or production-ready?

NLSOM is primarily a research position paper and open-source prototype, accepted by CVMJ 2025 and awarded at NeurIPS 2023. It is intended for experimentation and further development.

Can non-technical users use NLSOM?

Yes, the system can automatically recommend relevant plugins, models, and role-players aligned with user goals, making it accessible to non-technical individuals.