Hermes 3

Analysis

Why This Paper Matters

Large language models today often come with built-in guardrails that refuse to follow certain instructions on moral or ethical grounds. While safety is important, this can limit the model's utility in scenarios where neutrality is required—such as role-playing, creative writing, or unbiased information retrieval. Hermes 3 directly addresses this by fine-tuning Llama 3.1 models to follow prompts exactly and neutrally, without injecting personal opinions or moral judgments. This makes them highly sensitive to the system prompt, allowing developers to define the model's persona precisely. The paper also introduces structured reasoning capabilities and a standardized tool-use format, pushing the boundaries of what open-source models can do in agentic workflows.

How It Works

Hermes 3 is built by fine-tuning Llama 3.1 at three scales: 8B, 70B, and 405B. The training data strongly encourages the model to follow system and instruction prompts exactly, without deviation. This neutral alignment is a key differentiator from commercial models that may refuse instructions on moral grounds. The effect is most pronounced in the 405B version, where an empty system prompt does not automatically elicit a "helpful assistant" persona.

The model incorporates several features to improve step-by-step reasoning: XML tags for structured output, scratchpads for intermediate processing, internal monologues for transparent decision-making, Mermaid diagrams for visual communication, and step-labeled reasoning and planning. These are enabled by training on special tokens like <SCRATCHPAD>, <REASONING>, <INNER_MONOLOGUE>, <PLAN>, <EXECUTION>, <REFLECTION>, <THINKING>, <SOLUTION>, <EXPLANATION>, and <UNIT_TEST>.

Tool use and retrieval augmented generation (RAG) are handled via the Hermes Function Calling standard. Tool definitions are placed as JSON schemas in <tools> tags, invocations and responses in <tool_call> and <tool_response> tags, and RAG sources are cited using the <co> tag.

The dataset comprises ~390M tokens, with 69% being output tokens. It includes general instructions from existing sources (filtered for coherence and educational value) and domain-specific data generated via Evol-Instruct-inspired schemes. Filtering steps remove refusals, improperly formatted responses, and conversations with missing turns.

The training recipe has two phases. In supervised fine-tuning, multiple samples are packed into a single sequence using Flash Attention 2 to avoid cross-attention contamination, achieving 96% packing efficiency. The target sequence length is 8192 tokens. In the DPO phase, a LoRA adapter (r=32, α=16, dropout=0.05) is applied to all linear layers. DPO provides moderate gains for the 8B model but negligible improvements for larger sizes, so the final models are SFT checkpoints.

Results

For the 405B model, evaluations are performed under FP8 quantization. The paper reports that DPO offers moderate positive impact on benchmarks for the 8B model, but for larger model sizes the improvements are negligible. Therefore, the final released models are from the SFT phase. The models are available on HuggingFace.

Significance

Hermes 3 represents a significant step toward more controllable and neutral AI assistants. By prioritizing exact instruction following over built-in moral judgments, it enables developers to craft custom personas and agentic behaviors without unexpected refusals. The structured reasoning and tool-use capabilities make it suitable for complex, multi-step tasks in production environments. However, the lack of extensive benchmark comparisons and the marginal benefit of DPO for larger models suggest that further research is needed to optimize alignment techniques at scale. Overall, Hermes 3 provides a valuable open-source alternative for practitioners who need unbiased, instruction-following models.