Mixtral-8x7B
FreeHigh-quality open-weight sparse mixture of experts model
FreeFree tier
Inputs: textOutputs: text
About Mixtral-8x7B
Mixtral 8x7B is a high-quality sparse mixture of experts model (SMoE) developed by Mistral AI. It has 46.7B total parameters but only uses 12.9B per token, offering fast inference (6x faster than Llama 2 70B). It outperforms Llama 2 70B on most benchmarks and matches or outperforms GPT3.5 on standard benchmarks. The model supports a context window of 32k tokens and is multilingual (English, French, Italian, German, Spanish). It shows strong performance in code generation and can be fine-tuned into an instruction-following model that achieves a score of 8.3 on MT-Bench. Released under the Apache 2.0 license with open weights.
Key Features
Sparse Mixture of Experts architecture (8 experts, 2 active per token)
46.7B total parameters, only 12.9B used per token
6x faster inference than Llama 2 70B
Context window of 32k tokens
Multilingual: English, French, Italian, German, Spanish
Strong code generation capabilities
Can be fine-tuned for instruction following (MT-Bench 8.3)
Open weights under Apache 2.0 license
Pros & Cons
Pros
- Outperforms Llama 2 70B on most benchmarks
- Matches or outperforms GPT3.5 on standard benchmarks
- Fast inference due to sparse activation
- Open weights with permissive Apache 2.0 license
- Supports 32k token context
- Multilingual out of the box
Cons
- Requires significant computational resources for deployment (46.7B total parameters)
- Base model may need fine-tuning for specific tasks or instruction following (Instruct version available)
Best For
General language modeling and generationCode generation and understandingMultilingual tasks across 5 languagesBuilding instruction-following chatbots via fine-tuning
FAQ
What is Mixtral 8x7B?
Mixtral 8x7B is a high-quality sparse mixture of experts model (SMoE) from Mistral AI, released with open weights under Apache 2.0.
How does Mixtral compare to Llama 2 70B?
Mixtral outperforms Llama 2 70B on most benchmarks while being 6x faster during inference.
What license is Mixtral released under?
Mixtral is licensed under Apache 2.0.
What languages does Mixtral support?
Mixtral supports English, French, Italian, German, and Spanish.
What is the model size of Mixtral?
Mixtral has 46.7B total parameters but only uses 12.9B per token.
Is there an instruction-tuned version of Mixtral?
Yes, Mistral AI also released Mixtral 8x7B Instruct, optimized via supervised fine-tuning and DPO, achieving MT-Bench score of 8.30.