StableLM Zephyr 3B

StableLM Zephyr 3B is a lightweight large language model (LLM) developed by Stability AI, designed to provide accurate and responsive text generation on a wide range of devices without requiring high-end hardware. With 3 billion parameters, it is approximately 60% smaller than common 7B models, making it suitable for edge devices and environments with limited computational resources. The model is built upon the Stable LM 3B-4e1t base and was inspired by Hugging Face's Zephyr 7B. Its training involved supervised fine-tuning on multiple instruction datasets—including UltraChat, MetaMathQA, Evol Wizard Dataset, and Capybara Dataset—followed by alignment with human preferences using the Direct Preference Optimization (DPO) algorithm on the UltraFeedback dataset. According to Stability AI, StableLM Zephyr 3B demonstrates competitive performance against larger models such as WizardLM-13B-v1 and Llama-2-70b-chat on benchmarks like MT-Bench and AlpacaEval, emphasizing its efficiency in instruction following and question-answering tasks.