Neura Market
OpenELM-1.1|3B logo

OpenELM-1.1|3B

Free

Efficient open language models by Apple with layer-wise scaling.

FreeFree tier
Inputs: textOutputs: text
Type
Open Source
Company
Apple
Visit Website GitHub (38 stars)

About OpenELM-1.1|3B

OpenELM is a family of open-source language models developed by Apple, designed with a layer-wise scaling strategy to efficiently allocate parameters within each transformer layer for enhanced accuracy. The models are pretrained on approximately 1.8 trillion tokens from a diverse corpus including RefinedWeb, deduplicated PILE, a subset of RedPajama, and a subset of Dolma v1.6, using the CoreNet library. OpenELM includes pretrained and instruction-tuned variants at 270M, 450M, 1.1B, and 3B parameters, all available on Hugging Face for easy integration via the Transformers library. The models demonstrate competitive zero-shot performance across benchmarks such as ARC-c, ARC-e, BoolQ, HellaSwag, PIQA, SciQ, and WinoGrande.

Key Features

Layer-wise scaling strategy for efficient parameter allocation
Pretrained and instruction-tuned variants: 270M, 450M, 1.1B, 3B parameters
Trained on ~1.8 trillion tokens from diverse datasets (RefinedWeb, PILE, RedPajama, Dolma v1.6)
Open training and inference framework using CoreNet library
Available on Hugging Face with simple integration via Transformers
Supports speculative decoding for faster inference

Pros & Cons

Pros
  • Open-source and freely accessible on Hugging Face
  • Layer-wise scaling improves parameter efficiency and accuracy
  • Multiple model sizes accommodate different compute budgets
  • Instruction-tuned versions available for better task alignment
  • Trained on a large, high-quality dataset mix
  • Backed by Apple's research team with published performance benchmarks
Cons
  • Newer model family with less community adoption compared to established open models like LLaMA
  • Larger variants (1.1B, 3B) require significant computational resources for inference and fine-tuning
  • Documentation and ecosystem beyond Hugging Face are limited

Best For

Natural language generation and understandingResearch in efficient language modelingFine-tuning for downstream NLP tasksEducational and prototyping applicationsOn-device deployment exploration for smaller variants

FAQ

What is OpenELM?
OpenELM (Open Efficient Language Models) is a family of open-source language models released by Apple, using a layer-wise scaling strategy to allocate parameters efficiently for improved accuracy.
What sizes are available?
OpenELM offers pretrained and instruction-tuned models with 270 million, 450 million, 1.1 billion, and 3 billion parameters.
What data was OpenELM trained on?
The models were pretrained on approximately 1.8 trillion tokens from RefinedWeb, deduplicated PILE, a subset of RedPajama, and a subset of Dolma v1.6.
How can I use OpenELM?
OpenELM models can be loaded via the Hugging Face Transformers library using AutoModelForCausalLM with trust_remote_code=True. Example code and generation scripts are provided on the model page.
Is OpenELM free to use?
Yes, OpenELM is open-source and freely available under the terms on Hugging Face. Users should review license agreements and terms of the training datasets.