Neura Market
OPT logo

OPT

Free

Open Pretrained Transformers (OPT) by Facebook is a suite of decoder-only pre-trained transformers. [Announcement](https://ai.facebook.com/blog/democratizing-access-to-large-scale-language-models-with-opt-175b/). [OPT-175B text generation](https://opt.alpa.ai/) hosted by Alpa.

FreeFree tier
Inputs: textOutputs: text
Type
Open Source
Company
Meta AI
Visit Website GitHub (40 stars)

About OPT

OPT (Open Pretrained Transformers) is a suite of decoder-only pre-trained transformer language models introduced by Meta AI. The models range from 125 million to 175 billion parameters and are designed to democratize access to large language models for researchers. The 350M parameter variant (facebook/opt-350m) is available on Hugging Face, compatible with the Transformers library, and supports causal language modeling for text generation tasks. The models were trained to roughly match the performance of the GPT-3 class while being fully and responsibly shared with the research community.

Key Features

Decoder-only transformer architecture
Sizes from 125M to 175B parameters
Open-sourced for research and reproducibility
Compatible with Hugging Face Transformers and PyTorch/TensorFlow/JAX
Designed to match GPT-3 class performance
Supports zero-shot and few-shot learning

Pros & Cons

Pros
  • Fully open and accessible to researchers
  • Multiple model sizes for different computational budgets
  • Strong performance comparable to GPT-3
  • Actively maintained and integrated with Hugging Face ecosystem
  • Enables reproducibility and deeper understanding of LLM behavior
Cons
  • May exhibit biases and toxicity as noted in the original paper
  • Limited to English language (tagged 'en')
  • Large variants (e.g., 175B) require substantial compute resources

Best For

Text generation and completionZero-shot and few-shot learningResearch on large language model robustness, bias, and toxicityAcademic study of emergent capabilities in LLMs

FAQ

What is OPT-350M?
OPT-350M is a 350 million parameter decoder-only transformer language model from the OPT suite, released by Meta AI and hosted on Hugging Face.
Who created OPT?
OPT was created by Meta AI (formerly Facebook) and introduced in the paper 'Open Pre-trained Transformer Language Models' (arxiv:2205.01068).
How does OPT compare to GPT-3?
The OPT models were trained to roughly match the performance and sizes of the GPT-3 class of models, offering comparable capabilities with open access.