Neura Market
RedPajama logo

RedPajama

Paid

Open-source LLMs for everyone: 3B and 7B models with strong performance.

New AI ToolsContact
Inputs: text, codeOutputs: text
Type
Saas
Company
Together AI
Visit Website
RedPajama screenshot

About RedPajama

RedPajama is a family of open-source large language models released by Together Computer, including base, instruction-tuned, and chat variants in 3-billion and 7-billion parameter sizes. These models are trained on the RedPajama base dataset, a 5-terabyte collection inspired by the LLaMA training recipe, and are designed to offer strong performance while remaining accessible to the open-source community. The 3B model is notably fast and can run on consumer GPUs such as the RTX 2070, while the 7B model demonstrates competitive scores on HELM benchmarks, outperforming similar-sized models like Pythia-7B. The instruction-tuned versions are optimized for downstream tasks including few-shot learning, entity extraction, classification, and summarization, and the chat models enable dialogue interactions.

RedPajama is part of the broader Together platform, which provides an inference engine, GPU clusters (including on-demand B200s), and optimized hosting for various open-source and commercial models. Together's infrastructure delivers high tokens-per-second throughput, making it suitable for production workloads such as coding agents. The RedPajama models can be accessed locally via open-source weights or through Together's hosted APIs, with the pricing model being contact-based rather than publicly listed.

The project emphasizes transparency and reproducibility, aiming to deepen understanding of what factors drive model performance. By releasing both the dataset and trained models, RedPajama enables researchers and developers to build upon open-source foundations. However, potential users should note that the 7B base model was released at approximately 80% of its planned training, and the exact availability of free-tier inference through Together's services should be verified directly.

Key Features

Open-source base, instruction-tuned, and chat models in 3B and 7B sizes
Trained on the 5TB RedPajama base dataset, replicating the LLaMA recipe
3B model runs on consumer GPUs (e.g., RTX 2070) and is accessible to a wide audience
Strong HELM benchmark performance, with 7B model outperforming LLaMA 7B by 3 points
Optimized for few-shot learning, entity extraction, classification, and summarization
Available through Together's high-performance inference engine with competitive throughput

Pros & Cons

Pros
  • Full open-source release of weights and dataset promotes transparency and community development
  • 3B model offers strong performance per parameter and runs on consumer hardware
  • Instruction-tuned versions provide good out-of-the-box behavior for common NLP tasks
  • Part of Together's broader platform offering scalable inference and GPU resources
  • Active development and community support via Discord and Twitter
Cons
  • Pricing for hosted inference is contact-based and not publicly transparent
  • Free-tier limits for Together's API should be verified; low-cost access may be uncertain
  • 7B base model was released partially trained (approximately 80%), so full potential may not be realized
  • Performance may vary across different downstream tasks; careful prompt engineering is recommended
  • Availability of models on other platforms or through self-hosting requires technical setup

Best For

Building chat applications with the instruction-tuned or chat variantsPerforming few-shot text classification and entity extraction tasksSummarizing long documents or generating structured outputsPowering coding agent workloads when hosted on Together's infrastructureExperimenting with open-source LLMs for research and prototypingDeploying lightweight language models on local hardware for privacy-sensitive applications

Alternatives to RedPajama

Vector Magic

Convert JPG, PNG images to SVG, EPS, AI vectors

Plask AI

Plask.ai uses AI motion capture to animate characters from video, simplifying 3D animation workflows for creators and studios.

PlugSugar

Automate conversations, answer questions with Web Search plugin, and customize ChatGPT experience using powerful AI plugins.

Apploi

Simplify recruitment, improve job seeker visibility, and help employers find the perfect candidates efficiently and affordably.

MedARC

Monitor patient progress, streamline processes and generate insights for healthcare providers to improve patient care.

SumUp

Experience Unmatched Sound Quality with AmbienceSoundPro!

FAQ

What model sizes are available in the RedPajama family?
RedPajama-INCITE includes 3 billion and 7 billion parameter versions, each with base, instruction-tuned, and chat variants.
Can I run the 3B model on a standard consumer GPU?
Yes, based on available information the 3B model runs on a GPU with at least 8GB VRAM, such as an RTX 2070, making it accessible for local inference.
How do the RedPajama models compare to LLaMA?
The models are designed to replicate the LLaMA recipe. On HELM benchmarks, the 7B model reportedly outperforms the original LLaMA 7B by 3 percentage points, though exact comparisons should be verified.
Is the RedPajama dataset freely available?
Yes, the RedPajama base dataset (5 terabytes) has been released publicly and has been used to train other open models like MPT and OpenLLaMA.
How can I access the hosted version of RedPajama models?
The models are served through Together Computer's platform. Pricing is not publicly listed; interested users should contact Together for details on API access and costs.
Are there any limitations regarding free usage?
Free-tier availability for hosted inference is not clearly stated. Users should check Together's current policies or consider self-hosting the open-source models to avoid usage fees.