French AI company Mistral has released Shieldstral, a 3-billion-parameter open-weight safety classifier that matches the performance of much larger models on text safety benchmarks and sets a new high score for joint text and image classification. The model, described in a paper released on Arxiv and by Mistral, uses a runtime-definable yes/no question format instead of fixed safety categories. That design choice lets operators write their own screening rules in plain language, such as "Does this content promote violence?" The model answers only "yes" or "no," and the system uses the probability of each response to calculate a safety score between zero and one.
Shieldstral returns one token per classification. That keeps it fast and cheap to run, which matters because safety classifiers sit on either side of the main language model, screening prompts before processing and responses before they reach users. Every request passes through the classifier, so its size, speed, and cost add up quickly.
A Small Model That Ties the Big Ones
The paper reports that Shieldstral matches models three times its size on standard text safety benchmarks. On combined text benchmarks, Shieldstral scored an F1 of 84.9 percent. That ties OpenAI's GPT-OSS-Safeguard-20B, a model roughly 20 billion parameters in size, which also scored 84.9 percent. F1 combines precision and recall into one metric, with 100 percent representing a perfect score.
Shieldstral beat several other competitors on the same combined text benchmarks. Qwen3Guard-8B, an 8-billion-parameter model, scored 84.0 percent. Nemotron-3.5-Safety-4B, a 4-billion-parameter model, scored 83.3 percent. LlamaGuard-4-12B, a 12-billion-parameter model, scored 69.1 percent.
The advantage grows on multimodal tasks. On image and image-text benchmarks, Shieldstral scored 83.8 percent. OmniGuard-7B, a 7-billion-parameter multimodal safety model, scored 77.6 percent. LlavaGuard-7B, also 7 billion parameters, scored 71.6 percent. Mistral says Shieldstral sets a new high score for joint text and image classification.
Yes/No Questions Replace Fixed Taxonomies
Many guardrail models sort content using fixed taxonomies. Public safety datasets group risks too differently to support one common taxonomy, and the same safety rules don't fit every use case. Content suitable for a cybersecurity tool could be harmful on a mental health platform. Shieldstral's approach sidesteps that problem.
Operators write review criteria in plain language at runtime. The model answers only "yes" or "no" to each question. This gives operators more control over the tradeoff between filter strictness and usability. They can write screening criteria at runtime and tailor the filter to a specific app instead of adopting someone else's categories.
The researchers combined about 54.1 million examples covering safety, harmful content, and manipulation attempts into one format. Strict standards were applied to targeted manipulation data. Moderate standards were applied to general safety data. Lenient standards were applied to response quality data. Each training example includes task instructions, a specific yes/no question, and the content being reviewed. The answer in training is a single token.
The team used another language model to rewrite safe text into unsafe variants. Each example included a similar but different category that had to be rejected. That trained the model to separate closely related rules.
Adaptability Comes From Synthetic Data
Stay ahead of the AI curve
The most important updates, news, and content — delivered weekly.
No spam. Unsubscribe anytime.
The adaptability test categories were created separately from the training set. Test categories used different names and levels of detail than training categories. None of the fine-grained test categories directly matches a training category. Ten of the 12 broader test classes have rough counterparts in training categories.
On the adaptability benchmark, GPT-OSS-Safeguard-20B scored 94.1 percent. Shieldstral scored 91.3 percent. The authors still consider Shieldstral more practical than GPT-OSS-Safeguard-20B and Nemotron-3.5-Safety despite lower adaptability scores, because those models generate long intermediate reasoning sequences that raise compute costs.
Synthetic category data was the main driver of the model's ability to adapt to new rules. In a validation test with fine-grained categories, synthetic category data raised the F1 score by 23.3 percentage points. That jump shows how much the generated examples contributed to the model's flexibility.
Shieldstral is based on Mistral's Ministral-3B with the Pixtral vision encoder. It is available as an open-weight model under the Apache 2.0 license. Operators can update safety rules without retraining the main model.
Classifiers Are Everywhere, and Getting It Wrong Hurts
Classifiers already play a growing role across the industry. OpenAI uses classifiers for automatic age detection in ChatGPT. OpenAI routes emotional requests through a safety filter to stricter models. Anthropic's Claude Code uses a classifier to block external scripts, production deployments, and force pushes.
Anthropic's Claude Fable 5 showed how poorly tuned filters can affect real use. The model had poorly tuned filters that automatically routed tasks to weaker models and flagged benign content. Independent analysis firm Artificial Analysis found that Claude Fable 5 routed 8-9 percent of tasks to a weaker model. One medical physicist called Claude Fable 5 unusable because his work often includes the word "nuclear." Other users reported that Claude Fable 5 flagged MRI analysis as bioterrorism.
Anthropic tightened the filter after locating a safety issue in Claude Fable 5. The company says it has since blocked harmless coding tasks more often. Anthropic's Fable 5 review requires the company to store inputs and outputs for up to 30 days, and storage extends to two years after rule violations.
Mistral co-founder Guillaume Lample is one of the authors of the research paper describing Shieldstral. The paper argues that the yes/no format gives operators more practical control than fixed categories, even when a larger model scores slightly higher on adaptability tests.
The result is a safety classifier that runs fast, costs little, and lets each operator decide what counts as harmful. For teams running high-volume AI services, that combination may matter more than a few percentage points on a benchmark.