Real-time streaming moderation for LLMs

Alibaba’s Qwen team has released Qwen3Guard, a family of multilingual guardrail models designed to moderate both prompts and streaming responses in real time. The release showcases two operating modes and multiple model sizes to meet different latency and accuracy needs for global deployments.

Two operating modes: Gen and Stream

Qwen3Guard ships in two variants. Qwen3Guard-Gen is a generative classifier that examines the full prompt and response context before producing structured safety outputs. Qwen3Guard-Stream is a token-level classifier that scores each token as it is generated, enabling policy enforcement while a reply is still being produced. Both variants are available in 0.6B, 4B, and 8B parameter sizes.

How streaming moderation works

The Stream variant attaches two lightweight classification heads to the final transformer layer: one monitors the user prompt, the other scores generated tokens in real time as Safe, Controversial, or Unsafe. This token-time scoring enables earlier intervention—blocking, redaction, or redirection—without waiting for a full response to be decoded and re-filtered.

Three-tier risk semantics

Beyond binary safe/unsafe labels, Qwen3Guard introduces a Controversial tier. This middle label supports adjustable strictness, allowing teams to tighten or loosen how borderline content is treated. The Controversial tag is useful for routing, escalation, or review workflows instead of outright dropping content.

Structured outputs for pipelines

The Gen variant emits a predictable header such as Safety: …, Categories: …, Refusal: … that is easy to parse by downstream pipelines or RL reward functions. Category labels include Violent, Non-violent Illegal Acts, Sexual Content, PII, Suicide & Self-Harm, Unethical Acts, Politically Sensitive Topics, Copyright Violation, and Jailbreak.

Benchmarks and safety-driven RL

Qwen’s research shows strong average F1 performance across English, Chinese, and multilingual safety benchmarks for both prompt and response classification, with consistent leads over prior open models. For training assistants, the team used Qwen3Guard-Gen as a reward signal in safety-driven RL. A Guard-only reward maximized safety but increased refusals and slightly reduced some competitive metrics. A Hybrid reward that penalizes over-refusal while blending quality signals raised measured safety from about 60 to over 97 without degrading reasoning abilities, and in some cases improved other task win rates.

Where Qwen3Guard fits in production

Most open guard models classify completed outputs. Qwen3Guard’s dual heads and token-time scoring are built to align with production agents that stream responses, enabling lower-latency intervention and more flexible policy controls. The Controversial tier maps well to enterprise policy knobs where regulated contexts may treat borderline content as unsafe while consumer contexts allow review.

Open weights and multilingual coverage

Qwen3Guard is open-sourced with weights available on Hugging Face and code on GitHub. The family covers 119 languages and dialects, and offers a practical baseline for teams looking to replace post-hoc filters with real-time moderation and to integrate safety-aware reward shaping in RL training.

For more details, see the Qwen3Guard GitHub repository and Hugging Face collection linked in the original announcement.