Shipping Safer AI: A Developer's Guide to OpenAI Moderation and Production Safety

Why safety is non-negotiable

Deploying AI into production means accepting responsibility for the content and behavior your system generates. Safety isn’t just about meeting policy; it’s about protecting users, preserving trust, and avoiding legal or reputational consequences. When you design for safety, you not only reduce immediate risks like misinformation and offensive outputs, but you also build a foundation for sustainable innovation.

Core safety practices for production systems

Below are practical measures and tools developers should use to ship safer AI applications.

Moderation API overview

OpenAI provides a Moderation API to help detect potentially harmful content in text and images. The API flags categories such as harassment, hate, violence, sexual content, and self-harm—enabling developers to filter or block risky outputs before delivering them to users.

Supported models

• omni-moderation-latest: Recommended for most applications. Supports both text and image inputs, offers more nuanced categories, and provides broader detection capabilities.
• text-moderation-latest (Legacy): Text-only and supports fewer categories. Use omni for new deployments.

Use the moderation endpoint before publishing content. If the API flags material as risky, implement appropriate mitigation: filter the content, stop publication, or take action against offending accounts. The Moderation API is free and receives continuous updates to improve detection.

Example: moderating a text input with the official Python SDK

from openai import OpenAI
client = OpenAI()

response = client.moderations.create(
    model="omni-moderation-latest",
    input="...text to classify goes here...",
)

print(response)

The API returns a structured JSON that indicates whether the input is flagged, which categories are triggered, confidence scores per category, and (for omni models) which input types caused the flags.

Example response structure

{
  "id": "...",
  "model": "omni-moderation-latest",
  "results": [
    {
      "flagged": true,
      "categories": {
        "violence": true,
        "harassment": false,
        // other categories...
      },
      "category_scores": {
        "violence": 0.86,
        "harassment": 0.001,
        // other scores...
      },
      "category_applied_input_types": {
        "violence": ["image"],
        "harassment": [],
        // others...
      }
    }
  ]
}

The moderation tool helps you catch multiple categories, including harassment, hate, illicit content, self-harm, sexual content, and violence. The omni model’s multimodal capabilities extend detection across text and images.

Adversarial testing (red-teaming)

Adversarial testing intentionally probes your system with malicious, unexpected, or manipulative inputs to reveal weaknesses before real users encounter them. This process helps surface prompt injection attacks, jailbreak attempts, bias, toxicity, and data leakage. Red-teaming is ongoing: threats evolve, so testing should, too. Tools and frameworks such as deepeval can help structure these tests for chatbots, RAG pipelines, agents, and other LLM-based systems.

Human-in-the-loop (HITL)

For high-stakes domains like healthcare, finance, or legal work, human review of AI outputs is critical. Reviewers should have access to original source materials so they can verify claims and correct errors. HITL reduces risk and increases user confidence in the system’s reliability.

Prompt engineering

Careful prompt design reduces unsafe or irrelevant outputs. Provide context, example prompts, and explicit constraints to guide the model’s tone, domain, and permissible actions. Anticipate misuse scenarios and harden prompts against common manipulations to limit harmful behavior.

Input and output controls

Limit user input length to reduce prompt-injection risk and cap output tokens to control misuse and manage cost. Prefer validated inputs (dropdowns, selection lists) instead of free-text when possible. Where applicable, route queries to curated, pre-verified knowledge bases rather than generating new content from scratch to minimize hallucinations and harmful responses.

User identity and access

Requiring user sign-up and identity verification (email providers, OAuth, or stronger checks like card or ID verification when appropriate) adds accountability and deters anonymous abuse. Include hashed safety identifiers in API requests to help OpenAI trace misuse without exposing personal data. Example usage of a safety identifier in a chat completion:

from openai import OpenAI
client = OpenAI()

response = client.chat.completions.create(
  model="gpt-4o-mini",
  messages=[
    {"role": "user", "content": "This is a test"}
  ],
  max_tokens=5,
  safety_identifier="user_123456"
)

This practice enables more precise abuse detection and reduces the risk of penalizing entire organizations for the actions of a few users.

Transparency and feedback loops

Provide simple ways for users to report unsafe or unexpected outputs—visible buttons, a support email, or a ticket form. Ensure reports are triaged by humans who can investigate and respond. Communicate system limitations (hallucinations, bias) clearly to set expectations. Continuous monitoring in production helps detect regressions and emerging risks, enabling rapid iteration and mitigation.

How OpenAI assesses safety

OpenAI evaluates models and applications on multiple fronts: harmful content generation, resistance to adversarial attacks, clarity about limitations, and human oversight in critical workflows. With GPT-5, OpenAI added safety classifiers that categorize requests by risk. Organizations that repeatedly trigger high-risk thresholds may face limits on access to protect the broader ecosystem. Using safety identifiers helps OpenAI target interventions precisely.

OpenAI’s layered checks include blocking disallowed content (hate, illicit material), testing for jailbreak prompts, assessing factual accuracy to reduce hallucinations, and enforcing instruction hierarchy across system, developer, and user messages. This ongoing evaluation helps maintain safety standards while adapting to new threats and capabilities.

Making safety part of your development lifecycle

Safety isn’t a one-time checkbox. Embed moderation, adversarial testing, human review, and strict input/output controls into your CI/CD and monitoring pipelines. Treat safety as a continuous process of evaluation, refinement, and adaptation—this approach helps you meet policy requirements and deliver AI systems users can trust without sacrificing innovation.