Meta AI Launches LlamaFirewall: Open-Source Security for Autonomous AI Agents

Rising Security Risks in Autonomous AI Agents

As AI agents grow more autonomous and capable—writing production code, managing workflows, and handling untrusted data sources—their vulnerability to security threats increases. To tackle these challenges, Meta AI has introduced LlamaFirewall, an open-source security guardrail system designed to provide a robust system-level security layer for AI agents deployed in production environments.

Key Security Challenges Addressed

AI agents powered by large language models (LLMs) are integrated into high-privilege applications, such as reading emails, generating code, and making API calls. Existing safety measures like chatbot moderation and hardcoded constraints fall short for these advanced agents. LlamaFirewall targets three main security concerns:

• Prompt Injection Attacks: Malicious inputs crafted to manipulate agent behavior directly or indirectly.
• Agent Misalignment: When the agent's actions diverge from the user's intended goals.
• Insecure Code Generation: Risky or vulnerable code produced by LLM-based coding assistants.

Core Components of LlamaFirewall

LlamaFirewall consists of three specialized guardrails, each addressing specific risks:

1. PromptGuard 2 This BERT-based classifier detects jailbreaks and prompt injection attempts in real time, supporting multiple languages. It comes in two model sizes: an 86M parameter model for strong performance, and a lightweight 22M parameter version optimized for low-latency environments. It aims to identify high-confidence jailbreak attempts with minimal false positives.

2. AlignmentCheck An experimental auditing tool that assesses whether an agent’s actions align semantically with the user’s goals by analyzing the agent’s internal reasoning trace. Powered by large language models like Llama 4 Maverick, AlignmentCheck is effective at detecting indirect prompt injections and goal hijacking.

3. CodeShield A static code analysis engine that scans LLM-generated code for insecure patterns. Using Semgrep and regex rules, it supports syntax-aware analysis across multiple programming languages, helping developers catch common vulnerabilities such as SQL injection before code execution.

Performance and Evaluation

Meta tested LlamaFirewall with AgentDojo, a benchmark simulating prompt injection attacks across 97 task domains. Results showed:

• PromptGuard 2 (86M) reduced attack success rates (ASR) from 17.6% to 7.5% with minimal impact on task utility.
• AlignmentCheck lowered ASR further to 2.9%, albeit with slightly higher computational costs.
• Combined, the system achieved a 90% reduction in ASR, down to 1.75%, with a modest utility decrease to 42.7%.

CodeShield demonstrated 96% precision and 79% recall on a labeled dataset of insecure code completions, with response times suitable for real-time production use.

Future Developments

Meta plans to expand LlamaFirewall’s capabilities, including:

• Extending protection to multimodal agents handling image and audio inputs.
• Improving efficiency, particularly reducing AlignmentCheck latency via model distillation.
• Broadening threat coverage to include malicious tool use and dynamic behavior manipulation.
• Developing more comprehensive benchmarks for AI agent security in complex workflows.

LlamaFirewall represents a significant step toward modular, comprehensive security defenses for autonomous AI agents, combining pattern detection, semantic reasoning, and static code analysis to mitigate critical security risks in LLM-based systems.

For more information, check out the Paper, Code, and Project Page.