Meta AI Unveils Matrix: A Decentralized Data Generation Framework

Matrix: Decentralized Control for Synthetic Data

How do you keep synthetic data fresh and diverse for modern AI models without turning a single orchestration pipeline into the bottleneck? Meta AI researchers introduce Matrix, a decentralized framework where both control and data flow are serialized into messages that move through distributed queues. As LLM training increasingly relies on synthetic conversations, tool traces, and reasoning chains, most existing systems still depend on a central controller or domain-specific setups. This wastes GPU capacity, adds coordination overhead, and limits data diversity. Matrix instead uses peer-to-peer agent scheduling on a Ray cluster and delivers 2 to 15 times higher token throughput on real workloads while maintaining comparable quality.

Decentralized Architecture: From Controllers to Agents

Traditional agent frameworks keep workflow state and control logic inside a central orchestrator. Every agent call, tool call, and retry goes through that controller. This model is easy to reason about, but it does not scale well when you need tens of thousands of concurrent synthetic dialogues or tool trajectories.

Matrix takes a different approach. It serializes both control flow and data flow into a message object called an orchestrator. The orchestrator holds the task state, including conversation history, intermediate results, and routing logic. Stateless agents, implemented as Ray actors, pull an orchestrator from a distributed queue, apply their role-specific logic, update the state, and then send it directly to the next agent selected by the orchestrator. There is no central scheduler in the inner loop. Each task advances independently at the row level, reducing idle time and improving fault handling.

System Stack and Services

Matrix runs on a Ray cluster typically launched on SLURM. Ray provides distributed actors and queues. Ray Serve exposes LLM endpoints and can route to external APIs. Tool calls run inside Apptainer containers for isolation, while Hydra manages configuration. Grafana integrates with Ray metrics to track performance in real-time.

Matrix also introduces message offloading, where conversation histories larger than a set threshold are stored in Ray’s object store, reducing bandwidth and allowing high-throughput LLM serving.

Case Studies

1. Collaborative Reasoner

Collaborative Reasoner evaluates multi-agent dialogue. Matrix reimplements this with peer-to-peer orchestrators. On 31 A100 nodes using LLaMA 3.1 8B Instruct, Matrix achieves a token throughput of around 2 billion tokens in 4 hours—6.8 times faster than the original model.

2. NaturalReasoning Web Data Curation

NaturalReasoning uses Matrix to curate reasoning datasets from web corpora. It shows a 2.1 times higher throughput due to efficient scheduling of agents.

3. Tau2-Bench Tool Use

In a customer support scenario, Matrix generates 22,800 trajectories in about 1.25 hours, delivering 15.4 times higher token throughput compared to baseline implementations.

Key Takeaways

• Matrix replaces centralized orchestrators with a peer-to-peer architecture that treats each task independently.
• The framework is built entirely on an open-source stack and scales to large workflows.
• Matrix achieves significant improvements in token throughput while maintaining output quality.

Editorial Notes

Matrix is a significant contribution to operationalizing multi-agent synthetic data generation. It separates scheduling, LLM inference, and tools effectively. The case studies highlight that careful systems design is crucial for scaling synthetic data pipelines.