Meta AI Launches CATransformers: Pioneering Carbon-Aware AI and Hardware Co-Optimization for Greener Edge Computing

Addressing the Environmental Impact of Machine Learning

Machine learning systems, integral to applications like recommendation engines and autonomous technologies, demand vast computational power. This energy consumption during training and inference phases significantly contributes to carbon emissions. Beyond operational energy use, the hardware itself has embodied carbon — emissions from manufacturing, materials, and lifecycle operations — which adds to the ecological footprint. Tackling both operational and embodied carbon is critical as AI adoption grows across industries.

Limitations of Current Carbon Mitigation Approaches

Most existing strategies focus on reducing energy consumption during AI model operation or enhancing hardware utilization. However, they often overlook the embodied carbon embedded in hardware manufacturing. Additionally, the relationship between AI model design and hardware efficiency remains underexplored, especially for complex multi-modal models combining visual and textual data.

Innovations in AI Model Efficiency and Their Shortcomings

Techniques like pruning, distillation, and hardware-aware neural architecture search optimize AI models for latency and energy but rarely consider embodied carbon. While frameworks such as ACT, IMEC.netzero, and LLMCarbon assess embodied carbon independently, they lack integrated approaches that jointly optimize models and hardware for total carbon reduction. Edge-adapted models like TinyCLIP focus on deployment speed and feasibility but do not address comprehensive carbon footprints.

Introducing CATransformers

Meta's FAIR team, in collaboration with Georgia Institute of Technology, developed CATransformers — a framework that incorporates carbon footprint as a central design metric. It enables co-optimization of AI model architectures and hardware accelerators by simultaneously evaluating accuracy, latency, energy consumption, and total carbon emissions. Targeting edge inference devices, CATransformers accounts for both embodied and operational emissions, using a multi-objective Bayesian optimization engine to explore design trade-offs early in development.

Architecture and Workflow of CATransformers

The framework consists of three interconnected modules:

• Multi-objective optimizer: Balances performance metrics against carbon footprint.
• ML model evaluator: Generates variants by pruning a base CLIP model, adjusting layers, feedforward size, attention heads, and embedding width.
• Hardware estimator: Profiles each variant to estimate latency, energy use, and carbon emissions.

This process facilitates rapid assessment of how architectural decisions impact both emissions and performance, enabling informed optimization.

Results: The CarbonCLIP Models

CATransformers produced the CarbonCLIP family, showing impressive improvements over existing small CLIP models:

• CarbonCLIP-S matches TinyCLIP-39M’s accuracy but reduces carbon emissions by 17% and keeps latency under 15 ms.
• CarbonCLIP-XS surpasses TinyCLIP-8M accuracy by 8%, cuts emissions by 3%, and maintains latency below 10 ms.

Notably, models optimized solely for latency often doubled hardware demands, increasing embodied carbon drastically. In contrast, CATransformers’ balanced optimization achieved 19-20% emission reductions with minimal latency impact.

Key Insights and Impact

• Carbon-aware co-optimization addresses both operational and embodied emissions.
• Multi-objective Bayesian optimization integrates diverse metrics for holistic design.
• The approach proves that environmental sustainability and high AI performance can coexist.
• This research paves the way for more responsible AI development aligned with climate goals.

CATransformers represents a meaningful advancement toward sustainable AI by embedding carbon considerations directly into the design and deployment process, ensuring that future AI systems can be both efficient and environmentally conscious.