Salesforce Unveils New Benchmarks and Models to Enhance Trustworthy AI Agents

Tackling Inconsistent AI Performance with Targeted Benchmarks

Salesforce AI Research has identified a critical issue they call "jagged intelligence," where AI agents behave inconsistently across tasks of similar complexity. To address this, they introduced the SIMPLE benchmark, which consists of 225 reasoning-focused questions that humans answer consistently but challenge AI language models. This helps expose gaps in the models' reasoning and generalization abilities.

Alongside SIMPLE, the ContextualJudgeBench evaluates agents on the accuracy and contextual relevance of their answers, including their ability to abstain from answering when necessary. This feature is essential for high-stakes domains like legal, financial, and healthcare applications where trust and precision are paramount.

Enhancing Safety and Robustness with Trust Layers

Salesforce expanded its Trust Layer with the SFR-Guard model family, trained on both general and CRM-specific data to detect prompt injections, toxic outputs, and hallucinations. These models act as dynamic filters during real-time inference to maintain safe and reliable AI interactions.

Additionally, CRMArena simulates real-world CRM workflows to test agent performance in complex enterprise scenarios, ensuring agents generalize well beyond training prompts and operate predictably across varied tasks.

Specialized Models for Reasoning and Action

To enable more structured and goal-driven agent behaviors, Salesforce introduced two new model families:

• xLAM (eXtended Language and Action Models): Scalable models optimized for tool use, multi-turn conversations, and function calling, supporting enterprise deployments that integrate with APIs and internal knowledge bases.

• TACO (Thought-and-Action Chain Optimization): Models designed to enhance planning by explicitly modeling intermediate reasoning steps and corresponding actions, suited for document automation, analytics, and decision support.

Unified Deployment with Agentforce

All these advancements are integrated into Agentforce, Salesforce's platform for building and deploying autonomous agents. Agentforce features a no-code Agent Builder enabling developers and domain experts to define agent behaviors using natural language. It integrates tightly with the Salesforce ecosystem to provide agents access to customer data, workflows, and auditability.

A study by Valoir demonstrated that teams using Agentforce build production-ready agents 16 times faster and improve operational accuracy by up to 75%. Agents built on Agentforce inherit Salesforce's Trust Layer safety and compliance features, essential for enterprise use.

Salesforce's holistic approach to AI agent development combines new benchmarks, safety mechanisms, and specialized architectures to create more reliable, adaptable, and trustworthy AI agents tailored for enterprise needs.