Candela Brings Standardized AI Risk Benchmarking to Developers and Deployers

Organizations using AI need to understand when systems may produce unsafe responses and how frequently those failures occur. That measurement is difficult because behavior can change with every configuration, even for the teams that developed a model. Until now, there has been no widely shared method for testing these risks.

MLCommons has introduced Candela, a benchmarking-as-a-service platform for AI risk and reliability. MLCommons is an industry organization with more than 125 companies, including Microsoft, Google, Qualcomm, NVIDIA, OpenAI, and Anthropic, along with academic institutions and nonprofits. Candela makes its consensus-built AILuminate benchmarks available on demand across the development and deployment lifecycle, from a frontier model under development to a multi-model enterprise system with complex integration layers.

Measuring common AI risks

Many of the risks organizations face from AI are familiar operational risks: a customer-service bot may provide dangerous advice, a coding assistant may be manipulated into bypassing its safeguards, or an internal tool may expose information it should protect. Organizations already have established methods for managing risks that can be measured. For AI, the missing element has been consistent measurement.

Candela launches with new versions of both AILuminate benchmarks:

• Safety 1.2.1 evaluates how a system handles 12 hazard categories, including violent crime and specialized financial, health, and legal advice.
• Jailbreak 1.1 evaluates 11 of those categories using techniques from the MLCommons jailbreak taxonomy. These range from typos, lookalike characters, and prompts such as “ignore previous instructions” to more complex methods involving context framing, deception, and fragment assembly.

Safety 1.2.1 results for about 30 named open-weight models are publicly available, with Jailbreak results planned for a later release.

Additional benchmarks are under development. MLCommons working groups are creating evaluations for agentic security, privacy, and psychosocial harms. The organization is also developing agent reliability profiles for industries such as financial services and extending existing benchmarks to additional languages and cultural contexts.

How AILuminate scores systems

A benchmark is useful only if systems cannot simply learn its test set. AILuminate therefore keeps its prompts private. Only MLCommons staff can access them, including staff at member companies. The organization monitors and refreshes the prompt sets as models and risks change, while keeping other parts of the process as open as possible.

The standards are developed by MLCommons AIRR working groups. Participants from more than 80 organizations determine what should be tested and how responses should be graded. Participants include AI developers such as Google DeepMind, Meta, Microsoft, OpenAI, and NVIDIA, as well as Accenture, Deloitte, RAND, Stanford, Oxford, and civil-society organizations including NewsGuard and SaferAI.

MLCommons performs the scoring using a private machine-learning classifier designed specifically for safety evaluation rather than an LLM-as-a-judge approach. According to MLCommons, the focused training helps the classifier apply the assessment standard consistently, and its performance can be improved as more ground-truth data becomes available. A dedicated classifier also provides more stable results over time than using a different generative AI judge for each evaluation. MLCommons reports that this advantage is particularly relevant for jailbreak testing, where attacks are designed to bypass language-model safeguards. In trials, generative AI graders could not reliably determine whether a response had been manipulated by a jailbreak.

The scoring process follows the principles of a standardized test. AILuminate uses item response theory, the statistical method used by tests such as the SAT and GRE, to estimate the difficulty of each prompt and how effectively it distinguishes stronger systems from weaker ones. Results are reported as percentiles against a fixed panel of reference systems, so a score does not change based on which other systems have been evaluated.

Testing the system that is actually deployed

A model’s published score describes its behavior in its released form. In practice, organizations often add a system prompt, retrieval, tools, or an agent loop. Each of these changes can affect system behavior.

Candela can evaluate both the underlying model and the configured system through a local client relay. Its scores are designed to support comparisons across configurations, systems, and points in time.

A model that refuses a harmful request in its original form may respond differently after fine-tuning, the assignment of a persona, the introduction of a document containing embedded instructions through retrieval-augmented generation, or the addition of text from a trusted tool. The opposite can also occur: safeguards added around a model with weaker standalone safety performance may make the complete system safer than the original model.

This means that safety scores are not composable. The safety properties of individual components cannot simply be combined to predict the safety of the complete system.

Three levels of evaluation

Candela provides three types of results:

• Practice tests provide a score and summary report using practice prompts. They are intended for frequent internal use, including development smoke tests and initial evaluations of deployment candidates. Practice tests are free for MLCommons members.
• Annotated results include each prompt, response, and scored annotation behind the aggregate result. Teams can use this information to identify where a system performed well or poorly and determine whether a low hazard score is relevant to a particular use case.
• Official results use private official prompts and produce scores that organizations can cite in system cards, specification sheets, or procurement decisions. The official questions are completely separate from those used for practice and annotated tests, reducing the possibility that a system has memorized the answers. Developers and deployers can run Official Results themselves or use a third-party assurance provider for additional independence.

Perspectives from member companies

MLCommons member companies described Candela as a way to support AI safety evaluation across both model development and system deployment.

Microsoft Principal Product Manager for Responsible AI Abby Palia said enterprises need credible evidence when making model-safety decisions. She noted that no single benchmark covers every safety risk and that private test sets can help preserve meaningful measurement as public benchmarks become familiar. She also said that Candela gives enterprises another way to assess safety behaviors relevant to their use cases.

Eileen Long, Senior Director of AI Services at NVIDIA, said enterprises need trusted methods for evaluating the AI models and systems they deploy. NVIDIA uses Candela to test Nemotron and publish AILuminate Safety scores.

William Isaac, Principal Scientist at Google DeepMind, said Google DeepMind tested a production Gemini model against AILuminate inside a secure enclave. In that setup, MLCommons prompts remained hidden from Google DeepMind, while model weights remained hidden from the evaluators. He described the arrangement as a way to support independent evaluation while protecting data sovereignty and security.

Candela's role in AI governance

Organizations already use several tools for internal AI governance, including evaluations tailored to particular use cases and policies, red teams that search for exploitable failures, and production monitoring. These methods serve the organization operating the system.

Candela addresses a different need: a measurement that can be understood across organizations and used by developers, buyers, auditors, and boards. Each result includes version labels. MLCommons states that it will not change the meaning of a score without changing its label, allowing results to be cited in governance documents and compared over time.

Regulators and standards bodies face a similar challenge at a larger scale. Rules that apply to everyone are easier to implement and assess when organizations can measure compliance in a consistent way. Governments and institutions are still determining which AI requirements to establish and which properties can be measured reliably. MLCommons working groups and technical staff plan to adapt and expand AILuminate as those requirements develop.

In the meantime, organizations that assess AI safety consistently can build a record of performance before formal requirements take effect.

Using Candela

Candela's launch follows work involving more than 80 organizations, including organizations that often compete with one another. The service includes the prompt sets, evaluation classifier, and supporting infrastructure developed by MLCommons.

Organizations can use Candela in several ways:

• Run an evaluation: Practice tests are free for MLCommons members.
• Become an assurance partner: Firms that advise enterprises on AI adoption and risk can participate in the Global Assurance Program.
• Help develop benchmarks: The AIRR working groups accept participants, particularly representatives of enterprise buyers.
• Review documentation: MLCommons provides an FAQ and a guide to AI risk for businesses.