Open Measurement for AI Buyers: MLPerf Selected for EU AI Gigafactories Tender as Endpoints v1.0 Opens

As AI adoption expands across the economy, buyers are evaluating more than chip speed. They need to understand the performance a deployed service can provide, the cost of running specific workloads, and the energy it consumes. MLPerf has measured AI system performance since 2018 and is extending its benchmarks to address these broader purchasing questions.

Two recent developments reflect this shift. The European Union has selected MLPerf to evaluate training and inference performance in bids for its AI Gigafactories. At the same time, MLPerf Endpoints v1.0 is accepting submissions, formalizing a standardized approach to measuring AI services as they are delivered through live APIs.

Together, these developments show growing interest from industry, governments, and buyers in open, peer-reviewed measurements for AI adoption decisions.

MLPerf selected for the EU AI Gigafactories tender

In July, the European Union opened a call for AI Gigafactories: up to seven large-scale AI computing facilities supported by up to €10 billion in EU and national funding. The facilities are expected to unlock more than €30 billion in total investment. The tender is being conducted as a joint procurement by the EuroHPC Joint Undertaking and national contracting authorities, with bids due November 12, 2026.

The tender specifications identify MLPerf as the standard for measuring bidders’ technical performance. Proposals will be evaluated using MLPerf Training v5.1 and Inference v5.1 benchmarks.

Across the two benchmark suites, the tender references more than six models from MLPerf’s standard workloads. Training workloads include large models such as Flux.1 and Llama. Inference workloads include Llama 3.1 405B, DeepSeek R1, and SDXL 1.0.

The requested information builds on MLPerf’s standard metrics: time-to-train for training, and time to first token and time per output token for inference. Bidders must also provide additional information, including accelerator utilization. These results must be reported for the capacity delivered in Phase 1 and the planned expansion in Phase 2.

For procurement involving public funding at this scale, performance claims need to be comparable and independently reviewable. The tender’s use of MLPerf provides a common measurement framework for evaluating the proposals.

David Kanter, Head of MLPerf, said the benchmark is intended to help buyers assess AI deployments and that the organization was honored to see its work used in the European Union’s AI infrastructure initiative.

MLPerf Endpoints v1.0 opens for submissions

MLPerf Endpoints measures AI as it is consumed in practice, through the live API that applications use to access a model. The system under test is a URL that the Endpoints client contacts through standard interfaces.

This approach allows buyers to evaluate hardware vendors, hyperscalers, neoclouds, and managed API providers using the same framework. It applies whether the service runs on bare metal, through a managed cloud service, or in a containerized software stack.

Rather than producing one peak-performance number, Endpoints tests each system across a range of loads and reports a broader performance profile. Measurements include concurrency, interactivity in tokens per second per user, system throughput, and time to first token at the 90th percentile, or TTFT P90. Buyers can use these results to identify a verified operating point that matches their applications and service-level targets.

Endpoints 1.0 also introduces mandatory provisioned power measurement, a first for the MLPerf benchmark family. The measurement provides information for comparing energy efficiency and the cost profile of each solution.

Results are displayed as a Pareto frontier in the LoadStair visualizer. The visualization indicates where a system maintains performance and where it reaches a performance cliff as load increases.

Version 1.0 builds on the v0.7 foundation release from July. It includes generative AI workloads such as DeepSeek-R1, GPT-OSS 120B, and Llama models. It also adds agentic coding workloads using DeepSeek v4 Pro, Kimi K3, and Qwen 3.6.

Endpoints now supports rolling submissions. Members can publish peer-reviewed results after releasing a new chip or software optimization instead of waiting for a fixed submission round. Each result continues to undergo MLPerf’s peer-review and audit process. The first v1.0 results are expected to be published in November.

MLCommons members can access the Endpoints submission guide and rules through the MLPerf Endpoints benchmark documentation.