@huggingface/kernels Brings 207 WebGPU Kernels to Local AI in the Browser

@huggingface/kernels Brings 207 WebGPU Kernels to Local AI in the Browser

Hugging Face has released @huggingface/kernels, a JavaScript library for loading and running optimized WebGPU kernels from the Hugging Face Hub. The release includes an initial collection of 207 kernels for machine learning operations used in browser-based inference.

Browser inference depends on several layers. Models need browser-friendly representations, runtimes must create efficient execution plans, and individual GPU operations must perform well across different devices and browser implementations. The new package focuses on this low-level kernel layer.

Each kernel is published as a complete, versioned package. Its interface, shader templates, correctness tests, benchmark cases, and usage instructions are stored together in a repository on the Hub.

Hugging Face has also introduced Fleet, a browser-based GPU benchmarking and testing suite. Fleet runs kernels on a user's hardware and records correctness and performance evidence with consent. This helps identify incorrect results, unusually slow cases, device-specific behavior, and opportunities to improve kernel variants.

Overview

• 207 WebGPU kernels, published as individual Apache-2.0-licensed repositories in the webgpu-kernels organization.
• The @huggingface/kernels JavaScript loader, which downloads, prepares, and runs kernels directly from the Hub.
• Explicit contracts and reproducible evidence for every kernel, including manifests, correctness tests, benchmark cases, and WGSL shader templates.
• Fleet, a browser-based benchmarking tool that collects correctness and performance evidence across real-world GPUs.

Why kernels are important

A model running in a browser eventually becomes a sequence of GPU operations, including matrix multiplications, normalizations, convolutions, attention primitives, quantization operations, and data-layout transformations. WebGPU exposes these operations through a portable browser API, while WGSL provides a common language for the shaders that execute them.

Portability does not guarantee consistent performance. Two shaders can implement the same operation and produce the same output while behaving very differently across accelerators. Workgroup sizes, memory access patterns, vectorization, data types, and fusion strategies can all affect execution time. The best implementation can also vary according to input shape, device, browser, and available WebGPU features.

Kernels therefore provide a foundation for browser inference. Higher-level runtimes can only be as efficient as the operations they dispatch. Making those operations discoverable, testable, benchmarkable, and versioned allows the underlying implementations to improve independently while preserving a stable contract for higher layers.

A repository containing more than a shader

Every kernel has its own repository and kernel card. The card describes the operation's semantics, inputs, outputs, attributes, supported data types, source files, and a runnable @huggingface/kernels example.

One example is ai.onnx.Add, which implements elementwise addition with multidirectional broadcasting. The operation is used in neural networks for tasks such as residual connections and bias addition. Its card describes the inputs, broadcasted output shape, supported data types, and variants available for different shapes and devices.

The repository includes the artifacts needed to inspect and evaluate the implementation:

• manifest.json defines the operation contract, including inputs, outputs, attributes, type constraints, and shape-derivation rules.
• metadata.json records the kernel identifier, digests, and provenance.
• test.json contains correctness cases for checking expected behavior.
• bench.json contains benchmark and tuning cases used to evaluate the kernel.
• *.wgsl.jinja files contain parameterized WGSL implementations used to generate shaders for a particular request and device.

This structure makes the shader a reusable software artifact. The interface can be inspected without reading WGSL, correctness and performance cases remain attached to the implementation, and published versions can be loaded explicitly instead of relying on an unversioned file URL. The kernels can also serve as reference implementations for developers creating custom WebGPU kernels or integrating these operations into their own runtimes.

Loading a kernel from the Hub

Install the package from npm:

npm install @huggingface/kernels@preview

Running these kernels requires a browser with WebGPU support. Availability depends on the browser, operating system, GPU, and driver. JavaScript can check for support with:

"gpu" in navigator

The package connects a kernel repository with an application. getKernel receives a Hub repository ID and a contract version, then returns a function that accepts typed input data and tensor shapes.

The following example performs a bias-add operation:

import { getKernel } from "@huggingface/kernels";

const add = await getKernel("webgpu-kernels/ai.onnx.Add", { version: 1 });

const { c } = await add({
  a: {
    data: new Float32Array([1, 2, 3, 4, 5, 6]),
    shape: [2, 3],
  },
  b: {
    data: new Float32Array([10, 20, 30]),
    shape: [3],
  },
});

The second input broadcasts across the first dimension, producing an output with shape [2, 3]. The loader derives the output shape and logical data type from the manifest contract and the inputs, then allocates c automatically.

The six-element example is intentionally small. At that size, the GPU round trip takes considerably more time than the arithmetic itself. The example demonstrates the same calling pattern used for larger operations, such as matrix multiplication with ai.onnx.MatMul. In those cases, the repository ID and input data change, but the application-facing structure remains the same.

The Add kernel also demonstrates why multiple variants are useful. Equal-shape addition can use a direct vectorized path, while broadcasted inputs require different indexing logic. The published kernel includes variants for equal shapes, vectorized broadcasting, scalar processing, and general broadcasting. The runtime can choose an implementation based on the current call and device without changing the application API.

The { version: 1 } option selects version 1 of the published kernel contract. This is separate from an ONNX opset, an operator's since_version, or a model revision. Keeping these concepts distinct allows applications to depend on a stable JavaScript-facing contract while kernel implementations change behind it.

Performance results

Hugging Face compared the collection with ORT WebGPU on an Apple M4 GPU, using ONNX Runtime Web 1.30.0-dev.20260826-b1f76d586a. The evaluation began with 1,756 test cases across all 207 operations and retained 809 cases in which both implementations produced matching outputs and reliable timings.

Across those comparisons, the Hugging Face kernels were 2.57x faster by geometric mean and 1.90x faster at the median. They recorded 629 wins, 176 losses, and 4 ties.

Some individual cases showed substantially larger differences. A difficult bilinear Einsum case, i,ij,j with size 4096, ran in 0.136 ms with the Hugging Face kernel and 1,396 ms with ORT WebGPU, a difference of more than 10,000x. A row-wise CumSum over [256, 4096] was 301x faster, running in 0.016 ms compared with 4.784 ms.

These are unusual cases and are not representative of every workload. The measurements covered GPU execution only and excluded setup work such as loading kernels, creating sessions, uploading inputs, compiling shaders, and reading outputs. Very short workloads are also more difficult to measure, and small cases can benefit from the GPU cache.

The results apply to individual operations rather than complete models. Performance can change across GPUs and browsers, which is one reason Fleet is used to gather evidence from a broader range of hardware.

Hugging Face is also working with the ONNX Runtime team to upstream these improvements to the wider ONNX Runtime Web ecosystem.

Testing across a fleet of devices

WebGPU performance varies among GPUs, browsers, and drivers. Results from one machine therefore provide only a limited view. Fleet allows users to run correctness and performance checks in the browser and inspect how the kernels behave on their own hardware.

With consent, each run contributes private evidence that can help identify device-specific failures, compare kernel variants, and improve selection rules. Broader real-world coverage can support more reliable tuning than testing only a fixed set of laboratory devices.

A shared foundation for WebAI

The initial 207 kernels are a starting point. Publishing them independently on the Hub provides a shared location for inspecting contracts, comparing implementations, reproducing correctness checks, and improving performance without embedding every shader directly into each runtime.

The collection is part of the Hub's broader kernel ecosystem. On the Kernels page, WebGPU kernels appear alongside kernels for CUDA, ROCm, Metal, and other platforms. They can be filtered, sorted, and explored as Hub artifacts.

The system consists of four connected parts:

1. Kernel repositories define transparent, versioned operation contracts.
2. @huggingface/kernels loads and runs those operations from JavaScript.
3. Fleet gathers correctness and performance evidence across a broad range of devices.
4. Contributed runs can reveal failures, guide tuning, improve variant selection, and help validate future kernel versions.

These components provide a low-level foundation for browser inference. The next steps include connecting the kernels to higher-level model tooling and expanding operation coverage for local AI workloads across the WebAI ecosystem.