Code2Video-AA: How Well Do AI Models Design Videos with Code?

Code2Video-AA: How Well Do AI Models Design Videos with Code?

Code2Video-AA is an independent implementation by Artificial Analysis of HeyGen's Code2Video Bench. It evaluates how well AI models create short product and brand videos from 114 design briefs using HyperFrames HTML. Results in this launch article are current as of October 26, 2026, using dataset version 0.3.0.

Overview

Each model receives a design brief and creates the video in code. The benchmark measures the quality of the resulting rendered videos across five judge axes.

Key findings

• Claude Opus 5.5 (max with fallback) leads with 1536 Elo, 93 points ahead of the next model. It ranks first on all five judge axes.
• Claude Sonnet 5.5 (max with fallback) and GPT-6 Astra (max) are effectively tied for second at 1443 and 1427 Elo, respectively. GPT-6.1 Sol (max) ranks fourth at 1369.
• Cost per task varies by more than 300 times, from $0.01 to $3.89. GPT-6.1 Sol (max), which ranks fourth, costs $0.31 per task, compared with $3.08 for Claude Opus 5.5 (max with fallback).
• The leading open-weights models are DeepSeek V4.1 Flash (max) at 1053 Elo, MiMo-V2.6-Flash at 1052, and MiMo-V2.6-Pro at 1052. They trail the leader by at least 483 points.

Code2Video-AA Elo

The score is the mean Elo across the five judge axes. Scores are anchored to GPT-5.5 (xhigh) at 1000, with higher scores indicating better performance.

Model rankings

The models were ranked as follows:

Example task

One public Code2Video Bench task describes a roughly 7.3-second, 16:9 UI motion graphic in the style of a product screen recording. The video demonstrates a mobile-app wireframe builder.

The brief specifies a black title card followed by a flat, light-gray workspace. A white, mobile-shaped canvas appears in the center, with a left-hand block menu and a right-hand settings panel. The interface uses a light-gray workspace, darker gray panels, near-black text, muted gray labels, white canvas areas, and a lime-yellow accent used only for the primary button and selected items.

The camera must remain locked and front-facing throughout the recording. There are no zooms, pans, push-ins, or other camera movements. Motion is limited to the pointer, panel changes, and blocks appearing on the canvas.

The sequence is divided into five scenes:

1. Title card, 0.0 to approximately 0.9 seconds: A solid black background displays the centered white text Rapid wireframing, followed by a hard cut.
2. Editor base state, approximately 0.9 to 2.0 seconds: The workspace appears with the block menu, a blank mobile canvas, and controls including Discard changes and Save and exit. The pointer moves toward the menu.
3. Add a Spacer, approximately 2.0 to 3.5 seconds: Selecting Spacer adds a dashed spacer block to the canvas. The settings panel switches to Spacer block settings, with size options XS, S, M, L, and XL.
4. Add media or an image placeholder, approximately 3.5 to 5.0 seconds: Selecting Placeholder or Media adds a square light-gray image placeholder. The settings panel changes to Media block settings.
5. Add text, approximately 5.0 to 7.3 seconds: Selecting Text adds a text block. The placeholder Write something down… appears, followed by character-by-character text entry such as Welcome!. The settings panel changes to Text block settings, with formatting controls and a Generate text button.

All transitions are instantaneous swaps or hard cuts. The pointer drives each action, and the text is typed automatically. The public split contains rendered videos from all 27 models.

The briefs come from the public split of HeyGen's Code2Video Bench and are licensed under CC BY 4.0.

How Code2Video-AA works

The evaluation follows four stages:

1. Write: The model receives a design brief and HeyGen's system prompt in a single call, without tools.
2. Render: The model returns one HyperFrames HTML composition in index.html. HTML defines the layout, while GSAP defines the animation.
3. Judge: The resulting videos are compared across the benchmark's judge axes.
4. Rank: The comparison results are used to fit Elo scores.

The implementation selects briefs, generates and renders videos, judges comparisons, samples matchups, and calculates the final Elo scores.

Claude Opus 5.5 leads across all five judge axes

Claude Opus 5.5 has the highest score on every judge axis. Its largest lead is in craft, where it is 128 points ahead of Claude Sonnet 5.5 (max with fallback). Its smallest lead is in prompt intent, at 48 points.

Temporal consistency produces the narrowest score range between models, at 1057 Elo points. The other four axes have ranges between 1222 and 1349 points.

Each axis uses separate comparisons and its own Elo calculation. The overall Elo is the mean of the five axis scores. Model ordering is broadly consistent across the axes: no model moves more than six places relative to its overall position.

Cost per task varies by more than 300 times

Average cost per task ranges from $0.01 for MiMo-V2.6-Flash to $3.89 for Claude Fable 5.1 (max with fallback).

Eight models sit on the Elo-versus-cost Pareto frontier. The frontier runs from MiMo-V2.6-Flash, with 1052 Elo at $0.01 per task, to Claude Opus 5.5 (max with fallback), with 1536 Elo at $3.08 per task.

Other points on the frontier include:

• Claude Haiku 5.5 (max): 1192 Elo at $0.08 per task
• GPT-6.1 Sol (max): 1369 Elo at $0.31 per task

GPT-6 Astra (max) costs $1.62 per task, compared with $0.31 for GPT-6.1 Sol (max). It scores 58 more Elo points.

Output tokens per task vary 16 times

Output tokens range from approximately 11,000 for Nemotron 3 Ultra to approximately 176,000 for Claude Sonnet 5.5 (max with fallback). Most of the difference comes from reasoning tokens.

The final answers, which contain the video compositions, range from approximately 4,000 to 12,000 tokens across all models. The three Claude 5.5 models produce the most output tokens, approximately 152,000 to 176,000 per task, compared with a mean of approximately 42,000 and a median of approximately 34,000 for the other 24 models.

Every model renders at least 96% of its videos

A composition must render successfully before it can be judged. In total, 3042 of 3078 tasks rendered, representing 98.8% of all tasks. Eight of the 27 models rendered all 114 tasks.

GLM-5.3 (max) had the lowest render rate, with 110 of 114 tasks rendered.

A composition that fails to render loses every comparison against a video that rendered. The three lowest-ranked models rendered between 111 and 113 of 114 tasks, so their low Elo scores were primarily determined by judge evaluations of videos that rendered successfully.

Code2Video-AA resources

• Code2Video-AA evaluation results are available on the leaderboard.
• The methodology describes the implementation, judging process, and scoring system.
• HeyGen's announcement describes the Code2Video Bench.
• The HyperFrames renderer is available on GitHub.