Unveiling the Cost Dynamics of Output Token Pricing in Llama 3.3 70B

Introduction

In evaluating models for Llama 3.3 70B-class tasks, cost assessments often focus on the input token rate. However, this approach is incomplete. Output tokens, generated sequentially through autoregressive decoding, come at a higher price in many models. These tokens are priced at a premium, cannot be reduced by prompt caching, and influence overall costs more than input rates suggest. This article explores the cost mechanics of Llama 3.3 70B, verified pricing structures, and the output:input crossover that alters model rankings. Key considerations for production workloads are also discussed. Note: Always re-verify model pricing before deploying in production.

TL;DR

• Output token pricing forms the minimal cost of inference. A study by TraceLab found that output tokens only account for 11.2% of session costs in multi-agent workflows, indicating their cost dominance is per-step rather than session-wide.
• The cost crossover point at which Llama 4 Maverick ($0.25/$0.87 per 1M tokens) becomes more expensive than Llama 3.3 70B ($0.65/$0.65) is a ratio of 1.82. In a full reference task at 0.38:1, Maverick proves cheaper by $24.88 per 10,000 tasks.
• Gemma 4 ($0.18/$0.50 per 1M tokens) emerges as the most cost-effective model for these workloads at $29.17 per 10,000 tasks.
• Prompt caching reduces costs for cached input tokens only; output tokens always incur full charges.
• Artificial Analysis's default blending weight assumes a high cache hit rate, underestimating costs for output-heavy pipelines.

Understanding Output Token Costs

Why Output Tokens Cost More

The prefill stage processes all input tokens in a single, parallelized pass through the model layers. Conversely, the decode stage processes output tokens one at a time, each requiring a complete forward pass. This sequential process uses less GPU per token, resulting in a higher pricing rate. This accounts for the 2.2x to 5.4x cost premium over input tokens.

Influence of KV Cache on Cost Distribution

The key-value (KV) cache stores computed attention states, allowing the model to reuse previously processed tokens instead of recalculating them. As sessions extend, cached inputs replace fresh ones, significantly shifting the session cost towards cached reads. Data suggests that cache hit rates can reach as high as 97.5% for certain task types.

Reference Workload: A Five-Step Agent Task

The following estimates illustrate the token distribution per task, totaling 7,870 input tokens and 3,000 output tokens. This setup aligns with the cost ratio derived from DigitalOcean’s model cost table.

Token Estimates per Step

| Step | Task Description | Input Tokens | Output Tokens | Out:In Ratio | |------|-----------------------------------|--------------|---------------|--------------| | 1 | System prompt and user query | 2,200 | 100 | 0.05 | | 2 | Tool selection and planning | 2,000 | 380 | 0.19 | | 3 | Tool result ingestion | 2,370 | 60 | 0.03 | | 4 | Chain-of-thought decomposition | 300 | 2,010 | 6.70 | | 5 | Final response synthesis | 1,000 | 450 | 0.45 | | Total | Aggregate | 7,870 | 3,000 | 0.38 |

Misleading Nature of Aggregate Ratios

The aggregate ratio of 0.38:1 is below the crossover point between Llama 3.3 70B and Llama 4 Maverick, making Maverick the cheaper choice. However, the fourth step’s high output ratio makes Llama 3.3 70B more economical. Thus, relying solely on aggregate ratios for model selection can be misleading.

Pricing for Llama 3.3 70B in DigitalOcean’s Catalogue

Model Comparison Table

Below is a comparison of various models based on their cost per million tokens and the calculated cost per 10,000 tasks, using the reference workload.

| Model | Input ($/1M) | Output ($/1M) | Output:Input | $/10k Tasks | |------------------|--------------|---------------|--------------|-------------| | Llama 3.3 70B | $0.65 | $0.65 | 1.00x | $70.66 | | Llama 4 Maverick | $0.25 | $0.87 | 3.48x | $45.78 | | Qwen3-32B | $0.25 | $0.55 | 2.20x | $36.18 | | Gemma 4 | $0.18 | $0.50 | 2.78x | $29.17 | | Kimi K2.5 | $0.375 | $2.025 | 5.40x | $90.26 | | GLM-5.2 | $1.05 | $4.40 | 4.19x | $214.64 |

Flat-rate pricing of Llama 3.3 70B is rare, as most models charge significantly more for output tokens.

Calculating Cost per Agent Task

def cost_per_task(input_tokens: int, output_tokens: int, input_rate: float, output_rate: float) -> float:
    return (input_tokens * input_rate + output_tokens * output_rate) / 1_000_000

WORKLOAD_INPUT = 7_870
WORKLOAD_OUTPUT = 3_000

models = {
    "Llama 3.3 70B": (0.65, 0.65),
    "Llama 4 Maverick": (0.25, 0.87),
    "Qwen3-32B": (0.25, 0.55),
    "Gemma 4": (0.18, 0.50),
    "Kimi K2.5": (0.375, 2.025),
    "GLM-5.2": (1.05, 4.40),
}

from decimal import Decimal, ROUND_HALF_UP

def usd(value: float, places: str) -> Decimal:
    return Decimal(str(value)).quantize(Decimal(places), rounding=ROUND_HALF_UP)

for name, (in_rate, out_rate) in models.items():
    task_cost = cost_per_task(WORKLOAD_INPUT, WORKLOAD_OUTPUT, in_rate, out_rate)
    print(f"{name}: ${usd(task_cost, '0.000001')}/task  "
          f"(${usd(task_cost * 10_000, '0.01')}/10k tasks)")

Output

Llama 3.3 70B: $0.007066/task  ($70.66/10k tasks)
Llama 4 Maverick: $0.004578/task  ($45.78/10k tasks)
Qwen3-32B: $0.003618/task  ($36.18/10k tasks)
Gemma 4: $0.002917/task  ($29.17/10k tasks)
Kimi K2.5: $0.009026/task  ($90.26/10k tasks)
GLM-5.2: $0.021464/task  ($214.64/10k tasks)

Gemma 4 proves the most economical for this workload at $29.17 per 10,000 tasks, followed by Qwen3-32B and Llama 4 Maverick. The flat-rate Llama 3.3 70B costs more due to its input-heavy nature, whereas it performs better on output-heavy steps.

Cost Control Strategies for Output-Heavy Workloads

Setting Output Token Budgets

Limiting max_tokens to a calculated safe value helps manage costs. This value should be set at the p95 output token count plus a 10–15% buffer. Handling finish_reason: length is crucial to avoid incomplete responses.

import os
import openai

client = openai.OpenAI(
    base_url="https://inference.do-ai.run/v1",
    api_key=os.environ["MODEL_ACCESS_KEY"],
)

response = client.chat.completions.create(
    model="llama3.3-70b-instruct",
    messages=[{"role": "user", "content": "List the steps to configure prefix caching."}],
    max_tokens=512,
)

print(response.choices[0].finish_reason)
print(response.usage.completion_tokens)

if response.choices[0].finish_reason == "length":
    raise ValueError(
        f"Response truncated at {response.usage.completion_tokens} tokens; "
        "increase max_tokens or split the prompt."
    )

Routing by Expected Output Profile

Steps can be routed based on expected output:input ratios. For instance, steps below 1.82 should use models like Llama 4 Maverick, while those above this threshold should use Llama 3.3 70B.

Monitoring Output Token Distribution

Regularly tracking p50, p95, and p99 output token counts helps in identifying potential cost issues early, such as runaway CoT or looping tool calls.

Conclusion

Output token pricing is a significant cost factor for Llama 3.3 70B, challenging the assumption that lower input rates always equate to lower costs. While session averages provide a general overview, step-level analysis is crucial for accurate cost management, particularly for output-heavy tasks. As pricing can change, it’s essential to regularly verify model costs and adjust strategies accordingly.