GPT-OSS-20B: A Comprehensive Deployment-Centric Analysis of OpenAI's Open-Weight Mixture of Experts Model

TL;DR

This paper evaluates GPT-OSS-20B, a Mixture-of-Experts model, demonstrating its deployment efficiency and performance advantages over dense models in terms of throughput, energy consumption, and VRAM usage on a single GPU.

Contribution

It provides a comprehensive deployment-centric analysis of GPT-OSS-20B, highlighting its efficiency benefits and practical deployment metrics compared to dense models.

Findings

GPT-OSS-20B achieves higher throughput and energy efficiency than dense models.

It significantly reduces peak VRAM usage during deployment.

MoE routing overhead increases TTFT despite efficiency gains.

Abstract

We present a single-GPU (H100, bf16) evaluation of GPT-OSS-20B (Mixture-of-Experts; 20.9B total, approx. 3.61B active) against dense baselines Qwen3-32B and Yi-34B across multiple dimensions. We measure true time-to-first-token (TTFT), full-decode throughput (TPOT), end-to-end latency percentiles, peak VRAM with past key values (PKV) held, and energy via a consistent nvidia-smi-based sampler. At a 2048-token context with 64-token decode, GPT-OSS-20B delivers higher decode throughput and tokens per Joule than dense baselines Qwen3-32B and Yi-34B, while substantially reducing peak VRAM and energy per 1000 generated tokens; its TTFT is higher due to MoE routing overhead. With only 17.3% of parameters active (3.61B of 20.9B), GPT-OSS-20B provides about 31.8% higher decode throughput and 25.8% lower energy per 1000 generated tokens than Qwen3-32B at 2048/64, while using 31.7% less peak VRAM. Normalized by active parameters, GPT-OSS-20B shows markedly stronger per-active-parameter efficiency (APE), underscoring MoE's deployment advantages. We do not evaluate accuracy; this is a deployment-focused study. We release code and consolidated results to enable replication and extension.