What Is OpenAI GPT-OSS and How Does It Perform?

TL;DR

GPT-OSS provides downloadable model weights in 120-billion and 20-billion parameter versions under the permissive Apache 2.0 license. The models combine adjustable reasoning, tool use, function calling, and a 128k context window with efficient local deployment. Their benchmark results approach several OpenAI O-series models, making them suitable for private infrastructure, fine-tuning, and on-device applications.

Transcript

Open AAI has delivered on their promise to release a state-of-the-art open-source model. This is GPTOSS. Now, I think the mystery model Horizon Alpha that was on Open Router is actually this open source model from OpenAI, although they have not confirmed that to me, but we do have an incredible model. Let me tell you about all the details. So, firs... Read More

Key Insights

• GPT-OSS is an open-weight model family offered in 120-billion and 20-billion parameter sizes under the Apache 2.0 license. Releasing the weights lets organizations run the models locally, fine-tune them, add specialized knowledge, and deploy them without depending entirely on hosted proprietary systems.
• The 120-billion parameter model is designed to fit on a single 80 GB GPU. Its mixture-of-experts architecture activates only 5 billion parameters for each token, reducing inference computation after the full model has been loaded into memory.
• The 20-billion parameter model can run with 16 GB of memory and activates 3.6 billion parameters per token. Its smaller hardware requirement supports local inference, edge applications, rapid experimentation, and deployments that do not require expensive centralized infrastructure.
• GPT-OSS supports low, medium, and high reasoning settings. Developers can select lower reasoning for faster answers and increase the setting for more difficult mathematics, science, coding, or general reasoning tasks that benefit from additional computation.
• The models natively support context lengths of up to 128k tokens. Their architecture combines alternating dense and locally banded sparse attention, grouped multi-query attention with a group size of eight, and RoPE positional encoding to improve inference and memory efficiency.
• The 120-billion model scored 2622 on Codeforces with tools, compared with 2706 for O3 with tools. On GPQA Diamond, it scored 80.1 versus O3's 83.3, while the 20-billion model achieved 71.5 on the same PhD-level science benchmark.
• The post-training process included supervised fine-tuning and a high-compute reinforcement-learning stage. Its objectives included alignment with the model specification, chain-of-thought reasoning, and tool use before producing answers, following an approach described as similar to the training of O4 Mini.
• The safety approach avoids direct supervision of chain-of-thought reasoning so developers can monitor it for possible misbehavior. OpenAI advises applications to summarize and filter this reasoning rather than display it directly because raw reasoning may contain harmful or hallucinated material.

Questions & Answers

Q: What is GPT-OSS and what model sizes are available?

GPT-OSS is a family of open-weight language models released in two sizes: a 120-billion parameter model and a 20-billion parameter model. Both provide downloadable weights under the Apache 2.0 license, allowing users to run them locally, modify their deployments, and fine-tune them for specialized use cases. The models emphasize reasoning, tool use, efficiency, and deployment across varied hardware environments.

Q: What hardware is required to run GPT-OSS locally?

The 120-billion parameter version is designed to run on a single GPU with 80 GB of memory. The 20-billion parameter version can operate on edge devices with 16 GB of memory, making it more accessible for local inference and rapid experimentation. The transcript also describes running the larger model on computers with 96 GB of unified memory or two A6000 GPUs.

Q: How does GPT-OSS reduce the cost of inference?

GPT-OSS uses a mixture-of-experts transformer architecture that activates only a portion of its total parameters for each token. The 120-billion model activates 5 billion parameters per token, while the 20-billion model activates 3.6 billion. Once the complete model fits in memory, this limited activation reduces the computation required during inference and supports more efficient local deployment.

Q: How can developers control GPT-OSS reasoning effort?

Developers can choose low, medium, or high reasoning settings, which adjust how much reasoning the model performs before answering. A low setting is appropriate when response speed matters more than deep analysis. A high setting can be used for difficult mathematics, science, coding, and reasoning problems where spending more computation on the problem may produce a stronger answer.

Q: How does GPT-OSS perform on coding and science benchmarks?

The 120-billion model scored 2622 on Codeforces when equipped with tools, compared with 2706 for O3 with tools. The 20-billion model scored 2516. On the GPQA Diamond PhD-level science benchmark, the larger model reached 80.1 compared with O3 at 83.3, while the smaller GPT-OSS model scored 71.5. These results indicate competitive reasoning across both sizes.

Q: What tasks and interfaces does GPT-OSS support?

GPT-OSS supports tool use, few-shot prompting, function calling, chain-of-thought reasoning, coding, health-related conversations, and general question answering. Both model sizes are compatible with the Responses API and are designed for use with agentic frameworks such as CrewAI. Their adjustable reasoning settings allow developers to balance response speed against the depth required for a particular task.

Q: What architecture and context length does GPT-OSS use?

Each GPT-OSS model is a mixture-of-experts transformer. The architecture uses alternating dense and locally banded sparse attention patterns, grouped multi-query attention with a group size of eight, and RoPE for positional encoding. Both models natively support context lengths of up to 128k tokens. Their training data emphasized high-quality text covering STEM, coding, and general knowledge.

Q: How was GPT-OSS evaluated for safety risks?

OpenAI filtered harmful chemical, biological, radiological, and nuclear material during pre-training, then tested how the models might behave after malicious fine-tuning. The evaluation created specialized non-refusing biology and cybersecurity versions resembling what an attacker might attempt. According to the transcript, these versions did not reach high capability levels under the preparedness framework. A $500,000 red-teaming challenge was also announced.

Summary & Key Takeaways

• GPT-OSS consists of 120-billion and 20-billion parameter open-weight language models released under the Apache 2.0 license. Users can download, run, and fine-tune them for specialized applications. The larger model fits on one 80 GB GPU, while the smaller model can operate on devices with 16 GB of memory.

• Both models use a mixture-of-experts transformer architecture designed to limit active computation. The 120-billion model activates 5 billion parameters per token, while the 20-billion model activates 3.6 billion. They also use sparse attention, grouped multi-query attention, RoPE positional encoding, and support context lengths up to 128k tokens.

• Benchmark results show strong reasoning, coding, science, health, and function-calling performance relative to OpenAI's smaller O-series models. Post-training included supervised fine-tuning and reinforcement learning focused on reasoning and tool use. OpenAI also evaluated malicious fine-tuning risks and announced a $500,000 red-teaming challenge to uncover additional safety problems.