How to compare OpenAI Decisions API with Jev

TL;DR

OpenAI Decisions API and Jev are similar in question types and outputs, but they differ in cost and multimodal support. Jev is cheaper for input tokens, while the Decisions API supports image inputs and may offer faster responses in some tests. Refusals exist with Decisions API, whereas Jev always provides an answer within given boundaries.

Transcript

A few weeks ago, the entire world lost their collective minds over Jev. And ever since, we've seen dozens of open-source interpretations of it. But today, OpenAI released their long-awaited decisions API, which is meant to be their take on a Jevlike service. So, in this video, I'm going to walk you through what the decisions API is, how it works, a... Read More

Key Insights

• The Decisions API accepts text or image inputs and returns a usable answer for apps.
• The API supports three question types: true/false, multiple choice, and rubric based scoring.
• The evaluation uses a consistent flow: model, evidence, and judgment for each question.
• Jev and Decisions API are similar in answer types, including probabilities and refusals.
• Decisions API can process images, giving it multimodal capabilities Jev does not have.
• Cost is a differentiator: Jev at 0.042 per million input tokens versus Decisions API at 0.10 per million input tokens.
• In tests, Decisions API is often faster but more expensive than Jev.
• OpenAI Decisions API defines refusals for some questions, unlike Jev which always answers within provided boundaries.

Questions & Answers

Q: A real search query about the video: How does the Decisions API handle image inputs compared to Jev

The Decisions API supports multimodal inputs, meaning it can process both text and images, whereas Jev handles only text in the demonstrations. In practical terms, this lets Decisions API evaluate visual evidence like photos or product images, and return a classification or judgment with a confidence level. This capability expands its applicability to real world workflows that rely on image data.

Q: What are the three main question formats used by the Decisions API and Jev

Both systems offer three primary question formats that guide how you query them. The first is a binary true or false type, sometimes called a predicate. The second is a multiple choice question where you select the best option from a set. The third is a scored or rubric based question where you define what levels or categories mean and the model assigns the appropriate level.

Q: How does the cost comparison between Jev and the Decisions API appear in the video

The video cites Jev costing 0.042 per million input tokens while the Decisions API is priced at 0.10 per million input tokens. Based on these figures, Jev is described as cheaper by about 58 percent in the contexts tested, which include similar tasks like classification and evaluation of evidence.

Q: What is a notable behavioral difference the video mentions between the two services

A notable difference is that the Decisions API can refuse certain questions, whereas Jev is described as always answering within the defined boundaries the user provides. This behavior is highlighted as a potential guard against abuse when processing sensitive or problematic inputs.

Q: In the demos, which capability uniquely showcases the Decisions API

The image inspection demo uniquely showcases the Decisions API’s multimodal capability. The system analyzes product images to categorize issues such as visible or non visible damage, or inability to assess, demonstrating a practical scenario where image input is essential and Jev cannot perform the same task.

Q: What kinds of comparisons are made in the head to head demonstrations

The demonstrations compare speed, cost, and accuracy across tasks such as an outage investigation and a business report check. Both systems are used with identical evidence and question options to ensure a fair comparison. The results show Decisions API often being faster but more expensive, with similar accuracy on several checks.

Q: What practical guidance does the video offer for choosing between the two services

The guidance suggests choosing Jev when cost is the primary constraint and multimodal capabilities are not required. If you need faster responses with image processing and you can accept higher cost, the Decisions API is recommended. The video notes that this landscape will continue to evolve as services update.

Q: What is the final takeaway from the video about the two services

The final takeaway is that for cheaper operation, Jev is preferable, while for multimodal support and potentially faster responses, the Decisions API is advantageous. The speaker frames the choice as a trade off between cost and capability, indicating that either option may be preferable depending on the user’s priorities and the data involved.

Summary & Key Takeaways

• The video introduces OpenAI Decisions API as a Jev like classifier that accepts text or images and returns usable answers for apps, with three question modes similar to Jev. It then compares head to head against Jev, focusing on cost, speed, and performance across multiple demos.

• In the demonstrations, image processing is limited to Decisions API, while Jev handles text only. The results show Decisions API can be faster in some cases but generally more expensive, with notable differences in multimodal capabilities and the handling of refusals.

• The conclusion emphasizes choosing based on priorities: cheaper option favors Jev, while multimodal support and speed favor the Decisions API, with the landscape expected to evolve over time.