How to Build a Local Jev-like AI with Claude Opus

TL;DR

You can create a Jev-like classifier running locally with Claude Opus 5.5 by selecting an open source model, clearly defining the task, and benchmarking with a rubric. The method uses four steps: find a model, tailor the goal, run experiments, and add multimodal capabilities, achieving higher accuracy while keeping everything on your computer.

Transcript

So, I took the idea behind Jev and made a version that runs entirely on my computer completely for free. It even understands images, which is something that the current version of Jev doesn't support. And this isn't sped up. This is the actual speed. And Claude 5.5 Opus helped me build it. Now, imagine you run a travel agency. You've got 12 custome... Read More

Key Insights

• The project uses four core steps to reproduce Jev locally by selecting an open source model and fine tuning it for a specific task.
• The system relies on a clear goal definition and a rubric that measures accuracy and tolerance for mistakes to guide optimization.
• The model can be trained to classify multiple choice or binary tags with a rubric based approach, enabling flexible decision boundaries.
• Images can be used as a second source of evidence to improve classification confidence and correct potential mislabels.
• Open source repositories and variants are recommended as the backbone for extending the model, including options for multimodal analysis.
• The travel use case shows how to handle customer wish lists, budgets, and preferences to determine matches quickly.
• The process warns that retraining can take hours per run, making planning and version control essential.
• The absence of a traditional end to end pipeline is mitigated by using slash goals and modular architecture to isolate experiments.

Questions & Answers

Q: What is the main goal of the Jev like project described?

The main goal is to build a Jev like classifier that runs on a local computer, can analyze hotel and travel related information, and answer a set of yes no classification questions such as free cancellation, pool access, guided hikes, and accessibility. This is achieved by selecting an open source model, tailoring it to the travel domain, and using a rubric to measure accuracy, enabling practical, affordable local AI capability without cloud dependency.

Q: How does the video suggest you begin building this system?

The video recommends beginning by finding an appropriate open source model that can run locally, then explain to Claude the exact classification task and the goals. This starts the process of fine tuning the model to your use case and provides the foundation for subsequent experiments and benchmark setup. It emphasizes planning and clear brief for optimal results.

Q: What is Opus 5.5 used for in this setup?

Opus 5.5 is used to help build a local AI specialist by guiding the model selection, fine tuning on the travel related tasks, and adding an image understanding component. It also helps orchestrate experiments and prompts, making it possible to run multiple tests and gather results efficiently, ultimately aiming to reproduce Jev like performance locally.

Q: Why is a rubric important in this approach?

A rubric is important because it defines what constitutes acceptable performance, such as accuracy thresholds and how many mistakes are tolerable among the class of 12 travelers. It provides a measurable benchmark that the model must meet, guiding iterative improvements, and helping to avoid overfitting by evaluating performance on unseen data.

Q: What are the four core steps mentioned for building the system?

The four core steps are finding a model, explaining to Claude what you want to specialize in, running experiments with a defined rubric, and incorporating open source alternatives and image understanding. These steps structure the workflow so you can locate a suitable model, tailor it to the use case, validate with clear metrics, and extend capabilities with additional tools.

Q: What is Jev in the context of this video?

Jev is described as a classifier based model, specifically an encoder only model designed to understand information and bucket it into categories. The video contrasts it with decoder only or encoder decoder models to explain the design goal and the need for a clear classification task when building a local AI specialist.

Q: What role does image understanding play in this project?

Image understanding serves as a second evidence stream to support classification decisions. By analyzing photos such as pool pictures, the system can corroborate textual classifications, increasing confidence in the outputs and allowing the model to adjust its labels when visual cues contradict textual data.

Q: What is the expected performance improvement mentioned for the travel demo?

In the travel demo, the local model improved from about 60 percent to 95 percent accuracy, while Jev achieved around 98 percent. These figures illustrate the potential of a well configured local model to approach Jev level performance, given careful model selection, task framing, and robust benchmarking.

Summary & Key Takeaways

• This video demonstrates building a Jev inspired local AI travel assistant using Claude Opus 5.5, emphasizing open source models and a clear classification goal. It covers selecting a model, defining the labeling scheme, and setting up a rubric to measure accuracy. The result is a self contained system that runs on a typical computer and handles image input.

• The approach includes a practical four step process: choose a model, explain the specialization, benchmark with a rubric, and integrate open source alternatives. It also discusses managing versions and testing on unseen data to avoid overfitting. The travel scenario is used as a concrete example.

• The content highlights how to compare the local model to Jev, use image understanding, and keep a structured plan with prompts and prompts driven by Opus. It provides resources and prompts to replicate the setup, including a public starter repository and a guide for listeners who want to experiment themselves.