What New Open AI Models Can Run Locally?

TL;DR

Gemma 4 brings native text, image, and audio processing to consumer hardware, with smaller models intended for phones, Raspberry Pi devices, and other edge systems. The broader roundup also covers local video editing, game-scene restyling, search-grounded image generation, controllable video effects, voice tools, Qwen releases, GLM-5V, Wan 2.7, and a reported Claude Code leak.

Transcript

AI never sleeps, and this week has been absolutely insane. Google releases a ridiculously powerful open-source model that can even run on your phone or computer. Claude Code just got leaked. We have an image generator and editor that works offline on just your phone, and it can generate images in seconds. ZAI, the company behind the open- source GL... Read More

Key Insights

• Gemma 4 is a family of open models released under the Apache 2 license, with native support for text, images, and audio. Its four model sizes are designed to cover edge devices, consumer computers, and systems prioritizing higher-quality output.
• The smallest Gemma 4 models have 2 billion and 4 billion parameters and are intended for phones, Raspberry Pi systems, and other edge devices. They bring vision and audio understanding closer to the user, supporting offline operation and near-zero-latency interactions.
• The 24-billion-parameter Gemma 4 model uses a mixture-of-experts design in which only 3.8 billion parameters are active during use. The 31-billion-parameter dense model instead prioritizes raw quality and intelligence by using its full parameter set.
• Gemma 4 supports long inputs through a 128,000-token context window on smaller models and a 256,000-token window on larger models. These capacities can accommodate substantial documents, audio, and smaller codebases within a single prompt.
• Void is an open video-editing model from Netflix that deletes specified objects or interactions using a prompt. It fills the resulting gaps and adjusts downstream events, such as keeping bowling pins upright when the thrown ball and people are removed.
• Generative World Renderer uses game-scene GBuffers containing RGB, depth, surface normals, base color, metallic properties, and roughness. This structured scene understanding enables text-directed restyling of gameplay, including changes to lighting, geometry, environments, fog, fire, embers, and other effects.
• GenSearcher is a model-agnostic framework that gathers web images and factual evidence before generating an image. The references help an image model represent unfamiliar characters, remote landscapes, temperatures, buildings, architects, completion years, and other details more accurately.
• TokenDial is a video-control system that adjusts targeted attributes through prompts and strength sliders. Demonstrations include increasing the smokiness of an explosion and making a campfire bluer, showing how creators can vary an effect without replacing the entire generation workflow.

Questions & Answers

Q: What is Gemma 4 and what inputs can it process?

Gemma 4 is Google’s family of open AI models released under the Apache 2 license. It natively processes text, images, and audio within one model. The roundup describes uses including coding, answering questions, real-time spoken interaction, camera analysis, video analysis, long-document processing, and offline operation on phones or computers, depending on the selected model size and available hardware.

Q: Which Gemma 4 model is suitable for a phone or edge device?

The 2-billion-parameter and 4-billion-parameter Gemma 4 models are presented as the options for phones, Raspberry Pi systems, and other edge devices. Their smaller size enables local text, vision, and audio processing with minimal latency. The larger 24-billion and 31-billion models offer more capability but target stronger consumer hardware rather than the most constrained mobile devices.

Q: How do the 24B and 31B Gemma 4 models differ?

The 24-billion-parameter Gemma 4 model uses a mixture-of-experts architecture, described as activating only 3.8 billion parameters during use to improve efficiency. The 31-billion-parameter model is dense and is positioned as the choice for maximum raw quality and intelligence. Both larger models have a 256,000-token context window for substantial code, documents, or audio.

Q: How large are Gemma 4 context windows?

The smaller Gemma 4 models are described with a 128,000-token context window, roughly corresponding to about 100,000 words or an hour of audio in the video’s explanation. The two larger models double that capacity to 256,000 tokens, allowing users to place smaller codebases, lengthy documents, or extensive audio material into a single prompt.

Q: How does Netflix Void remove objects from videos?

Void accepts an existing video and a prompt identifying an object or interaction to delete. It removes the requested content, reconstructs the missing visual regions, and attempts to keep the resulting scene natural and physically plausible. One example removes a ball and the people throwing it, causing bowling pins to remain upright instead of reacting to an absent collision.

Q: What hardware is needed to run Netflix Void locally?

Void divides its models into two passes with a combined size of 22 GB. The roundup says this requires a higher-end GPU for local execution. Netflix released the code and included installation and local-running instructions, but the stated model size means it is not presented as a lightweight phone or low-resource edge application like the smallest Gemma 4 models.

Q: How does Generative World Renderer edit video game scenes?

Generative World Renderer extracts structured graphics information called GBuffers from AAA game footage. That information includes RGB imagery, depth, surface normals, base colors, metallic properties, and roughness. Because the system understands these scene attributes, users can prompt changes to environments, lighting, geometry, fog, fire, embers, neon styling, firefly swarms, and other visual properties.

Q: How does GenSearcher improve factual image generation?

GenSearcher first searches the web for images and evidence relevant to the requested generation, then gives those references to an image model. This helps when the model may not know an unfamiliar character, remote location, temperature range, specific building, architect, or completion year. The framework is model-agnostic and can work with open-source or closed-source image generators.

Summary & Key Takeaways

• Google’s Gemma 4 family combines native text, image, and audio processing with models intended for hardware ranging from phones to high-end consumer computers. Four sizes address different needs, while long context windows and training across more than 140 languages support coding, document analysis, real-time conversation, camera analysis, and offline applications.

• Netflix’s Void removes prompted objects and interactions from existing videos, then reconstructs the missing content while preserving plausible physical consequences. Generative World Renderer instead extracts scene data such as depth, normals, base color, metallic properties, and roughness, enabling prompted changes to environments, geometry, lighting, fog, fire, and other visual effects.

• GenSearcher improves image generation by searching the web for relevant visual and factual references before producing an image. TokenDial offers slider-based control over video attributes such as smoke intensity or color. The wider roundup also introduces audio generators, DreamLite, Qwen models, OmniVoice, GLM-5V, Wan 2.7, and other research tools.