How to run six open source AI tools locally today

TL;DR

Unsloth, Diagram Design, Obsidian Skills, Buzz, Ego Lite, and Modly run locally and are open source. They cover local fine tuning, inference, diagram creation, note taking, agent collaboration, browser control, and image to 3D generation. All are cross platform and can operate without cloud dependencies, with many offering one click or simple install flows.

Transcript

Here are six incredible open- source projects that you can install right now. Let's get into the first one, Unsloth. It started as a way to make LLM fine-tuning locally much easier for the average person like me. Now, it has evolved into a powerhouse of a tool for everything from fine-tuning to running inference, training models, and everything in ... Read More

Key Insights

• Unsloth is an open source tool that simplifies local AI model fine-tuning and inference, featuring a point-and-click interface that reduces the need for coding.
• Diagram Design provides diagram creation capabilities with multiple chart types, and can be installed as a plugin in several agent environments for visual design workflows.
• Obsidian Skills connects agents to Obsidian markdown notes, enabling local knowledge bases and markdown driven workflows across different agent runtimes.
• Buzz is an open source Slack-like platform where agents are first class citizens, supporting private hosting, privacy, and built in automation workflows.
• Ego Lite is a fast browser automation tool designed for AI agents, allowing shared login states and rapid browser control with minimal setup.
• Modly converts images to 3D meshes locally, enabling asset creation for 3D printing or games without network dependence.
• The video highlights hosting and cloud options through Hostinger for running diagrams and agent tools, demonstrating practical cloud hosting integrations.
• All projects emphasize local-first workflows, cross-platform support, and varied capabilities from note taking to 3D generation to browser automation.
• More videos from this channel:
  • Is Gemini 3.8 Flash the Best Low-Cost AI Model?
  • What Is Jev and How Does Its AI Model Work?
  • How to visualize room designs with AI in minutes
• More from this channel: Matthew Berman

Questions & Answers

Q: How to run open source AI tools locally without cloud services

You run them on your own hardware or a local network, avoiding cloud dependency. Unsloth provides a local interface for fine tuning, training, and running inference, while other tools like Diagram Design, Ego Lite, and Modly operate entirely offline or with minimal cloud reliance. This setup enhances privacy and control.

Q: What is Unsloth used for in local AI workflows

Unsloth started as a way to simplify local fine-tuning of LLMs and has evolved into a comprehensive tool for fine tuning, running inference, and model training. It supports various hardware, offers a point-and-click UI, and allows remote access and memory integration, enabling full life cycle management of local models.

Q: How does Diagram Design help agents create diagrams

Diagram Design provides built in diagram options such as flowcharts, architecture charts, state machines, and timelines that agents can generate. It can be installed as a plugin in multiple environments, allowing agents to produce polished visuals that accurately depict architectures and processes without external drawing tools.

Q: What makes Obsidian Skills useful for AI agents

Obsidian Skills lets agents access and utilize Obsidian markdown notes, enabling a local knowledge base that agents can reference. This integrates markdown driven data with agent workflows, facilitating memory, context retention, and knowledge management within a local or hybrid setup.

Q: Why is Buzz described as agent native

Buzz is described as agent native because agents are treated as first class citizens within the platform, alongside human users. It supports self hosting, privacy, and embedded automations, creating a central hub where teams and agents collaborate, track work, and maintain an auditable log of actions.

Q: What is Ego Lite and how does it help AI agents

Ego Lite is marketed as the fastest browser for AI agents, designed to share a logged in browser state with agents like Codeex or Cloud Code without disturbing the user. It offers zero configuration, fast performance, and is open source, making browser automation practical for AI workflows.

Q: What does Modly do in AI toolchains

Modly is an open source project that converts images to 3D meshes locally. It enables asset generation for both 3D printing and game development, runs on Windows, Linux, and Mac, and works with desktop GPUs, providing a privacy focused, offline 3D design capability.

Q: How can hosting be used with these tools to improve accessibility

Hosting, as shown with Hostinger in the video, allows you to run diagrams and agent tools in the cloud while still keeping core AI processing local. This enables remote access, 24 hour background processing, and scalable deployment, while preserving the local first nature of the tools for sensitive data.

Summary & Key Takeaways

• Unsloth provides local fine tuning, training and running inference for various open source models, and adds an agent UI with memory and tools, all local. This enables end-to-end AI workflows without relying on remote servers.

• Diagram Design enables agents to generate clean diagrams such as flowcharts and architecture charts, installable across Cloud Code, Codeex, Pi, Hermes Agent, and can run in hosted or local environments.

• Obsidian Skills enhances agent capabilities by giving them access to Obsidian markdown notes, enabling local-first knowledge bases and markdown based workflows that integrate with various agent platforms.