How to Run Wan VACE 14B Offline with ComfyUI

TL;DR

Wan VACE by Alibaba is a free, open-source AI video generator that runs locally through ComfyUI, even on low-VRAM GPUs via QuantStack's quantized GGUF models. It supports text-to-video, image-to-video, and reference-video-to-video, letting you transfer motion from one video onto new characters or restyle scenes while keeping consistent characters.

Transcript

This is the most powerful AI video generator you can use offline. It's called Vase by Alibaba. And this is completely free and open source. And this uses Alibaba's Wand 2.1, which is the best open- source video generator you can use right now. In fact, it's one of the best generators out there, period. It's also the most uncensored, so you can gene... Read More

Key Insights

• Wan VACE is a free, open-source all-in-one AI video editor by Alibaba built on Wan 2.1, described in the video as the best open-source video generator available and also the most uncensored.
• The full VACE version has 14 billion parameters and generates video up to 1280 by 720 resolution, but it requires around 80 GB of VRAM, which most users do not have.
• QuantStack provides quantized GGUF versions of VACE 14B that lower the requirement, with models usable on as little as 8 GB of VRAM; the Q6 version needs only about 14.5 GB.
• VACE requires ComfyUI to run, and you install it by dragging a provided example V2V workflow JSON onto the canvas rather than building the node graph from scratch.
• Reference-video-to-video lets you transfer motion from a source clip onto new characters, so a dancing-ladies video can be mapped onto dancing cats, bears, or the same women in different outfits.
• You only need an image of your character plus a reference video with the desired movements to generate videos with consistent characters doing whatever action you want.
• A low-VRAM offload node lets users with limited GPU memory but high system RAM (e.g. 8 GB VRAM and 64 GB RAM) shift part of the computation to RAM by specifying available VRAM.
• An optional Wan 2.1 CosVid 14B LoRA can be toggled on to make generation roughly four times faster, and the Notta AI notetaker sponsored the video.

Questions & Answers

Q: What is Wan VACE and who made it?

Wan VACE is a free, open-source, all-in-one AI video editor made by Alibaba. It is built on Alibaba's Wan 2.1, which the video calls the best open-source video generator you can use right now and one of the best generators available overall. It is also described as the most uncensored, meaning you can generate a very wide range of content. Alibaba recently released the full version of VACE.

Q: How much VRAM do you need to run VACE 14B?

The full VACE 14B version has 14 billion parameters and generates video up to 1280 by 720 resolution, but it requires around 80 GB of VRAM, which most users do not have. Fortunately, QuantStack provides quantized GGUF versions that lower the requirement. Those models can run on as little as 8 GB of VRAM, and the creator uses the Q6 version, which needs only about 14.5 GB with his 16 GB card.

Q: How do you install VACE in ComfyUI?

You must have ComfyUI installed first. Download the quantized VACE model into the ComfyUI models unet folder, then drag and drop the provided V2V example workflow JSON onto the ComfyUI canvas. The first time you open it, many nodes appear red and missing, so open the manager and click install missing custom nodes, installing each one. After restarting, install any remaining nodes like the multi-GPU node, then restart again.

Q: What can reference video-to-video do in VACE?

Reference video-to-video transfers the movements from a source clip onto new characters or styles. In the demos, a video of ladies dancing is mapped onto cats or bears dancing, or the same women in bikinis at a beach. A boxing video's motion is transferred onto two men in suits fighting, anime girls with pink hair, or 3D Pixar-style kung fu masters, all matching the reference movements exactly.

Q: Can you run VACE if you have low VRAM but lots of RAM?

Yes. The workflow includes a node useful when you have low VRAM but high system RAM. For example, if you only have about 8 GB of GPU VRAM but around 64 GB of computer RAM, you can offset some of the computing to your RAM. You toggle the node on, select your VACE model, and specify how much VRAM your GPU has, such as eight, and the rest is offloaded to RAM.

Q: How can you make VACE generation faster?

There is an optional step that makes generation a lot faster. You toggle on a node and download an additional LoRA called the Wan 2.1 CosVid 14B LoRA using the convenient link provided, placing it in the ComfyUI models loras folder. This step is optional and can be left off, but leaving it on makes your generation around four times faster. You then select the CosVid LoRA in the workflow.

Q: What files do you need to download besides the VACE model?

After the quantized VACE model, you need three more things. Download the UMT5 XXL text encoder, choosing the file that matches your model; for the Q6 GGUF version you download the Q6 encoder into models text encoders. You also need the VAE (Wan 2.1) placed in the models VAE folder. The optional CosVid LoRA is the fourth item. Then select each downloaded model in the correct workflow node.

Q: What generation modes does VACE support?

VACE supports text-to-video, image-to-video, and reference video-to-video, plus combined video and image to video. Text-to-video generates from a prompt, image-to-video animates a supplied image, and reference video-to-video transfers motion from a source clip onto new characters or restyled scenes. For example, a drone video of a city can be turned into a cyberpunk city at night with neon signs or an abandoned city overgrown with plants, keeping the original camera motion.

Summary & Key Takeaways

• Wan VACE by Alibaba is a free, open-source AI video generator built on Wan 2.1, presented as the best and most uncensored open-source option. The recently released full version has 14 billion parameters and outputs up to 1280 by 720 video, but demands around 80 GB of VRAM.

• QuantStack offers quantized GGUF versions that run on far less hardware, from 8 GB of VRAM upward; the Q6 model needs about 14.5 GB. VACE runs inside ComfyUI, and users install it by dropping a provided V2V example workflow onto the canvas and adding missing custom nodes.

• The tool supports text-to-video, image-to-video, and reference-video-to-video. Motion from a reference clip can be transferred onto new characters or styles, drone footage can be restyled into cyberpunk or overgrown cities, and an optional CosVid LoRA speeds generation about fourfold.