Professional video camera viewfinder on a film set, illustrating Prism AI video generation with synchronized sound

A research team from Fudan University, Tencent Hunyuan and Zhejiang University has released Prism, an image-to-video model that generates synchronized sound (speech, sound effects and music) together with the picture, at native 720p, 1080p and 2K. The checkpoints are on Hugging Face under the MIT license, the code is on GitHub, and the paper, “Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training,” went up on October 4, 2026.

For creators, the license is the headline. Prism AI video is free to download and free to use commercially. The catch is hardware. Running Prism locally takes a single 80 GB data-center GPU at 720p, and four or more of them at 1080p or 2K. There is still a free way to try it today, plus two open alternatives that run on 24–32 GB consumer cards. This post covers all three.

Prism AI Video at a Glance

  • What it is: Image + text in, video with audio out. Your prompt tags the sound with <speech>, <sfx> and <music>.
  • What’s free: The weights (MIT) and code. There’s also a free community demo on Hugging Face ZeroGPU.
  • What it costs: $0 to try in the browser. Running it locally means renting or owning an 80 GB GPU (A100/H100 class).
  • Download size: About 143 GB for one checkpoint plus the MOVA base model it needs. The full repo is about 208 GB.
  • The catch: It’s a research preview. The quality claims come from the authors, nobody has benchmarked it independently, and there’s no ComfyUI node yet.

What Prism Actually Is (and Isn’t)

Prism is first a training method and second a model. Generating video and audio together at 2K is expensive because standard “full” attention compares every token with every other token, and the cost grows quadratically as resolution rises. Prism splits each clip into spatiotemporal “macro-zones” of 8×8×8 tokens. It checks how much each zone changes visually and how strongly the audio affects it, then gives busy or sound-driven regions (a mouth that’s talking, a drum being hit) finer attention blocks while static background gets coarser ones.

The authors report a 2.5× training speedup over full attention, with generation quality that they say beats full attention. That figure is the paper’s own measurement, and the abstract names only full attention as the baseline. The released checkpoints are built on OpenMOSS’s MOVA-360p, an Apache-2.0 video-audio model with two towers, one for video and one for audio. Two Prism variants are out: preview-alpha (labeled stable) and preview-beta (tuned for motion). A “Prism-pro” checkpoint is listed but hasn’t been released.

Prism dynamic sparse attention diagram showing macro-zone partition, block shape pool and top-k block selection for joint video-audio generation
Prism’s framework figure from the official model card. Each 8×8×8 macro-zone gets its own attention block shape, based on visual variance and audio-to-video attention strength.

What It Generates: Speech, Sound Effects and Music in One Pass

The official project page has 23 sample clips. They include dialogue in English, German, French, Portuguese, Russian and Malayalam, store and street ambience, engine roars, marimba over canal water, and two clips with singing. The prompts read like shot lists. They state the camera angle and lighting first, then describe the action beat by beat, with the audio wrapped in tags. For example: Continuous <sfx>sound of waves and water currents around the boat</sfx>, followed by the visual description and quoted lines inside <speech>.

The default output is 205 frames at 24 fps, roughly 8.5 seconds. The recommended sizes are 720×1280, 1072×1920 (1080p) and 1440×2560 (2K). The project page also shows side-by-side clips against Ovi, LTX-2.3, MOVA, MagiHuman, Kling 3, MiniMax-H3, Wan 3.0 and Seedance 2.5. Treat those as curated examples, not a benchmark. The page gives no scores, and the authors picked the prompts.

Opening frame of a Prism AI video demo of a boat moving along the Grand Canal in Venice, generated with synchronized marimba and water audio
The opening frame of one of Prism’s official demos: a low-angle boat shot on the Grand Canal. The prompt asked for “upbeat rhythmic marimba melodies and gentle canal water ripples.”

Prism vs. the Open Video-With-Sound Alternatives

If you just want open-weight video with sound, Prism isn’t the only choice, and for most home setups it isn’t the practical one. Here’s how it compares with the two models you’re most likely to run instead:

ModelFree?LicenseMax native resolutionMinimum GPU (official)ComfyUI
Prism previewWeights + free ZeroGPU demoMIT (base model MOVA is Apache-2.0)2K (1440×2560)1× 80 GB at 720p; 4+× 80 GB for 1080p/2KNot yet
LTX-2.5Weights + free Spaces/ColabLTX-2 Community License (free under $10M annual revenue)Up to 4K with the 2× upscaler pipeline32 GB+ per the official ComfyUI docs; INT8 ConvRot and NVFP4 files cut thatYes, official templates
Ovi 1.1Weights + free demo SpaceApache-2.0960×960, 5 or 10 s32 GB; about 24 GB with fp8 + CPU offloadNo official template
Open video-with-audio models compared, using each project’s official docs, as of October 11, 2026.

We covered another route to fast video with sound in our post on DMAD MiniMax-H3 4-Step. For older footage, see how LTX-2.5 handles AI video restoration and detail refinement.

Try It Free: Three Ways to Run Prism Today

1. In your browser (no GPU)

The fastest free path is the community Prism Video Audio Space by bibibi12345. It runs the preview_alpha checkpoint on Hugging Face ZeroGPU “xlarge” hardware (an RTX Pro 6000 with 96 GB). Here’s the click path:

  1. Sign in to a free Hugging Face account. ZeroGPU time is tracked against a daily per-account quota.
  2. Drop a reference image into Reference image (first frame).
  3. Write the shot in Prompt (video), with sound in <sfx>, <music> or <speech> tags. The Audio prompt box is optional.
  4. Choose 480p or 720p, set the frame count (25–121 at 24 fps) and the denoising steps (10–50), then click Generate.

Watch the quota estimate under the sliders. The default of 720p, 49 frames (2 seconds) and 20 steps shows “≈ 7.8 min” of GPU time, which counts as about 15.6 minutes of your daily quota because xlarge hardware is billed at 2×. On a free account, start at 480p with 25 frames and fewer steps.

2. Locally or on a rented GPU

You’ll need Python 3.10+, CUDA 12.4+, system ffmpeg and one 80 GB GPU for 720p. Install steps from the official repo:

git clone https://github.com/Tencent-Hunyuan/Prism && cd Prism
pip install torch==2.7.1 torchvision==0.22.1 torchaudio==2.7.1 --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt
pip install flash-attn --no-build-isolation
pip install "huggingface_hub[cli]"
# Grab only the stable checkpoint + MOVA base (~143 GB instead of ~208 GB)
huggingface-cli download FrancisRing/Prism --include "preview_alpha/*" "pretrained_models/*" --local-dir ./checkpoints
bash prism_infer.sh   # 720p, single GPU; use --offload cpu for the frozen encoders

For 1080p or 2K, use prism_infer_fsdp.sh on four or more GPUs and set ENABLE_TILING=true so the VAE decode doesn’t run out of memory. Frame counts must satisfy (n − 1) divisible by 4. The repo snaps other values automatically.

3. The 24–32 GB alternatives

On a 24 GB RTX 3090/4090-class card, use Ovi (Apache-2.0, free demo Space) with --cpu_offload --fp8. With 32 GB (an RTX 5090, for example), use LTX-2.5 through its official ComfyUI templates: open Templates, search “LTX-2.5,” and pick Image to Video. The docs list 32 GB+ VRAM for the standard BF16 files. The INT8 ConvRot files lower that. Both models generate sound alongside the video today.

Honest Trade-Offs

  • Not a home-GPU model. Even 720p needs 80 GB of VRAM. A 24 GB card won’t run it, and there’s no official quantized build yet.
  • Huge download. Each Prism checkpoint is 65.3 GB, and you also need about 78 GB of MOVA base weights.
  • Unverified quality. The 2.5× speedup and the “better than full attention” result are the authors’ own numbers. Artificial Analysis and other third-party leaderboards haven’t measured it yet.
  • The free demo is a community Space. It runs the official checkpoint, but the authors don’t maintain it, and the ZeroGPU quota runs out after a clip or two at 720p.
  • License fine print. Prism’s weights are MIT and MOVA is Apache-2.0, so commercial use looks clear. Still, check both license files before shipping client work, and get rights for any faces or voices you use as reference.
Opening frame of a Prism AI video demo of a surfer riding a turquoise wave, generated with acoustic guitar and ocean audio
Opening frame of another official Prism demo. Wide tracking shots of water and motion are where the higher native resolution is most noticeable.

Workflow Tips for Prism AI Video

  • Prototype the prompt on a cheaper model. Get the camera move and timing right in Ovi or LTX-2.5 first, then spend ZeroGPU minutes or rented H100 time on Prism.
  • Write the sound before the picture. The official prompts open with the audio bed (Continuous <sfx>…</sfx> or “No background music, quiet room”), then describe the shot. Copy that order.
  • Put dialogue in the order it happens. Place each <speech> line next to the action it belongs to (“Then he nods, saying…”) so the lip-sync has a clear moment to land on.
  • Rent for one session and batch it. Download only preview_alpha plus the base model to a cloud volume, then queue all your shots in one sitting. The 143 GB download is the slowest part.
  • Speed up local runs where you can. If you also run Wan- or LTX-family models, our SageAttention in ComfyUI guide covers a sparse-attention approach that’s usable on consumer GPUs today.

For a hands-on look at how open video-with-audio models hold up in practice, watch our LTX 2.3 Audio-to-Video honest review. LTX-2.3 is one of the models Prism compares itself against. More side-by-side tests are on the Creative AI Show YouTube channel.

FAQ

Is Prism AI video free?

Yes. The weights and code are free under the MIT license, and you can try it at no cost in the community ZeroGPU Space, within Hugging Face’s daily GPU quota. The real cost of running it yourself is the 80 GB GPU.

Can I use Prism commercially?

The Prism checkpoints are MIT-licensed, and the MOVA-360p base they need is Apache-2.0. Both are permissive licenses that allow commercial use. Read both license files and clear the rights to any people or voices in your reference material.

What are Prism’s system requirements?

Python 3.10+, CUDA 12.4+, ffmpeg, about 143 GB of disk for one checkpoint plus the base model, and one 80 GB NVIDIA GPU for 720p. 1080p and 2K need four or more 80 GB GPUs.

Does Prism work in ComfyUI?

Not yet. As of this writing there’s no official node, and inference runs through the repo’s prism_infer.sh scripts. If you need a node-based workflow with sound today, use LTX-2.5.

Accurate as of October 11, 2026. Checkpoints, demo quotas and leaderboard positions change quickly.

Categories:

No responses yet

Leave a Reply