Mira-Scene: The Free Open-Source Image to 3D Scene Model (And the No-GPU Way to Try It Today)

VAST’s research team just open-sourced Mira-Scene, an image to 3D scene pipeline that takes one photo of a room, a café patio or a street corner and rebuilds it as separate, editable 3D objects sitting on a floor, lit by a generated environment map. The paper landed on September 20, the GitHub repo is still getting commits as of this morning, and the weights are on Hugging Face with no gate.

For set designers, game-level blockers, previs artists and anyone who has tried to turn a mood-board photo into a 3D layout, that is the interesting part: not one fused blob, but a sofa you can move, a lamp you can delete and a table that stays on the floor. The catch is that Mira-Scene is a research pipeline, not an app. Below is what it does, what it really needs, and the free way to try single-photo 3D in your browser today while you decide whether the full install is worth your afternoon.

At a Glance

• What it is: an open pipeline that reconstructs an editable 3D scene (per-object meshes, floor, environment panorama) from a single image.
• What’s free: code, the 6.4 GB inference checkpoint, a pretraining checkpoint, example cases and fine-tuning data.
• What it costs: nothing to download, but the default setup calls a hosted vision-language model and an image-generation API for two stages, and those need your own API keys.
• Hardware: Linux, NVIDIA GPU, CUDA toolkit. The mesh backends it relies on ask for 24 GB (TRELLIS.2) or 32 GB (SAM 3D) of VRAM.
• The one catch: the repo ships with no license file, so commercial use is unclear. Try it for learning and previs, not client deliverables, until that changes.

What Mira-Scene Does Differently in Image to 3D Scene Reconstruction

Most image-to-3D tools you have used, including the Pixal3D multi-view model we covered earlier this month, are built for one object on a clean background. Point them at a whole room and you get either a single melted mesh or objects floating at the wrong angles.

Mira-Scene attacks the layout problem. Instead of guessing each object’s position and rotation as a handful of numbers, it predicts a Canonical Coordinate Map (CCM): for every pixel of an object, where that pixel sits on the object in its own normalized space. Pairing that with a depth-based point cloud lets it solve pose by matching dense correspondences, which is far less fragile than regressing a rotation. A diffusion transformer generates object geometry and layout together, then a “support-aware” solver applies gravity so cups sit on tables and tables sit on floors.

The pipeline runs in seven resumable stages: segmentation (SAM 3 masks plus a scene graph), depth (Pixel-Perfect Depth or MoGe), CCM, mesh (SAM 3D or TRELLIS.2), floor, scene assembly and an equirectangular environment map. If you read our piece on Marigold V2 depth and normals, you will recognize the depth stage; Mira-Scene is what happens when you stack depth, segmentation and object generation into one scene builder.

The Benchmarks: Promising, but Self-Reported

The authors compare Mira-Scene against Meta’s SAM 3D on their own evaluation and report 3D-IoU of 0.727 vs 0.520, 2D-IoU of 0.783 vs 0.672 and lower Chamfer distance (0.021 vs 0.027). That is the “39.8% relative gain in 3D-IoU” in the abstract. They also point out they trained on roughly 80k scenes built by a fully automatic pipeline, versus SAM 3D’s million-plus manually annotated layouts.

Two honest notes. First, these are the authors’ numbers on the authors’ eval scripts; no independent leaderboard has measured them yet. Second, it is not a clean sweep: SAM 3D still edges it on Earth Mover’s Distance (0.163 vs 0.169). The layout gains look real and large, but treat “beats SAM 3D” as a claim to test, not a settled fact.

What You Actually Need to Run It Locally

This is where most coverage of research releases goes vague, so here are the specifics from the environment guide:

• Five separate Conda environments (segmentation, geometry, CCM, SAM 3D, TRELLIS.2), because the upstream projects need incompatible PyTorch and CUDA builds.
• Linux plus an NVIDIA GPU and CUDA toolkit. Several extensions (nvdiffrast, flash-attn, PyTorch3D) compile from source. The defaults target an A800 (compute capability 8.0); you change TORCH_CUDA_ARCH_LIST for your card.
• VRAM: Mira-Scene does not publish its own figure. Its mesh backends do: TRELLIS.2 requires at least 24 GB, and SAM 3D Objects asks for 32 GB. Plan on a 24 GB card as the floor.
• Downloads: the Mira-Scene pipeline/ checkpoint is 6.4 GB (12.8 GB if you grab the pretraining checkpoint too). SAM 3 adds about 6.9 GB, SAM 3D Objects about 13.2 GB, or TRELLIS.2-4B about 16.2 GB plus RMBG-2.0 and DINOv3.
• Gated models: SAM 3, SAM 3D Objects and DINOv3 require you to request access on Hugging Face first.
• API keys: automatic segmentation defaults to a hosted vision-language model for object naming, and the environment-map stage calls an image-generation API. Both need your own credentials.

Mira-Scene vs. the Free Alternatives

Try It Free: Single-Photo 3D Today, Mira-Scene When You’re Ready

No GPU, five minutes: open Meta’s Segment Anything Playground image-to-3D demo, accept the research-demo terms, upload a room photo, click the objects you want and it generates them as 3D. This is the closest free, in-browser taste of what Mira-Scene does, since SAM 3D is both its main baseline and one of its mesh backends. For a single hero prop with proper materials, drop the cut-out object into the TRELLIS.2 Space or the Hunyuan3D-2.1 Space and download the GLB. Both run on free ZeroGPU time with a daily quota.

The full Mira-Scene install (Linux, 24 GB+ GPU):

git clone https://github.com/VAST-AI-Research/Mira-Scene.git
cd Mira-Scene
MIRA_ENV_ROOT=/path/to/conda-envs bash infer_scripts/environments/create_envs.sh
python -m pip install -U huggingface_hub
hf auth login
hf download Yang-Tian/Mira-Scene --include "pipeline/*" --local-dir checkpoints/mira-scene
cp infer_scripts/config/example.yaml infer_scripts/config/local.yaml
python infer_scripts/pipeline.py --input my_room.png --output results --config infer_scripts/config/local.yaml --preflight-only

After create_envs.sh, run install_external_projects.sh on the GPU machine to build the CUDA extensions, fill in the checkpoint paths in local.yaml, and use --preflight-only to validate everything before spending GPU time. Drop the flag to run the real pipeline. The shortcut: the repo ships prepared cases in example_cases/, so you can start from --from-stage ccm --to-stage scene and skip the API-backed segmentation entirely on your first run.

Honest Trade-offs

• Licensing is unresolved. The GitHub repo reports no license and the model card sets none. The paper’s CC BY-SA license covers the paper, not the code. On top of that, SAM 3 and SAM 3D carry Meta’s SAM License, and the RMBG-2.0 background remover in the TRELLIS.2 path is non-commercial.
• “Free” has asterisks. The downloads are free, but the default segmentation and environment stages call hosted third-party APIs. You can review segmentation by hand in the web UI, but budget for API calls or stick to the example cases.
• Setup is a weekend, not a click. Five environments and several from-source CUDA builds. If you are on Windows or a Mac, this is not for you yet.
• Automatic segmentation makes mistakes. The authors themselves say it can miss objects or mislabel support relations and recommend reviewing masks first.
• No ComfyUI nodes or hosted demo at the time of writing.

Workflow Tips for Creators

• Feed it clean, wide shots. A mid-height, slightly wide photo with visible floor gives the depth and floor stages something to anchor to. Tight crops and extreme wide-angle lenses hurt layout.
• Generate your own input. Several bundled examples are AI-generated images. An open image model like the one in our Qwen-Image-2.1 breakdown can make a concept room you then block out in 3D.
• Fix the scene graph before the 3D stages. Spend two minutes in the segmentation UI marking what rests on what. It is cheaper than rerunning mesh generation.
• Use the resumable stages. --from-stage mesh --force-stage mesh rebuilds only meshes and the scene, so you can swap between SAM 3D and TRELLIS.2 without redoing depth.
• Treat output as blockout. Bring the meshes into Blender for scale checks and retopology; they are a layout head start, not final assets.

FAQ

Is Mira-Scene free?

Yes, the code and checkpoints are free to download from GitHub and Hugging Face. The default pipeline also calls hosted vision and image-generation APIs, which may cost money unless you use the bundled example cases or review segmentation manually.

Can I use Mira-Scene commercially?

Not with confidence yet. The repository has no license file, and several dependencies (SAM 3, SAM 3D, RMBG-2.0) carry their own restrictions. Check the repo for a license before using outputs in paid work.

What are Mira-Scene’s system requirements?

Linux, an NVIDIA GPU with CUDA toolkit, Conda, and realistically 24 GB of VRAM or more, since its TRELLIS.2 mesh backend needs 24 GB and the SAM 3D backend asks for 32 GB. Expect roughly 25 to 40 GB of model downloads, depending on the backend.

How can I turn an image into a 3D scene without a GPU?

Use Meta’s free Segment Anything Playground to generate 3D objects from a photo in your browser, or the free TRELLIS.2 and Hunyuan3D-2.1 Hugging Face Spaces for single objects.

Does Mira-Scene work in ComfyUI?

Not yet. It runs through its own Python pipeline and web UI. We found no ComfyUI nodes for it as of this writing.

Want to see how a room concept gets made before you rebuild it in 3D? Watch our Qwen Image 2.1 review, and find more hands-on tests on the Creative AI Show YouTube channel and at Creative AI Show.

Accurate as of September 28, 2026. Research repos, licenses and demo quotas change fast, so check the links before you commit a project to them.