3D Gaussian Splatting (3DGS) turns captured or generated imagery into photorealistic 3D world simulations that users can freely explore, yet these worlds remain silent. Because existing audio generation methods condition on a single image or viewpoint, their sound is tied to that observation and cannot stay consistent while a listener moves. We introduce the task of generating a spatially consistent soundscape for a given 3DGS world through auditory grounding, identifying which objects in the world should emit sound and anchoring each to a persistent 3D position, and present Scene2Sound, a training-free soundscape generation framework built on this grounding. From the input world alone, our pipeline selects viewpoints that jointly cover the scene, identifies sound-emitting objects with a vision-language model, and associates the multi-view detections into 3D instances through Gaussian set matching, which measures the overlap between the Gaussian sets that render each detection. Each source then receives generated audio that a standard object-based audio engine spatializes in real time at arbitrary listener poses. We further propose two metrics that evaluate the spatial consistency of a generated soundscape, one testing whether rendered audio responds consistently to listener motion and one testing whether the claimed sources are supported by views held out from their placement. On a curated set of generated 3DGS worlds and on 3DGS scenes generated from real-world 360-degree captures, Scene2Sound preserves the audio quality of strong per-viewpoint baselines while remaining spatially consistent where per-viewpoint and single-panorama pipelines do not, and a user study confirms the perceptual benefit.
Given a 3D Gaussian Splatting world, Scene2Sound selects a small set of viewpoints that jointly cover the scene and asks a vision-language model to identify candidate sound-emitting objects in each view. Gaussian set matching then associates detections across views into 3D instances by measuring how much of the underlying Gaussian set each detection shares, which anchors every accepted source to a persistent 3D position instead of a single image. Each anchored source receives generated audio that a standard object-based audio engine spatializes in real time as the listener moves through the world.
WASD to move, mouse to look around. Headphones are recommended for spatial audio.
Click a scene to play its generated soundscape with sound. Only one video plays at a time.
Loading gallery…
Each panorama is overlaid with the 3D instances that Gaussian set matching associates across views. Click any image to enlarge.
Panoramas rendered from the five viewpoints selected by each candidate sampling strategy (greedy selection, K=5 from N=20 candidates). Raycast-based sampling places cameras inside the scene's hollow structure, yielding interior views that cover diverse scene elements, while AABB Uniform, FPS Floor, and Density Peak often place cameras outside the scene boundary or in low-content regions. Click to enlarge.
@misc{yoshida2026scene2sound,
title={Scene2Sound: Auditory-Grounded Soundscape Generation for 3D Gaussian Worlds},
author={Yoshida, Masaki and Togo, Ren and Ogawa, Takahiro and Haseyama, Miki},
year={2026},
eprint={XXXX.XXXXX},
archivePrefix={arXiv},
primaryClass={cs.CV}
}