Gaussian Splatting: Innovative 3D Rendering Techniques Showcased in AI Experiment

Alex Carlier’s experiment yielded captivating results – when the camera’s perspective is changed, a mesmerizing transition effect occurs, smoothly shifting one image into another. The splats demonstrated their ability to seamlessly display distinct images from varying angles, showcasing the potential of this innovative technique.

The practical implications of this experiment extend beyond mere experimentation. The implementation of Gaussian Splatting rendering has been integrated into the renowned NerfStudio library, adding to the array of tools available to graphic enthusiasts. NerfStudio stands as a comprehensive platform offering a simplified API for creating, training, and testing NeRFs (Neural Radiance Fields). By modularizing each component, the library ensures a more user-friendly and interpretable implementation of NeRF technology, promoting exploration and creative expression.

The collaborative spirit of this initiative shines through as NerfStudio emerges as a contributor-friendly repository. The project aims to foster a community where users build upon one another’s contributions, driving innovation and advancement in the field. Originally introduced as an open-source project by students at Berkeley AI Research (BAIR) in October 2022 as part of a research endeavour, it has continued to evolve with contributions from Berkeley students and the wider community.

A captivating illustration of the algorithm’s prowess lies in a video that showcases the algorithm’s ability to reconstruct a 3D scene using images captured by a drone. This remarkable achievement is rooted in the recent reinvention of a seemingly “old” neural rendering technique.

This innovative technique involves utilizing video data through Structure from Motion (COLMAP) to extract a point cloud. Subsequently, a cluster of small translucent Gaussians is initialized over the point cloud. These Gaussians are meticulously optimized to ensure accurate restoration of the original frames after rendering. The result? A vivid, immersive 3D scene that can be navigated in real-time.

While this method may sound cutting-edge, it bears a resemblance to the 2019 Neural Point-Based Graphics approach, which involved training flat ellipsoids for each point in a similar manner. This technique’s simplicity proves to be its strength, enabling both efficient learning and rapid rendering.

Read more about AI: