Preferred Networks, Inc. (PFN) has developed a technology that converts photographs of buildings, interiors and other real-world locations into photorealistic, high-definition 3D data. Expected usage of the new technology includes video production for movies and advertisements, game development, promotions for tourism, real estate and more.

For details of the new technology and 3D data samples, please visit the webpage.

Example use case: In-camera VFX virtual production (Location courtesy: Kadokawa Daiei Studio)

Using this technology, content producers and developers can reconstruct a digital 3D model from hundreds of photographs taken from multiple angles at a real-world location thanks to PFN’s in-house supercomputers and 3D Gaussian splatting (3DGS), one of the latest 3D reconstruction techniques. PFN has also developed a dedicated plugin for Unreal Engine 5, a standard 3D engine commonly used for video production and game development, allowing users to take advantage of the technology in their usual production workflow.

By using 3D data created with the new technology, users can produce videos and interactive content in which they can freely move their viewpoints within the detailed 3D space reconstructed faithfully to the real-world location, without physically visiting the site for filming multiple times.

Sample videos of reconstructed 3D spaces:

Location courtesy: Royal Chester Maebashi wedding hall (Gunma, Japan), Kadokawa Culture Promotion Foundation’s Bookshelf Theater in Kadokawa Musashino Museum (Saitama, Japan), and more

Example use case in video production:

The interactive 3D background is projected on the large LED screen and mixed with a real-life person in front of it in real time

Going forward, PFN plans to apply the new technology to new products and services infor a range of domains including entertainment, real estate, tourism and manufacturing.