April 14, 2021 7 min read
Opinions expressed by Entrepreneur contributors are their own.
” Star Wars: A New Hope “, the 1977 classic directed by George Lucas , was, for many of us, the first story that brought us closer and amazed by showing us a hologram .
” Help me Obi-Wan Kenobi, you are my last hope, ” said the hologram of Princess Leia projected by R2D2 in the installment of ” Star Wars: Episode IV – A New Hope. ”Without a doubt, a memorable scene.
The truth is that this innovative technology has transcended screens and science fiction and is currently being developed successfully thanks to advances in Artificial Intelligence . Until today, virtual reality headsets have not brought down television screens and computers as a reference device for watching videos. A reason? virtual reality can make users feel bad. While users are staring at a fixed distance 2D screen, virtual reality creates the illusion of a 3D screen, which can lead to nausea and eyestrain. A solution to achieve a better 3D visualization would be precisely the holograms , a 60-year-old technology redesigned for the digital world.
Holograms provide an excellent representation of the 3D world around us , because they offer a perspective that adjusts to the visual position of each viewer.
Researchers had spent years investigating how they could generate these computer holograms. However, the process was extremely complex as they required a supercomputer that could perform physical simulations.
Now, with the new technologies that have been developed and with the new findings in the digital field, researchers from the Massachusetts Institute of Technology (MIT) , with the collaboration of Sony , have managed to produce holograms almost instantly using a methodology based on the deep learning .
“People previously thought that with existing consumer hardware, it was impossible to do real-time 3D holography calculations,” says Liang Shi , lead author of the study and a PhD student in the Department of Electrical Engineering and Computer Science. (EECS) from MIT.
The development of this new method is called tensor holography , which allows the creation of holograms for virtual reality , 3D printing , medical imaging and more .
This deep learning- based technique is so efficient that it can “be run on a simple computer in the blink of an eye,” say the researchers.
For the study published in Nature , Shi worked with his advisor and co-author Wojciech Matusik . Other co-authors include: Beichen Li of EECS and MIT’s Laboratory of Computer Science and Artificial Intelligence, as well as former MIT researchers: Changil Kim (now on Facebook ) and Petr Kellnhofer (currently at Stanford University ).
Towards real-time 3D holography with deep neural networks
Image: Michael Dziedzic via Unsplash
First of all, it should be noted that the science behind holograms is different from that of a photograph. While the latter encodes the brightness of each light wave, the former encodes the brightness and phase of each light wave, thus offering a more faithful image.
Holography generated by computer aims to simulate an optical configuration, but the problem is that the process is a computationally hard work. This is because each point in the scene (or image) has a different depth, so the same operations cannot be applied to all.
This difficulty led Shi’s team to take a different approach: letting the computer learn physics on its own .
To achieve this they used deep learning , which allows them to accelerate computer-generated holography, thus allowing the generation of holograms in real time .
The scientists designed a tool based on a processing technique that uses a chain of trainable tensors to mimic the way humans process visual information, called a convolutional neural network . Training these types of networks requires a large, high-quality set of data.
For this reason, the research team designed a custom database of four thousand pairs of computer-generated images. Each pair matched an image, including each pixel’s color and depth information, with its corresponding hologram.
To create the holograms in the new database, the researchers used new scenes that had complex and variable shapes and colors, with the pixel depth evenly distributed from the background to the foreground, and with a new set of calculations based on the physics to manage occlusion. That approach resulted in photorealistic training data.
Subsequently, the algorithm was put to work. He learned each pair of images and modified the parameters for his own calculations, successively improving his ability to create holograms. The fully optimized network operated orders of magnitude faster than physics-based calculations.
This study demonstrated that, in just milliseconds, tensor holography has the potential to create holograms from images with depth information, which are provided by typical computer-generated images and can be calculated from a multi-camera setup. This is precisely the breakthrough that paves the way for the real-time 3D hologram .
Notably, real-time 3D holography could enhance a host of systems, from virtual reality to 3D printing .
The team led by Shi highlights that the new system could help immerse virtual reality viewers in more realistic scenarios, while eliminating eyestrain and other side effects of prolonged use of the virtual environment .
3D holography could also drive the development of volumetric 3D printing, as the technology could be faster and more accurate than traditional layer-by-layer printing.
As Princess Leia told Obi-Wan , ” Technology and innovation are the only hope .” And, at this rate, it is quite likely that before long we will communicate through hologram interfaces similar to the ones we saw in the Star Wars movies .