Capturing the 3D world with a handheld camera

TUM in the media

12/9/2016

Imagine if you could use your smartphone to take pictures of static and moving objects and “re-construct” the real world into a 3D image – almost instantaneously. Instead of the usual 2D photos snapped inside Rome’s Sistine Chapel, you’d be able to create an immersive 3D tourism experience to share with friends. A plastic surgeon could use the technology to measure the progress of rebuilding a nose in real time during an operation. Or carmakers could utilize the technology to make autonomous cars safer and more reactive to their immediate environment.

Prof. Daniel Cremers, Chair for Computer Vision and Pattern Recognition. (Picture: A. Heddergott)

That’s the goal for Prof. Daniel Cremers, Chair for Computer Vision and Pattern Recognition with TUM’s Informatics department. Cremers’ trailblazing research into mathematical image pro-cessing and pattern recognition earned him the 2016 Gottfried Wilhelm Leibniz Prize – Germany’s most esteemed award in the sciences. His question: How can we use a camera to capture and “recover” the 3D world and reconstruct it in real time? The answer? It might lie in something called “Direct Image Alignment,” which is a core component of his current research into realizing the 3D world in images – faster, with greater accuracy and with more robustness.