Skip to main content
Part of complete coverage on

A camera that can see around corners

By Ramesh Raskar, Special to CNN
August 19, 2012 -- Updated 1305 GMT (2105 HKT)
STORY HIGHLIGHTS
  • Ramesh Raskar's team at the MIT Media Lab is re-imagining what photography can do
  • By creating movies of light in motion at a trillion frames per second, camera can see around corner
  • Raskar: Technology is years away from applications in the real world
  • He says it could be helpful in preventing car crashes and aiding rescues

Editor's note: Ramesh Raskar is Associate Professor at MIT Media Lab and head of the Camera Culture research group. He is an Alfred P. Sloan Research Fellow (2009), a recipient of the DARPA Young Faculty award, the co-author of "Spatial Augmented Reality", and the holder of more than 50 US patents. He spoke at the TEDGlobal conference in June. TED is a nonprofit dedicated to "Ideas worth spreading" which it makes available through talks posted on its website.

(CNN) -- A camera that can see around a corner?

I know this sounds like something in a sci-fi movie or a superhero comic, but this is a real-world technology we've made possible with a camera that is aware of the travel time of light, an imaging technique that can create movies of light in motion with an effective rate approaching a trillion frames per second: the speed of light.

Before I joined the MIT faculty in 2008, I had done deep research in "computational photography," a field of new imaging techniques dramatically improving the capture and synthesis of photos. But, I knew there was more to photography than just depicting what the eye can see. I wanted to create a camera that could see beyond the line of sight. The speed of light isn't infinite: light travels about a foot per billionth of a second.

Photography could make X-rays obsolete

If I could build a camera fast enough to analyze light at high speeds in room-sized environments, I knew we could then create cameras to solve major problems in scientific and consumer imaging, and enable completely new functionality.

I spoke to top researchers in ultrafast lasers and photonics to understand what was currently possible. When I did, most of them asked some version of: "Why? Why spend years building a camera to look around corners when no commercial application is screaming for it and no funding agency has a call for it?" In addition, it's rare to shoot light pulses and analyze at such high speeds in large environments. Ultra-fast imaging experiments are usually limited to centimeter- or smaller-size samples.

I continued the work and in the spring of 2008, with James Davis from UC Santa Cruz, wrote a proposal that laid out the mathematical foundation and various experimental solutions for exploration.

TED.com: Boaz Almog 'levitates' a superconductor

I also began working with Media Lab graduate student Matthew Hirsch to build a prototype, hoping that we would have something to demonstrate within a few months. The grant proposal was rejected for administrative reasons (I made a paperwork error!), which meant we had to wait nearly a year to apply again. But those two years didn't yield any meaningful results, as our lab components weren't designed to be used the way we wanted.

After nearly three years of experimental work, the team -- especially postdoctoral associate Andreas Velten and MIT professor Moungi Bawendi, many students, and several collaborators -- cobbled together pieces of the puzzle and built a software program to create a first demonstration of looking around corners. Very soon afterward, we could also start creating surreal movies of light in motion.

We call this new imaging technology femto photography because we capture a segment of the image with a flashlight (in this case, a laser pulse) on for a few millionths of a billionth of a second (or a few femtoseconds) and an exposure time approaching a trillionth of a second.

Just how fast is femto photography? Think of it this way: if we took one-thousandth of a second of footage from the femto camera video output and slowed it down to the speed of 30 seconds per frame -- the approximate speed of a standard TV broadcast -- it would take us a lifetime to watch.

Photographers know that at very short exposures and even at the most sensitive setting for dark scenes, we will record barely any light. So what about in a trillionth of a second? We actually record and average millions of photos to get enough light, each photo made to look the same via carefully timed synchronization with the light pulse. So even if our exposure time is indeed nearly a trillionth of a second, to get sufficient light we must take an average. Thus, as of now, we can only record repeatable events, but this is not a fundamental limitation.

TED.com: A robot that flies like a bird

Unlike conventional cameras, our femto camera captures an image as one thin slice at a time of one-dimensional space using a "streak tube," a laboratory instrument that is commonly used by chemists to study light passing through chemical samples. We then take hundreds of these narrow videos (each shot at a slightly different angle) and create a carefully synchronized, slow-motion composite of the light pulses. It takes about an hour to collect and aggregate the data (view a demonstration of a light pulse as it travels through an ordinary Coca-Cola bottle).

To see around the corner, we use femto photography to analyze scattered light. We bounce light off of visible parts into hidden parts and then measure the time and direction of returned light.

Usually the scattering of energy is considered a nuisance -- whether driving in fog or poor reception from a cell phone tower -- and most techniques either try to avoid it (by turning on fog lights) or reduce the impact of scattering (by selecting energy for the phone only from direct paths).

In contrast, we exploit the scattering. For the camera, a laser pulse is fired at a wall, and the impact of hitting the wall causes the particles of light to scatter. Some of the scattering particles return to the camera at different times. This is repeated about 60 times per image as the camera measures how long it takes for the light to travel back and where the particles land. An algorithm then crunches the data to reconstruct the hidden image. This technique even allows us to see a three-dimensional object such as a mock-up of running person.

TED.com: Test for Parkinson's with a phone call

As exciting as this work is, don't look for this technology to be in practice tomorrow -- we're still years away from bringing this to market. But, we can already imagine multiple ways that it could have a significant, positive impact on our everyday lives.

By allowing us to "see" around a corner, for example, this technology added to our cars could let us know if there's another vehicle approaching around a blind curve. It also could give us a new way to look deep inside our bodies without X-rays, or to look through a window into a burning building from a distance to see if anyone is left inside --without risking a firefighter's life.

When I gave a TEDGlobal talk on femto photography in June, I began with a reference to Doc Edgerton, a very popular MIT professor of electrical engineering who, in 1964, wowed the world with an image of a bullet in midair, having just passed through an apple. He accomplished this by using a strobe light to freeze the action of the bullet at a millionth of a second.

What we're talking about here -- the speed of light --is a million times faster, and is opening the door to a complete rethinking of what we mean by, and can do with, photography. It is a first step toward a new world of imaging that far exceeds human ability, resynthesizing data and depicting it in ways that are within human comprehensibility.

Follow @CNNOpinion on Twitter

Join us at Facebook/CNNOpinion

The opinions expressed in this commentary are solely those of the author, Ramesh Raskar, who would like to thank his collaborators: Moungi G. Bawendi, Professor, MIT Department of Chemistry; Andreas Velten and Christopher Barsi, postdoctoral associates, MIT Media Lab; Everett Lawson, Otkrist Gupta, Nikhil Naik, Amy Fritz, Di Wu, MIT Media Lab; Matt O'toole, Kyros Kutulakos, University of Toronto; Diego Gutierrez, Belen Masia, and Elisa Amoros, Universidad de Zaragoza; Kavita Bala, Shuang Zhao, Cornell University; Ashok Veeraraghavan, Rice University.; Thomas Willwacher, Harvard University.

ADVERTISEMENT
Part of complete coverage on
April 16, 2014 -- Updated 1201 GMT (2001 HKT)
Rick McGahey says Rep. Paul Ryan is signaling his presidential ambitions by appealing to hard core Republican values
April 16, 2014 -- Updated 1539 GMT (2339 HKT)
Paul Saffo says current Google Glasses are doomed to become eBay collectibles, but they are only the leading edge of a surge in wearable tech that will change our lives
April 15, 2014 -- Updated 1849 GMT (0249 HKT)
Kathleen Blee says the KKK and white power or neo-Nazi groups give haters the purpose and urgency to use violence.
April 16, 2014 -- Updated 1156 GMT (1956 HKT)
Sen. Sheldon Whitehouse and Rep. Henry Waxman say read deep, and you'll see the federal Keystone pipeline report spells out the pipeline is bad news
April 16, 2014 -- Updated 1153 GMT (1953 HKT)
Frida Ghitis says President Obama needs to stop making empty threats against Russia and consider other options
April 15, 2014 -- Updated 2129 GMT (0529 HKT)
Peter Bergen and David Sterman say the Kansas Jewish Center killings are part of a string of lethal violence in the U.S. that outstrips al Qaeda-influenced attacks. Why don't we pay more attention?
April 15, 2014 -- Updated 1641 GMT (0041 HKT)
Danny Cevallos says families of the passengers on Malaysian Airlines Flight 370 need legal counsel
April 14, 2014 -- Updated 1523 GMT (2323 HKT)
David Frum says Russia is on a rampage of mischief while Western leaders and Western alliances charged with keeping the peace hem and haw
April 14, 2014 -- Updated 1156 GMT (1956 HKT)
Most adults make the mistakes of hitting the snooze button and of checking emails first thing in the morning, writes Mel Robbins
April 14, 2014 -- Updated 1754 GMT (0154 HKT)
David Wheeler says as middle-class careers continue to disappear, we need a monthly cash payment to everyone
April 14, 2014 -- Updated 1155 GMT (1955 HKT)
Democrats need to show more political spine when it comes to the issue of taxes.
April 14, 2014 -- Updated 1555 GMT (2355 HKT)
Donna Brazile recalls the 50th Anniversary of the Civil Rights Act as four presidents honored the heroes of the movement and Lyndon Johnson, who signed the law
April 14, 2014 -- Updated 1317 GMT (2117 HKT)
Elmer Smith remembers Chuck Stone, the legendary journalist from Philadelphia who was known as a thorn in the side of police and an advocate for the little guy
April 13, 2014 -- Updated 1856 GMT (0256 HKT)
Al Franken says Comcast, the nation's largest cable provider, wants to acquire Time Warner Cable, the nation's second-largest cable provider. Should we be concerned?
April 11, 2014 -- Updated 1522 GMT (2322 HKT)
Philip Cook and Kristin Goss says the Pennsylvania stabbing attack, which caused grave injury -- but not death, carries a lesson on guns for policymakers
April 11, 2014 -- Updated 1906 GMT (0306 HKT)
Wikipedia lists 105 football movies, but all too many of them are forgettable, writes Mike Downey
April 11, 2014 -- Updated 1432 GMT (2232 HKT)
John Sutter and hundreds of iReporters set out to run marathons after the bombings -- and learned a lot about the culture of running
April 11, 2014 -- Updated 1649 GMT (0049 HKT)
Timothy Stanley says it was cowardly to withdraw the offer of an honorary degree to Ayaan Hirsi Ali. The university should have done its homework on her narrow views and not made the offer
April 11, 2014 -- Updated 1416 GMT (2216 HKT)
Al Awlaki
Almost three years after his death in a 2011 CIA drone strike in Yemen, Anwar al-Awlaki continues to inspire violent jihadist extremists in the U.S, writes Peter Bergen
April 12, 2014 -- Updated 0121 GMT (0921 HKT)
David Bianculli says Colbert is a smart, funny interviewer, but ditching his blowhard persona to take over the mainstream late-night role may cost him fans
April 10, 2014 -- Updated 1731 GMT (0131 HKT)
Rep. Paul Ryan says the Republican budget places its trust in the people, not in Washington
April 10, 2014 -- Updated 2128 GMT (0528 HKT)
Aaron David Miller says Obama isn't to blame for Kerry's lack of progress in resolving Mideast talks
April 14, 2014 -- Updated 1522 GMT (2322 HKT)
David Weinberger says beyond focusing on the horrors of the attack a year ago, it's worth remembering the lessons it taught about strength, the dangers of idle speculation and Boston's solidarity
April 10, 2014 -- Updated 1632 GMT (0032 HKT)
Katherine Newman says the motive for the school stabbing attack in Pennsylvania is not yet known, but research on such rampages turns up similarities in suspects and circumstances
April 11, 2014 -- Updated 1103 GMT (1903 HKT)
Simon Tisdall: Has John Kerry's recent track record left Russia's wily leader ever more convinced of U.S. weakness?
April 10, 2014 -- Updated 1640 GMT (0040 HKT)
Mel Robbins says Nate Scimio deserves credit for acting bravely in a frightening attack and shouldn't be criticized for posting a selfie afterward
April 9, 2014 -- Updated 1839 GMT (0239 HKT)
Wendy Townsend says the Rattlesnake Roundup -- where thousands of pounds of snakes are killed and tormented -- is barbaric
April 10, 2014 -- Updated 1345 GMT (2145 HKT)
Dr. Mary Mulcahy says doctors who tell their patients the truth risk getting bad ratings from them
April 9, 2014 -- Updated 1328 GMT (2128 HKT)
Peggy Drexler says the married Rep. McAllister, caught on video making out with a staffer, won't get a pass from voters who elected him as a Christian conservative with family values
April 9, 2014 -- Updated 1143 GMT (1943 HKT)
David Frum says the president has failed to react strongly to crises in Iran, Syria, Ukraine and Venezuela, encouraging others to act out
April 9, 2014 -- Updated 2057 GMT (0457 HKT)
Eric Liu says Paul Ryan gets it very wrong: The U.S.'s problem is not a culture of poverty, it is a culture of wealth that is destroying the American value linking work and reward
April 9, 2014 -- Updated 1151 GMT (1951 HKT)
Frida Ghitis writes: "We are still seeing the world mostly through men's eyes. We are still hearing it explained to us mostly by men."
April 10, 2014 -- Updated 1408 GMT (2208 HKT)
Chester Wisniewski says the Heartbleed bug shows how we're all tangled together, relying on each other for Internet security
April 9, 2014 -- Updated 1926 GMT (0326 HKT)
Danny Cevallos says an Ohio school that suspended a little kid for pointing his finger at another kid and pretending to shoot shows the growth in "zero tolerance" policies at school run amok
ADVERTISEMENT