MIT Uses Math To Detect Micromotion in Video (And you can try it!)

So here's a fascinating use of science: MIT has developed an algorithm that can be run on a video stream to amplify micro movements that would otherwise be imperceptible to the naked eye. Now you might have caught this on the news recently, but when you dig down behind the catchy stories and the hype, the math and science behind this discovery is (naturally) quite amazing. (It is MIT after all.)

In a paper entitled: "Eulerian Video Magnification for Revealing Subtle Changes in the World" the researchers demonstrated how their method...
takes a standard video sequence as input, and applies spatial decomposition, followed by temporal filtering to the frames. The resulting signal is then amplified to reveal hidden information. Using our method, we are able to visualize the flow of blood as it fills the face and also to amplify and reveal small motions. Our technique can run in real time to show phenomena occurring at temporal frequencies selected by the user.
Applying their algorithm to a video sequence, they can filter colour variations as well low-amplitude movement. That's right, they used math to analyze video data and discover this hidden information. Apparently this isn't the first time this type of sequencing has been attempted, according to the paper, but these previous attempts used fluid dynamics and required a lot more precision. That precision would require a lot more computing power which would be cost-prohibitive for real-world applications. Their algorithm is much more forgiving and therefore requires a lot less computing power.

So, not only were they able to render a functional solution, but it's scalable and less costly than other prior attempts. (Plus they refer to it as "space-time video processing", which just oozes of sci-fi-ness.) And it doesn't require any special production techniques or equipment. In fact, they ran the algorithm on a scene from Batman, and suddenly Christian Bale's pulse and jaw motion appears vividly.

That's nice, you're thinking but what's it for? If you check out the video below, you'll see them highlight blood flow in a face to indicate human pulse (which would be useful to help diagnose medical conditions) and observe the breath patterns of a sleeping baby (which would be especially useful for nervous parents).

You know my favourite use though? Answering the question:
"Why do I need to learn (insert ambiguous math concept here)?"
Now we haven't actually had that question yet from the Dude, but at his age, we're always looking for opportunities to show him how science and math have real-world applications. Keeping him engaged at school is completely helped by seeing the curriculum in the real world, and I don't mean in calculating the angles of street signs or calculating the cost of a ridiculously large order of lemons (seriously, text books need to get real) but by showing him innovations in technology that make the world a better place, and then ta-da-ing him with the "that was done using math!" angle. (Maybe we're trying to avoid that question pre-emptively, and you know what? so far, so good :)

Take a Look and Give It a Try!

If you're curious, check out the video below to see the tech in action, visit the project site, download the report or even try it for yourself! Quanta Research Cambridge developed a web app called Videoscope, where you can upload your own video and see what the algorithm picks up! You can even customize the amount of magnification that it applies.

If you feel so inclined to attempt to replicate their results, all of the raw data is available for download. (Don't worry, the MIT folks have submitted a patent, so no one can commercialize this technology without their permission.)

Take a look, it is quite remarkable. I feel like I need to preface this with a sideshow-style: Prepare to see what you have not been able to see before! ;)

(Source: @kiamousavi via Facebook)

Popular posts from this blog

Designing the team experience: Building culture through onboarding (Slides from PPPConf, Chicago 2018)

UX Theatre: Are You Just Acting Like You're Doing User-Centered Design?

UX Theatre: The Poster