View All Stories View All Stories (Latest Issue)


View All News


To the casual passerby, Janusz Konrad may seem a bit fanatical about tai chi: standing in his office, waving one arm, then spreading both arms and bringing them together. Duck inside, however, and you’ll notice he’s not stretching for his health; he’s stretching for a camera, and images on a computer monitor are responding to each gesture—zooming in and out of photos or leapfrogging through a photo series.

Konrad, a College of Engineering professor of electrical and computer engineering, and Prakash Ishwar, an associate professor, designed the computer’s algorithms to recognize specific body motions. They’re not making video games. This, they hope, is the future security portal to your smartphone, tablet, laptop, or the locked door: software programmed to recognize a gesture, from your torso, your hand, or perhaps just your fingers.


ENG professors demonstrate the computer algorithms they are developing to recognize specific body motions. Video by Alan Wong

Armed with an $800,000 grant from the National Science Foundation and collaborating with colleagues at the Polytechnic Institute of New York University, the BU duo is developing algorithms for ever-smarter motion sensors. In doing so, they have to thread a tricky technological needle. “On the one hand,” says Ishwar, “you want security and privacy; nobody else should be able to authenticate on your behalf” by aping your gesture. On the other hand, if the system demands a perfectly precise gesture, you may have to flail your arms or other parts 10 times to get into your own account. “That’s annoying,” he says.

A workable system must be able to screen out distractions, like the motion of someone moving behind you or of the backpack you’re wearing, or changes in ambient lighting.

Yet using gestures as keys to cyberlocks would have some great advantages.

A gesture, like a lateral swipe of your hand, has “subtle differences in the way people do it,” Ishwar says—and people vary in arm length, musculature, and other traits that might help a detector distinguish between you and Arnold Schwarzenegger or Elle Macpherson. True, gestures aren’t as unique as fingerprints or as irises, for which there are authentication scanners. But unlike those traits, which theoretically are vulnerable if someone hacks the database storing them, an authenticating gesture that’s been compromised by an impostor can be replaced immediately. Getting a new fingerprint? Well, “you wouldn’t like it,” says Ishwar.

Security passwords pose another problem: the most effective ones tend to be inconveniently complex. Konrad surveyed one of his classes and found that no one used a smartphone passcode longer than four digits. An effective motion sensor could “simplify, make more secure and more pleasant the process of logging in,” he says. He and Ishwar are developing gesture-based authentication algorithms to be test-run on Microsoft’s motion-sensing Kinect camera, used with the Xbox video game console and the Windows computer operating system. “It can track your body,” Ishwar says, “get some skeleton approximation for your body, and then that information is provided to you in some real-time format.”

They also hope to use start-up company Leap Motion’s smaller motion-sensing device for notepads and laptops. The company claims that its device, the size of an iPod, will be able to read “micro-motions of your fingers,” says Konrad. In the next three to four years, “we want to develop something that’s extremely simple, inexpensive, and can be embedded into other products and could be used daily by millions of people.”

One thing that’s clear is that certain body parts, like hands, lend themselves to identity authentication better than others. “The degree of freedom that you have with your hands is significantly higher,” Ishwar says. “Maybe if I’m a yoga master, I can move my right leg and put it across my left shoulder, but most people can’t do that.” They’d like to experiment also with the torso, says Konrad, since people’s posture can vary. Then there’s Leap Motion and its potential finger recognition.

“We plan to involve more and more body parts” as the research progresses, Konrad says. If that sounds vaguely Frankenstein-ish, consider that today’s security technology already involves fingerprints, iris scans, and face recognition. “Wouldn’t it be nice,” muses Ishwar, “if we could do that using our everyday body language or gestures?”