BU Today

Science & Tech

Big Meaning in the Smallest Movements

Student software opens worlds for motion-impaired


Born with spastic quadriplegia and cerebral palsy, 49-year-old Rick Hoyt has never been able to speak or use his hands to write. But that doesn’t mean he can’t communicate.

Team Hoyt, Boston Marathon

Hoyt (SED’93) (left), who is best known for competing in more than 1,000 races, pushed in a wheelchair by his father, Dick Hoyt, is testing assistive technology developed through a decadelong collaboration between Margrit Betke, a College of Arts & Sciences associate professor of computer science, and James Gips, a Boston College professor of computer science, with help from more than 50 students.

Camera Mouse, a tool for people with medical conditions such as cerebral palsy, spinal muscular atrophy, ALS, multiple sclerosis, and traumatic brain injury, employs a computer webcam to lock onto and track a chosen section of the user’s face—a nostril or the tip of an eyebrow, for example—and then links that person’s head movement to a cursor on the screen. Move right and the cursor goes right. If Hoyt pauses for more than one second, dwelling over a button or link, it clicks to active.

Betke and fellow researchers have adapted Camera Mouse to work with several popular programs, such as Microsoft Word. They’ve also created custom software that allows computer users with disabilities to type emails, edit photographs, create music, and fight space aliens, among other activities.

Now two graduate students (below) in Betke’s lab, Chris Kwan (CAS’11, GRS’11) and John Magee (GRS’04,’11), have crafted companion software that piggybacks on Camera Mouse’s underlying technology.While testing Camera Mouse with Hoyt, Magee noticed that Hoyt moved his head more easily in a diagonal motion—a trend Magee sees in other people with motion impairments—making it difficult for Hoyt to propel the mouse pointer in a straight line. So Magee built an adaptive mouse functions software that would translate Hoyt’s actual movement into his intended movement, sending the mouse pointer straight across the screen and helping Hoyt to more easily hit his target commands.

Chris Kwan, UROP

Magee hopes to finish developing adaptive mouse functions and release it free on CameraMouse.org.

Hoyt has also tested Camera Canvas, software custom-built by Kwan, which received the 2011 Science Day Graduate School of Arts & Sciences Dean’s Award and is already free online. The software allows Camera Mouse users to edit photos and create illustrations, performing tasks that require physically complex mouse moves with traditional software.

Kwan, who began developing the software with classmates Won-Beom Kim (CAS’08) and Igor Fedyuk (CAS’08), and further refined it through the University’s Undergraduate Research Opportunities Program (UROP), hopes Camera Canvas will give people with motor impairments an outlet to express their creativity.

Working with cocreator Isaac Paquette (CAS’11), Kwan also developed a tool within Camera Canvas that could have much broader applications. The two developers created a menu whose settings, like button size and screen placement, can be modified based on users’ abilities. Called Menu Controller, it eliminates the need for pull-down menus with small, crowded entries that can be difficult to navigate with Camera Mouse.

Margrit Betke

Menu Controller “is an important development,” Betke (right) says, “because it can be applied to any commercially available software to convert it into a piece of software that can then be used by people with severe motion disabilities.”

Since 2007, Camera Mouse has been downloaded more than 350,000 times worldwide, from as far away as Greece, South Africa, India, Northern Ireland, Pakistan, Australia, and Malaysia. Magee believes this community will continue to grow as more software is built for use with Camera Mouse.

“These projects have the possibility to be used by a small group of people,” he says. “But the change that it can make in their lives, if you do a good job, is huge.”

Robin Berghaus can be reached at berghaus@bu.edu.


2 Comments on Big Meaning in the Smallest Movements

  • Anonymous on 06.02.2011 at 9:46 am

    Accessibility for visually impaired?

    These efforts are wonderful, I’m so glad to see them! I’ve been wondering for a while what measures BU takes to make sure its technologies are accessibility to visually impaired users. There is no accessibility statement on the homepage and when I searched, most of the results addressed accessibility of physical facilities more than websites etc.

  • Brittany on 06.03.2011 at 1:39 pm

    Great job!

    I just wanted to say I’m proud to call you guys my classmates! Good luck in the future, this program looks wonderful. You’re going to make a real impact in the everyday lives of those disabled. :)

Reply to Anonymous

cancel reply

(never shown)