Not many people can say they Skype and joke around with Stephen Hawking. But IEEE Member Lama Nachman has developed that rapport after working closely with him over the past three years to upgrade his communication device.
Living with amyotrophic lateral sclerosis (ALS), a neurodegenerative disease that causes muscle twitching, weakness, and speech impairment, Hawking relies on an Intel computer system to type and voice his thoughts as well as navigate computer applications and Internet browsing. Nachman, director of Intel’s Anticipatory Computing Lab in Santa Clara, Calif., leads the team that is improving this system to make it simpler and faster for Hawking to use.
Nachman has worked on intelligent wireless sensors for environmental and health monitoring since joining Intel in 2003. Using the technologies that can sense and predict users’ activities or the environment as tools to aid the disabled hadn’t crossed her mind until she was asked to work on this project in 2011, which she calls an eye-opener. “As I started working with Stephen Hawking, I realized there are so many technologies on the market for the general population that can be leveraged for people with disabilities,” she says.
Nachman is now making the Intel system open source so that researchers can expand it to help others with neurodegenerative diseases.
Because Hawking wanted to continue using the same equipment he’s had for more than 20 years, Nachman’s challenge was to update its functions. The original system allowed Hawking to, for example, select letters on a keyboard screen or click a mouse with a twitch of his cheek. This facial movement is picked up by an infrared sensor mounted on his glasses. For Hawking to “type” words, a cursor scanned letters on a screen. Hawking selects a letter by moving his cheek when the cursor hovers over the letter he wants, thus creating words one letter at a time.
The new system reduced the number of words Hawking needed to spell out completely by adding word-prediction technology that is used in smartphones. Working with the London-based company SwiftKey, Nachman’s team used Hawking’s lectures and writings to train word-prediction software. “If he spells out the word ‘black,’ the next word is likely to be ‘hole,’” Nachman says. “For the general population, it would be ‘dog’ or ‘cat.’” The software, which learns such preferences over time, has doubled his typing speed.
The new system has also sped up common tasks such as opening a document or browsing the Web. Hawking previously used a screen-scanning cursor to select a new tab or icon the same way he selects letters to sift through menus and toolbars.
“Instead, we created contextual menus specific to the application he’s using,” she says. For instance, the software knows he is in Microsoft Word and will suggest options to perform common actions such as opening a new document or saving the one he’s using. “It used to take him three or four minutes to open a document. Now the new interface allows him to do it in five seconds with one click.”
Nachman and her team made several trips to London to meet Hawking in person and spent hours on Skype calls with him to watch how he interacted with his equipment. The current version is a result of at least 70 iterations.
“Initially, I couldn’t believe I was working with him,” she says. “He’s brilliant but is limited physically. What I learned from him is patience. He’s also a perfectionist. He’d find a bug in five minutes that we wouldn’t spot in a week. I joked with him that I’d give him a job in my group as a system validation engineer.”
While the current platform is tailored to Hawking, Nachman says it should be easy to adapt for others by making the word-prediction software more general and expanding the motion-sensing capabilities to detect movements beyond a twitch of the cheek. Her team is now working on a facial-gesture recognition tool so that users can, for example, choose an application or open a new document using various facial expressions.
Growing up in Kuwait, Nachman was the technology whiz in her family. “When someone needed to figure out how a gadget worked, they came to me,” she says.
Her love of math and science led her to the United States to earn bachelor’s and master’s degrees in computer engineering from the University of Wisconsin-Madison, which she received in 1992 and 1995.
She joined Intel as a computer architect out of graduate school, but left in 1999 to work in startups. “I was itching to do something different,” she says.
Nachman returned to Intel in 2003. Her first project involved developing wireless sensor networks to monitor water pipelines, predict heavy equipment failure, and detect earthquakes. Three years later, she became manager of Intel’s sensing and context team. Long before the rise of fitness trackers, her team developed wearable health sensors and fitness-coaching apps for people with pre-diabetic conditions. The system logged diet details and used heart rate and accelerometer data to measure users’ energy levels while they were both active and sedentary. It presented the information to patients and their physicians in an accessible way to help people meet their health and fitness goals.
Her sensor research took a new direction when she formed the company’s Anticipatory Computing Lab in 2009. Sensors have shrunk in size and cost in recent years, while computing capabilities have increased. This makes it easy to use smartphones and other devices to get a higher level of understanding of the user’s health, interests, and behavior through sensors and machine learning.
Working on Hawking’s communication system might seem like a big leap from monitoring pipelines, but there’s a common thread. “My focus is on making sense out of sensor data,” she says. “I’m trying to make intelligent technologies and devices that really understand what’s taking place, anticipate needs, and act ahead of time.”