Startup to Outfit Self-Driving Cars With ‘Emotional Intelligence’

Braiq applies software to build greater trust between passengers and their vehicles

28 November 2016

Imagine getting into a self-driving car for the first time. As it accelerates, the way it drives could make you nervous. The car might, for example, move closer than you’d like to a cyclist or to the car up ahead, or it could go faster than you usually do. Although self-driving vehicles might be safer than cars driven by people, will that reassurance alone be enough for you to feel comfortable giving up control? The startup company Braiq aims to get passengers to trust their vehicles more.

Launched this year through Techstars Ventures, which provided Braiq with US $120,000 in funding, the New York City–based startup is developing a platform that takes advantage of technology already built into the vehicle, such as its suite of sensors and the passenger-facing camera that some cars have to detect distracted driving. In Braiq’s model, that camera also could sense if the owner became particularly nervous during a drive. The system then could adjust the car’s behavior accordingly. Braiq says its system will equip autonomous vehicles with “emotional intelligence”—the ability to recognize how people feel.

“As intelligent beings, we rely on our senses to gather data about one another: We process facial expressions and body movements through our eyesight, or use our hearing to distinguish a tone of voice,” says Braiq CEO Sameer Saproo. “We’re trying to do something similar for a car.”


The startup’s three founders are Saproo, an associate research scientist in Columbia University’s department of biomedical engineering; Victor Shih, the company’s CTO and a biomedical engineering Ph.D. student at Columbia; and IEEE Fellow Paul Sajda, Braiq’s science advisor and director of Columbia’s Laboratory for Intelligent Imaging and Neural Engineering. Sajda also chairs IEEE Brain, the initiative that is leveraging IEEE members’ expertise to advance worldwide efforts in brain-related research and technology.

While working on a project for the U.S. Defense Advanced Research Projects Agency that studied interactions between humans and machines, the trio recognized the importance of nonverbal communication in bringing people to trust artificial intelligence systems. “Subtle cues can go a long way,” Sajda says. That could mean having a machine back away on its own when a person feels it is uncomfortably close, or modifying a talking machine’s tone of voice to match that of the person it’s communicating with.

For its emotional intelligence system, Braiq relies on deep learning, an AI architecture that incorporates big data, supercomputing, and advanced algorithms so the machine can learn to make its own decisions. With deep learning, the machine observes the owner of the vehicle over time to learn her preferences, such as whether she likes a faster drive or is a cautious rider. It would then fine-tune the vehicle’s controls to adjust to her driving style. It also might learn other preferences, such as whether she generally prefers the music off or the sunroof open on a sunny day.

Because cars differ in what’s built into them, the platform is customizable to collect information from whatever sensors car makers have built in, and what aspects of  a driver’s style is to be personalized. “The more variety of biometric sensors we have, the better the car will be able to respond to the passenger,” Saproo says.

For the most part, Braiq’s ideas are in its laboratory. Its next step is to build relationships with carmakers, some of which are already equipping autonomous vehicles with enough features—like video cameras and seats that detect heart and respiration rates—to make Braiq’s program work. They well could add sensors and equipment to the mix if they choose to optimize the software, Saproo says.

Braiq also is planning ahead for a future when individuals don’t necessarily own autonomous cars but rely on ride-hailing services such as Uber and Lyft. In that scenario, an individual’s ride-preference profile could be stored in the cloud and downloaded to the autonomous taxi that arrives to pick him up.


Braiq is experimenting with incorporating data from an electroencephalogram that would monitor brain activity and lead to even more accurate observations about passengers. Carmakers, however, are not keen about asking their customers to wear EEG caps, Saproo points out. Instead, his team is working on integrating the sensors into headrests. With those in place, an autonomous car in the future could, for example, know which part of the sleep cycle a passenger is in, and learn to drive gently for a more restful nap. But, Saproo concedes, carmakers might decide that the cost of the additional neuroimaging sensors are not worth their benefits.

Although Braiq’s technology might start off as a “nice-to-have” option, Saproo says, it might become standard equipment if passengers turn out to have negative experiences in self-driving vehicles. “We don’t know how people will react until they start riding in them,” he says.

IEEE membership offers a wide range of benefits and opportunities for those who share a common interest in technology. If you are not already a member, consider joining IEEE and becoming part of a worldwide network of more than 400,000 students and professionals.

Learn More