Scientists have been trying for decades to build supercomputers that can replicate the brain’s information-processing abilities. But it has not been easy.
Computers operate sequentially, with one or several cores executing a preprogrammed set of instructions. But the brain operates quite differently, with many interconnected neurons processing highly parallel information that’s distributed throughout the neural network. For years, researchers in the field of neuromorphics—which involves developing computer architectures that process information in a manner inspired by the brain—have worked on bridging this gap between mind and machine. Among them is IEEE Graduate Student Member Sam Fok.
Fok and Alex Neckar [shown above, left to right], both doctoral candidates in electrical engineering at Stanford University, are developers of a neuromorphic device called the Neurogrid. It uses silicon neurons that perform computations by mimicking the way biological neurons function. Neurogrid simulates 1 million neurons and 6 billion synapses—the interfaces between neurons—in real time. “Using different parameters in the differential equations yields different spiking rates, so we have control over the spike rate and can make it vary between 0 and about 1 kHz,” Fok says. The neurons spike—or fire electrical impulses—at an average rate of 10 times a second. That means the Neurogrid’s simulations rival those done by such supercomputers as the one used in the Blue Brain Project, an ongoing effort by the Brain and Mind Institute of the École Polytechnique Fédérale de Lausanne, in Switzerland, to create a synthetic brain by reverse-engineering a mammalian brain to the molecular level. However, Fok and Neckar’s machine uses a millionth of the power needed for Blue Brain. The two were awarded a US $100 000 Qualcomm Innovation Fellowship in June for their work.
“Neurogrid’s purpose is twofold: to learn how neural networks in the brain work and how to harness these networks for tasks at which the brain vastly outperforms computers,” Fok says. “While computers have far exceeded humans in many tasks, we’ve yet to see computers even come close to human performance in tasks as simple as walking around.” Fok hopes Neurogrid can help change that. The fellowship money will go toward advancing the machine with help from researchers at Qualcomm, who will mentor the two.
Work on Neurogrid began in 2006 at Stanford’s Brains in Silicon lab, led by IEEE Member Kwabena Boahen, an associate professor of bioengineering and a pioneer in neuromorphics. The lab’s focus has been on simulating the brain using silicon chips, and members work on a variety of projects, including studying interesting single-neuron spiking patterns emerging from neural models and implementing large neural networks of biologically plausible neurons. This year, the group working on neural networks comprised Fok and Neckar. “In a sense, the fellowship really is a credit to Professor Boahen and the many previous graduate students and post-docs who worked so hard to develop the technology before we even arrived,” Fok says.
To develop a device that mimics the brain, it is, of course, necessary to understand some of the workings of the brain first. There are approximately 100 billion electrically excitable neurons in the brain to process information. “A defining characteristic is that they communicate with each other via electrical impulses, which are analogous to digital packets,” says Fok.
Over the years, neuroscientists have developed mathematical models to describe the behavior of neurons. The Boahen lab, building on this work, took the equations for those models and implemented them with analog very large-scale integration (VLSI) chips. “At the heart of Neurogrid are individual silicon neurons consisting of 337 transistors each,” Fok explains.
The group put 64 000 silicon neurons together like tiles on a silicon chip, and 16 chips were then networked together to create the Neurogrid. All told, the device has more than 1 million silicon neurons. A packet router system connects the chips and enables digital communication among the neurons. Fok and Neckar’s contributions to the Neurogrid included developing the capacity for arbitrary connectivity between silicon neurons using a bank of static random access memory. They also coded and implemented neural networks to demonstrate the mapping of arbitrary functions to a silicon neural substrate. The resulting device is extremely energy efficient, using less than 2 watts to perform a simulation.
“In a sense, we simulated the region of the brain controlling movement, known as the motor cortex,” Fok explains. “Specifically, neurons in the motor cortex are selective in their representation of, say, arm movement. A neuron will be active when moving the arm in a certain direction but not others. We used silicon neurons with similar directionally selective properties to represent the movement of a cursor.”
Fok and Neckar have high hopes for the Neurogrid, including applying it to brain-machine interfaces (BMIs), which could benefit from the device’s energy efficiency. “With BMIs, the heat produced by a device can damage the very tissue it interfaces with,” Fok explains. “Concurrently, the performance demanded by BMI applications requires signal-processing and communication techniques that only power-hungry digital processors provide.” He believes Neurogrid could be especially useful in this area over the next few years.
Fok also envisions many other applications. “I hope the lessons derived from Neurogrid will be used to design new, high-performance, low-power systems for tasks such as neural prosthetics and robotic controllers, as well as better neuromorphic platforms that can further improve our understanding of the brain,” he says.
Fok says receiving the fellowship has made him more confident about his work. “During the day-in, day-out travails of research, you often wonder when your work will bear results and whether it will ever affect anyone,” he says. “It probably takes a madman to work on something few others may value. But receiving the fellowship reaffirmed my belief that what we’re doing isn’t that crazy at all, but rather something worthwhile and meaningful that could well affect the future of computation.”