There’s a lot of concern that designers of autonomous and intelligent systems (A/IS) will overlook the impact their creations could have on society. That’s why the IEEE Standards Association and the MIT Media Lab recently launched the Council on Extended Intelligence (CXI).
The group seeks to foster the responsible creation of intelligent systems by recognizing they are part of larger human and environmental systems that need to be handled in a holistic way. The idea is encapsulated in the phrase extended intelligence and moves away from the “us versus them” mentality in the “robots versus humans” language found in many news media articles.
“Extended intelligence recognizes that we exist in a system with institutions, the natural environment, and other people,” says Joi Ito, the Media Lab’s director. “This way of thinking encourages us to address some of the more fundamental problems of inequality and other dehumanizing elements of society so that the machines can help us, instead of making things worse.”
CXI has identified three areas that it believes need a concerted global effort: responsible participatory design, ways for people to reclaim their digital identity, and new metrics for determining economic prosperity.
CXI wants A/IS developers to incorporate the principles of participatory design into their projects. This approach actively involves all stakeholders in the design process, according to Andre Uhl, a research associate with the Media Lab director’s office. Stakeholders include employees, customers, citizens, and end users, but Uhl says they also can include cultural legacies and the environment.
“We’re going to orient ourselves toward principles of participatory design to integrate humans, technologies, and the environment into symbiotic systems that enable all of these actors to cooperate with each other,” he says. “If we want to restore human well-being, we need to ensure that the entire complex adaptive system—the extended intelligence that we’re part of—flourishes.”
RECLAIMING DIGITAL IDENTITY
Widespread surveillance of citizens—combined with social-engineering techniques—has eroded trust, says John Havens, executive director of CXI. He is also executive director of the IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. People no longer have control over their identity or their data—which is fundamentally at odds with an open and free society, Havens says. The situation, he adds, can lead to authoritarianism and the proliferation of systems that reinforce systemic biases rather than correcting them.
“We don’t control our personal data—which means that others can control it,” he says. “Some of this may feel innocuous. For example, if an advertiser believes you prefer one brand of shoes over another, that’s not the end of the world. But when facial-recognition systems that are tracking you in a public place could falsely identify you, the ability to reclaim your digital identity is essential.”
Ito adds that the notion of digital identity is connected to the social processes that could perpetuate historical biases. Heart disease kills more women than men, for example, but women have been largely excluded from clinical trials. Even today they receive less than a third of all diagnostic tests.
Ito says he’s concerned that when machine-learning predictive algorithms use datasets based on the results of studies that have primarily focused on heart disease in men, they could make inaccurate assumptions about women’s heart health.
CXI’s goal is to support the use of tools from fields including statistics, machine learning, and causal inference to understand, expose, and tackle any inequities that are present in the collection and use of large-scale personal data.
“We need to think about this as a complex adaptive system that has technical, social, and political layers,” Ito says.
RETHINKING METRICS FOR SUCCESS
The widely used gross domestic product (GDP) is no longer a sufficient indicator of societal prosperity, economic growth, and productivity, many experts say.
Although governments often use GDP to determine citizen happiness or well-being, Havens says, the statistic largely focuses on a country’s overall wealth and productivity. It does not take into account environmental health, education, caregiving, or similar factors.
Measuring human and environmental well-being requires new metrics or indicators to determine holistic societal benefits, CXI says.
“If the prioritized metric of success for society is profit and growth as exemplified in GDP, then there’s no compelling business or economic reason for not replacing humans with machines,” Havens says. “We need to redefine progress for the algorithmic age. To say that increasing only economic factors such as profit or growth means everyone will be happy or prosperous in a specific country is inaccurate and myopic.”
With the current prioritization of short-term gains, autonomous and intelligent technologies are likely to increase inequality and social tensions, he says, and they are apt to further concentrate wealth and power among an ever-smaller class of privileged people.
More enlightened indicators, he says, would be the Better Life Index from the Organisation for Economic Co-operation and Development or the United Nations’ sustainable development goals, which include a range of quantitative metrics to gauge social progress.
“Our goal,” he says, “is to give policymakers a simple tool kit or set of recommendations to help them know which existing metrics favor this more holistic view of wealth and worth, and how to use them with autonomous and intelligent systems.”
NEW WAY OF THINKING
CXI is creating templates to teach people what extended intelligence, participant design and system thinking mean. It encourages thought leaders from religious and spiritual communities as well as representatives of indigenous cultures to join the council—to ensure that technologies will transform societies in ways that are positive, and beneficial to all.
To learn more, view the “Reversing Reductionism With the Council on Extended Intelligence” webinar.