As augmented and mixed reality applications emerge and we move toward a ubiquitous computing environment, the technologies will begin to take a toll on human consciousness, according to John Rousseau, executive director at Artefact, a design and innovation firm in Seattle. He says our perception of reality could be influenced by a constant “feed” of information that—like today—is created and controlled by a variety of self-interested parties.
At the annual Augmented World Expo in June, Rousseau shared a set of principles for how we should think about, design, and evaluate future human experiences in mixed reality. The Institute asked him to explain the first of his so-called laws of mixed reality: Mixed reality must enhance our capacity for mindful attention.
In your talk at AWE, you suggested we challenge the “feed” as the default way of experiencing digital information—especially when it comes to mixed reality. What do you mean by that?
Today, we consume multiple streams of content across various devices and platforms that keep us engaged through notifications and the inherently bottomless nature of the “feed” paradigm. The feed is problematic because it encourages mindless consumption. It preys upon—and exacerbates—our fear of missing out. It also limits our exposure to diverse points of view by aggregating everything into one stream designed to keep us engaged.
A growing body of evidence suggests that we simply lack the willpower to resist the feed—which will only be exacerbated by mixed reality. Sure, you could turn off notifications, exercise self-control, or even stop using smartphones altogether. While these options seem sensible on the surface, they are not realistic and underestimate the central role that technology plays in our lives.
How can AR platforms be designed so that they help people live in the moment?
These technologies are a new medium, and they demand new approaches to the way content is displayed and experienced. The screen of the future is a projection of the world itself—not a self-contained, rectangular array of pixels like the displays on our smartphones. If we clutter the mixed-reality world with the digital equivalent of empty calories, we will become overloaded and increasingly disconnected.
The amount and type of content in a person’s line of vision while wearing an AR device should change depending on what she’s doing, the context in which she’s doing it, and in response to her choices. For example, most digital experiences are one-size-fits-most, and either on or off. Designing for a more meaningful continuum, and allowing the user to define key parameters, like notifications, would help us get past the binary nature of current systems.
What might future AR platforms look like?
The app paradigm will likely go away. Mobile apps currently offer too many disparate and disconnected experiences that simply won’t work for mixed reality. In a scenario that pairs mixed reality with ubiquitous computing, content will be more contextual and will simply be delivered via a single platform.
A good example of this is WeChat, China’s popular mobile platform. It is technically an app, but it lets users do a variety of things, such as text messaging, photo sharing, shopping, and even accessing public services. I think most of our mixed reality experiences will be similar to WeChat, but within a framework that has yet to emerge.
Searching for information will take on greater importance, and we will interact directly with intelligent systems to express intent and get the answers we want. These will be fundamentally new operating systems that require new rules and behaviors. Designers of these new experiences will need to be explicit about their assumptions regarding users and the context in which they will interact with the mixed reality experience.
What are the most important things developers and designers need to consider when creating experiences for AR?
Developers have an opportunity to create a more enjoyable experience that might even make us better people. To do this, they should take a human-centered approach to designing mixed-reality systems, and challenge the feed as the default way of consuming digital information. This new user interface for mixed reality should help strike a balance between staying connected and living in the present moment.
Future algorithms should tailor experiences to the user’s needs, ask for input, and respond to feedback. There is a clear role for AI—both as a means to facilitate dialog and learning and of paying closer attention to context. AI must look out for our best interests and save us from our worst selves. With more intelligent and empathetic systems, the nature of our digital lives—and maybe even our ability to focus on what’s important—could change for the better.
This article is part of our December 2016 special issue on digital senses.