Twitter is being used increasingly by people who find themselves in the middle of a disaster to report what’s happening, who needs help, and the extent of damage. But when a disaster strikes, the volume of tweets can be overwhelming for anyone trying to monitor them.
The Commonwealth Scientific and Industrial Research Organization(CSIRO), Australia’s science agency, has come up with a systematic way of sifting through the 140-character messages and feeding important details to crisis coordination centers, which in Australia organize assistance from government agencies.
“Twitter provides a new source of data from which crisis coordinators can obtain awareness about developing situations,” says Mark Cameron, the project’s leader.
Cameron, along with researchers Andrew Lampert, Bella Robinson, and Jie Yin, wrote “Using Social Media to Enhance Emergency Situation Awareness,” published in the November/December issue of IEEE Intelligent Systems magazine.
Situation awareness relies on the ability to identify a set of perceptions about an event, comprehend its meaning based on patterns and their location, and forecast and project an event’s outcome. Already, Twitter has played a critical part in helping emergency responders make effective decisions, according to the researchers.
Cameron and others at CSIRO saw the need for such a system after traditional ways of communicating failed during the tragic bushfires that devastated the state of Victoria in 2009. The fires killed 173 people, destroyed more than 2000 homes, and cost an estimated Australian $4.4 billion. Power failures, communications failures, and a lack of real-time information about local conditions were widespread. The event spurred the government to seek proposals for projects that could increase situation awareness.
CSIRO’s proposal was approved and funded in 2010 by the Australian cabinet’s National Security Science and Technology branch.
“My team had the skills in text mining, streaming data analysis, spatial databases, and rapid prototyping,” Cameron says. “Our ESA [emergency situation awareness] system offers convenient access to a great many independent observations about a real event.”
The researchers worked with the Australian government’s Crisis Coordination Center, a 24/7 facility in the capital city of Canberra. The CCC is part of the National Crisis Coordination Capability Program, which was established in 2008 after the completion of the Homeland and Broader Security Review. The review pinpointed a need to centralize information and to improve the way the prime minister and cabinet are briefed during crises. It coordinates the government’s response to concurrent domestic crises and provides support during international incidents to the Department of Foreign Affairs and Trade and the Australian Agency for International Development.
The “CCC coordinates information flows among government agencies to ensure appropriate resources and responses for crises are brought to bear where they’re needed,” Cameron says. “It also seeks situation awareness information about large-scale events so it can be well prepared. The Canberra center’s watch officers are the country’s eyes and ears, continuously seeking information about possible hazards, as well as assessing and evaluating information from many sources throughout the prevention, preparedness, response, and recovery phases of an emergency.”
BURSTS OF ACTIVITY
The ESA prototype adapts data-mining techniques to harvest high-volume Twitter streams and identify early indicators of an incident, explore its impact, and monitor its evolution. Using a data-capture module, the program continuously collects and analyzes tweets from locations throughout Australia. It relies on Twitter’s location-based search and automatic processing interface, off-the-shelf harvesting software tools, and a near-real-time search interface of its own.
But first a baseline had to be created. To do that, approximately 2.6 million tweets sent between June and September 2010 were captured and used to build a background model of words as a point of reference for an average, non-crisis day in the Australian Twitter-sphere. The system then analyzes the streams to detect bursts of telltale words or phrases coming in during a crisis. The bursts can last for more than a minute and are statistically unusual during a crisis when compared with the background model. Examples of the words it looks for are “earthquake,” “hurricane,” and “gunman.”
The group realized that reading every tweet could be overwhelming, so the messages are summarized and compressed and yet can still give watch officers a fairly accurate picture of what is going on.
“We’ve found that these bursts can be correlated with real-world incidents that people report,” Cameron says. “The ESA user interface displays these word bursts in a way that links to the statistical significance of the underlying deviation, so watch officers have some clue about how significant the words are, enabling them to more quickly and easily decide which incidents to investigate.”
Once an officer decides to dig deeper, the system’s clustering component provides an overview of topics drawn from all the messages contributing to the bursting words. That incremental clustering algorithm automatically groups similar tweets into topic clusters and allows only those that contain bursting words to form clusters. Each cluster corresponds to an event-specific topic. The system also includes a geotagging module that displays the content of a tweet along with its location on a map.
“This summary is valuable in quickly organizing large volumes of tweets—at times there can be hundreds contributing to a burst—enabling a very rapid review,” Cameron says. The near-real-time search and alert tracking enables watch officers to follow words and perhaps events that dip below the “radar” of the burst detection.
The researchers also developed a text classifier for the CCC to help identify messages that contained reports of damage and the impact a disaster has on roads, buildings, bridges, and other infrastructure.
Various prototypes of the system have been used during incidents such as Cyclone Ului, which damaged the Queensland coast in 2010, a gunman loose on the streets of Melbourne that same year, and the earthquakes that struck Christchurch, New Zealand, in 2010 and 2011.
Cameron reports the system has moved from prototype to the proof-of-concept stage. It is part of CSIRO’s Social Media Analysis Suite. The group is also working with Geoscience Australia, a government agency, to explore how the system could be configured to provide e-mail alerts for what it calls zero-warning events, like earthquakes.
“We have invited federal, state, and local government agencies to trial CSIRO’s advanced social media analysis tools and work together with other organizations to collaboratively get the most impact from Twitter,” Cameron says.