agile gloBal DeVelopment
Harnessing tHe power of real-time information
2 closing tHe information gap witH Digital Data
“ today’s new world of data holds the promise of giving decision-makers real-time
awareness of how crises are impacting vulnerable populations, and real-time feedback on whether our policies and programmes are working to protect them. together, we will create a more agile, 21st-century approach to global development. ”
-UN Secretary-General Ban Ki-moon
tHe potential of real-time Data
from Data to action
Decision makers working to protect vulnerable communities are confronting a new challenge: in today’s increasingly connected world, socioeconomic crises emerge and spread around the globe almost with the speed of natural disasters. At a time when our need for policy agility has never been greater, we have found that our 20th-century tools for tracking development cannot keep up. Global Pulse, an innovation initiative in the Office of the Secretary-General, represents a new opportunity to help protect the vulnerable through innovation. Global Pulse is partnering with interested governments, UN agencies, academia and the private sector to explore how emerging technologies and new sources of data can allow Policymakers to stay ahead of fast-moving crises. Global Pulse functions as an innovation laboratory for the UN and interested Member States to learn how to harness new data for development. To that end, Global Pulse is primarily focused on three interdependent areas of work: Data Research: Exploring innovative methods of combining new types of real-time data with traditional development indicators to detect early impacts of global shocks. Developing Collaborative Technologies: Assembling a technology platform to house cutting edge tools for sharing ideas and hypotheses and for analyzing data. Network of Pulse Labs: Launching a catalytic network of in-country innovation centers that will explore real-time impact monitoring and decision-making at the country-level and will contribute to developing the best practices for using real-time data in development work. You can read more about our work in the following pages and on our website: www.unglobalpulse.org
a feeDBack loop approacH
leVeraging un system expertise
finDing Digital signals
We hope you will be inspired to join our effort to harness the tremendous potential of real-time data for the future of development.
closing tHe information gap witH Digital Data
Policymakers in today’s hyper-connected world are grappling with a volatile environment in which crises in one part of the world emerge suddenly, reverberate around the globe, and amplify the effects of crises already underway.
analyzing this data for a real-time understanding of their customers. The time has come for us to learn how to use this new data to understand when households are struggling to make ends meet. When combined with the wealth of traditional mechanisms for data collection that are already in place, this new data holds enormous potential to improve our understanding of how populations are impacted by global shocks. Populations under stress change their behavior. As they cope with lost income or higher food prices, they change what they buy and sell. They change how they communicate. They seek different kinds of information, and they begin using services in different ways. They may look for additional employment, buy less expensive foods or ask relatives for financial help. In 2011, these changes
in behavior increasingly leave behind trails in data that provide a powerful new tool to help flag early signals of stress or incipient harm. To protect communities against the impacts of fast-moving crises and keep global development on track, we need real-time information on how communities are coping with the impacts of crises and real-time feedback on how well programs and policies are working. If we are to adapt to this new landscape, we need a 21st century approach to tracking development. Developing this new approach is the mission of Global Pulse. Global Pulse functions as an innovation laboratory, bringing together expertise from Governments, UN agencies, academia, and the private sector to
research, develop, test and share tools and approaches for harnessing real-time data for more effective and efficient policy action. Global Pulse is working to discover real-time proxy indicators of changes in human well being, developing free and open source technologies for analyzing data and sharing hypotheses, and working with interested Governments to establish a network of Pulse Labs in developing countries. Today’s ocean of digital data represents an unprecedented opportunity to improve the accuracy and agility of policy action to protect vulnerable populations and strengthen our resilience in a complex and volatile world. The technology tools to make sense of this data are available today. The time has come for us to learn, together, how to take the global pulse. n
e are witnessing an unprecedented phenomenon: socioeconomic crises that move almost with the speed of natural disasters. At a time when our need for policy agility has never been greater, we have found that our 20thcentury tools for tracking development cannot keep up. Too often, by the time we have hard evidence of what is happening at the household level, the harm has already been done. The recent waves of global shocks— food, fuel, and financial—have revealed a wide gap between the onset of a global crisis and the availability of actionable information for decision makers. Traditional statistics have been effective in tracking medium- to longerterm development trends, but—given the latency of the data generated—are of little use to decision makers trying to help vulnerable populations cope with fast moving crises. Much of the data used to track progress toward the Millennium Development Goals (MDGs), for example, dates back to 2008 or
earlier—before the onset of the current economic downturn. Knowing in close to real time the “how, where and when” of a crisis is crucial for the design of effective policy responses. While this may be obvious in the aftermath of a natural
knowing in close to realtime the how, where and when of a crisis is crucial for the design of effective policy responses.
disaster, it is also increasingly evident in the context of global events such as the recent financial crisis. Finding out today how a community began coping with a crisis two years ago is generally too late to prevent longer-term damage. It is clear that the price a country pays for not knowing how specific segments of its population are coping with shocks is high.
Waves of compound global shocks cause households to lose ground in their struggle to escape poverty and drive many of those who had escaped back into destitution. Yet today’s tools do not allow us to track this descent into poverty as it actually happens, or to catch it early enough to make agile adjustments to our policies and programmes where required. This lack of timely and actionable information creates a risk of reversing years of development gains in a matter of months. Yet we also face an extraordinary opportunity today. The mobile digital revolution of the past decade now has us swimming in an ocean of real-time data, most of which did not exist even a few years ago. From the content of social media sites to the data we leave behind in communicating, transacting business, and searching for information, people everywhere are generating vast quantities of data, for free, merely by going about their daily lives. Private sector companies are increasingly
the information gap
Household-level data is so hard to collect that the information being used to track development progress is frequently out of date. We know what happened in the past but not what’s happening in the present. Real-time data could strengthen progress indicators by closing the information gap.
INFORMATION GAP Age of Most Recent Data
HUNGER AND POVERTY
HIV AND MALARIA
Source: Millennium Development Goals Report 2011
Harnessing innovation to protect the vulnerable
ata from blogs and other online discussions together with “data exhaust,” (the digital trails we leave behind as a by-product of daily activities) are increasingly providing the kind of information that previously could only be gleaned from faceto-face surveys. An overview of the new data landscape reveals that real-time analysis of such data will be integral to policy planning in the future. Recognizing this potential, the Executive Office of the Secretary-General launched the Global Pulse initiative in 2009 to explore how this new data could help the United Nations’ development community obtain a more real-time understanding of populations’ emerging vulnerabilities. “The United Nations is uniquely placed to leverage these new data sources for the global common good,” says Zazie Schafer, Deputy Director of the initiative. The term “real-time data” indicates little or no delay between data collection and dissemination. In the context of Global Pulse, real-time data refers to data that is available soon enough to initiate policy actions to prevent harm. Some of the new data sources that Global Pulse will be looking at include: Online news and media outlets where trending topics can reveal what areas are of concern to readers. Data generated from information and communication technology (ICT) devices like mobile phones and “smart phone” platforms, that can show how populations are changing their spend-
ing, whether they are using services like mobile banking or whether users are moving en masse from areas affected by crises like drought, disease or shortages of food or services. Aggregated e-commerce data that can show changes in buying and selling behavior e.g. people cutting back on non-essential purchases or buying more medicines than in previous months. Feeds from social media sites where people voluntarily share information about their health and finances. In aggregate this can tell a lot about the well-being of communities. What makes the billions of pieces of data culled from these sources useful—in essence what transforms this new data into insightful information—is expert human analysis. Defining frameworks for such analysis, and developing tools when necessary, is at the core of Global Pulse’s work. The future use of real-time data in development work rests on learning how to effectively and efficiently mine raw data for actionable information, i.e. information that can be useful in programme planning. To be successful it will be necessary to answer some big questions, paramount among them: What are the meaningful signals of distress for a given community? Can those signals be inferred from the data? Once detected, what do they reveal about well-being? And how can they be useful to policymakers? Global Pulse has launched several research projects to further define
and tackle questions like these. Additionally, the Global Pulse technology team is designing a tool called HunchWorks that will allow development experts to share hypotheses about worrying trends in data, link to additional evidence and debate and verify those “hunches” within a social network of trusted colleagues. Those colleagues may bring in additional evidence to support or to refute the hypothesis. (For more, see HunchWorks on page 14). “The projects we are undertaking here at Global Pulse are designed to bring global experts in the emerging field of data science together with those who are best positioned to understand the needs of decision makers in developing countries,” says Eva Kaplan, Programme Specialist. “Through these collaborative projects, we are ensuring that the specific needs of government decision makers are at the forefront of how we understand the opportunities in new data.” To get a better picture of the potential of real-time data, Global Pulse teamed up with PSFK, a research consultancy specializing in emerging innovation trends, to explore some of the best examples of real-time data use. The resulting publication, The Future of Real-Time, highlights key areas of opportunity including instant mapping, mobile communities, contextual cartography, intelligent infrastructure as well as the institutional use of real-time technologies. Look for more on Global Pulse’s research projects in this publication. n
tHe potential of real-time Data
The use of real-time data is providing Governments, corporations and organizations with rapid ways of detecting and responding to the needs of the populations they serve without requiring significant new infrastructure.
finD out more online: For a blog post on The New Data Landscape that explores new sources of real-time data, visit www.unglobalpulse.org/blog/new-datalandscape. Find the PSFK Future of Real-Time report at www.unglobalpulse.org/projects/future-real-time-report
Harnessing innovation to protect the vulnerable
from Data to action
Big Data can yield information that speeds up decision-making while still protecting individual privacy.
SOCIAL MEDIA SOCIAL MEDIA
PRIVACY PRIVACY ANALYSIS ANALYSIS ACTIONABLE DATA ACTIONABLE DATA
NEWS FEEDS NEWS FEEDS
MOBILE PHONE DATA MOBILE PHONE DATA
A A G G G G R R E E G G A A T T E E
A A N N O O N N Y Y M M I I Z Z E E D D E E T T E E C C T T V V E E R R I I F F Y Y
GEOSPATIAL DATA GEOSPATIAL DATA
UN DATA UN DATA
GOVERNMENT DATA GOVERNMENT DATA
CITIZEN REPORTS CITIZEN REPORTS
1. Data Real-time data originates both from new technologies as well as from new approaches to mining traditional data.
2. priVacy Once aggregated, data is fully anonymized to protect privacy and to facilitate analysis that is focused on the community, not on the individual.
3. analysis Detection of anomalies in the data trigger follow-up investigation and verification using both new technologies as well as traditional analytics.
4. action Real-time data enables decision makers to protect vulnerable populations with more agile and targeted responses.
lobal Pulse is built on the understanding that today digital data is being generated at an unprecedented rate and can hold real insights about changes in population well-being. Knowing how vulnerable populations are changing their behavior, especially during the unfolding of a global crisis, can help decision makers accelerate and target programme planning and policy making in ways that can have great impact, especially in the fight to protect development gains. Global Pulse’s work is focused on making data actionable for decision makers to use in shaping programmes and policies in real-time. Moving from data to action requires several steps:
step 1. Data collection
policymakers and researchers. Data is also generated from governments and agencies tracking the progress and effectiveness of their programmes in the form of surveys, hotline records, and inquiries. When people’s needs change, they change how they participate in such programmes, and those changes are captured in the data. Global Pulse calls this data “programme exhaust.”
step 2. ensuring priVacy anD anonymity
The Internet supports the flow of massive amounts of user-generated information such as news, social media, blogs and e-commerce sources among many others. Populations also generate and share data as by-products of their daily lives. For example, as communities in the developing world increasingly adopt and use new technologies—particularly mobile phones and services provided over mobile—they generate ambient data or “data exhaust.” Much of this data is not publicly available but is owned— and held closely—by the private sector. Global Pulse is developing innovative public-private partnerships to allow analysis of this data to be shared with
community level indicators; the identity of individuals plays no role whatsoever. Global Pulse does not handle data with personally identifiable information, and its country-level research is conducted within the context of national data privacy laws. Because Global Pulse and its research partners are interested in the collective picture that emerges from data sources, private and public sector partners must anonymize parts of the data that are not statistically important to the research. In fact, the information Global Pulse is exploring is most valuable in aggregate, when it can show how a population in a particular region is collectively changing its behavior in response to global shock or a policy change.
step 3. analysis
Of course, it is not enough just to have this data. Most important is developing an understanding of how it can be useful. Part of the equation is exploring how new data can be integrated with the traditional sources of data. Another important factor is developing local expertise, because recognizing the signs of stress in a particular population often depends on familiarity with the local context. In addition, the availability of data and the types of signals the data yields will differ from country to country.
step 4. action!
sion makers to target and refine their interventions more efficiently than is possible today. In a fast-moving, hyperconnected, and volatile world, public policy must be agile and continuously adaptive if it is to protect vulnerable populations from shocks, keep global development on track, and meet the changing needs of citizens. Global Pulse is developing methodologies with which decision makers will be able to integrate these data streams into their existing monitoring efforts to detect anomalies, form hypotheses, gather evidence, verify the underlying causes and take action to improve the well-being of the populations they serve, especially in addressing the impact of global shocks and crises. n
Mining data for the pulse of communities does not equate to monitoring individuals. Global Pulse takes privacy very seriously. Global Pulse’s data research is focused solely on analysis of
At Global Pulse, we recognize that analysis of patterns within new streams of real-time data can enable deci-
Harnessing innovation to protect the vulnerable
high frequency, household-level data available through surveys and quick impact assessments. A model for this process has been used for years in the field of public health to monitor for outbreaks of infectious diseases. When public health officials see people not coming in to work or school, buying certain drugs, and showing up at clinics with fever and headache, they may suspect an outbreak of flu. Detection of these anomalies triggers a process of rapid investigation (collection of specimens) and verification (laboratory testing) that leads to early and effective response. The framework, which draws on similar lines, will be developed and tested through Pulse Labs in consultation with government, UN agencies, academia and the private sector and will involve: Proposing and investigating hypotheses about the causal factors driving these collective behavior changes based on analysis of additional contextual information; Sharing of hypotheses within trusted networks where experts share information to support or refute hypotheses; Testing promising hypotheses by soliciting feedback—perhaps via SMS text messages—from trusted contacts within the local community, such as local government, teachers, community health workers, or radio station hosts; Verifying hypotheses and gathering information required to design appropriate responses through targeted, agile collection of new household-level data; Taking action by working with UN agencies to support governments in implementing policy responses and using this new monitoring approach to track results. n
leVeraging un system expertise
lobal Pulse is dedicated to developing new information streams to protect vulnerable populations in the context of increased global shocks. This requires strengthening the ability to understand the signals in new data; it also requires understanding the dynamic complex processes that determine how people are impacted by crisis. Thus, one of Global Pulse’s first undertakings, the Rapid Impact and Vulnerability Analysis Fund (RIVAF), was dedicated to leveraging the expertise already in the UN. Global Pulse invited UN agencies to submit project proposals designed to examine the multiple ways that those impacts manifested. Participants were encouraged to submit plans that could be implemented in partnership with at least one other agency. In the end 11 UN agencies undertook eight rapid impact assessments that covered 38 countries and four cities. The collaborations added value to agency work in multiple ways: Several projects developed online tools for data transfer, expert collaboration and analysis of large data sets. RIVAF research findings provided inputs for agency programming, policy formulation and advocacy. New methodologies and statistical models were tested for their general applicability. The project participants met at a conference in New York City in July 2011 to discuss the findings of their research and to identify common challenges and lessons learned. They also discussed how common research challenges could be addressed using new data, and—building on the findings of the first round of RIVAF projects—explored ideas for a future round of research. In particular, the collaborations confirmed the existence of significant data gaps, affirming the need for access to compelling real-time data, which is one of the core goals of the Global Pulse initiative. Participants expressed strong interest in further collaborating with Global Pulse to test how new digital data sources could help fill existing knowledge gaps. Everyone agreed that a future round of RIVAF funding would be ideal as it would facilitate further innovative collaborations between UN agencies, as well as with leading academic and public and private sector experts in the field of real-time data. n
The UK’s Department of International Development (DfID) and the Government of Sweden provided funding for the RIVAF projects. The following eight agency research projects were chosen for exploration: unDp/unicef: Monitoring Households’ Coping Strategies During Complex Crises unesco: Reviewing the Impact of the Crisis on Education in Selected Developing Countries unfpa: The Impact of the Global Financial Crisis on Reproductive and Maternal Health in Colombia, Ethiopia and Jordan unicef/wfp: Food and Nutrition Security Monitoring and Analysis Systems: A Review of Five Countries (Indonesia, Madagascar, Malawi, Nepal and Zambia) unoDc: Monitoring the Impact of Economic Crisis on Crime unoDc: Smugglers and Vulnerable Migrants in Central America and Mexico unoosa/uneca/unep: A Visual Analytics Approach to Understanding Poverty Assessment through Disaster Impacts in Latin America and Africa unwto/ilo: Economic Crisis, Tourism Decline and its Impact on the Poor: An analysis of the effects of the global economic crisis on the employment of poor and vulnerable groups in the tourism sector
At Global Pulse we say where there is a digital smoke signal—an unexplained change in how people communicate or use digital services—there may also be fire: a need for action.
nderstanding quickly where the fire is requires additional reporting and analysis. So does understanding what the digital smoke signal means, and what action is required. Smoke signals aren’t hard evidence. They aren’t actionable information by themselves. But where they appear, something may be happening that policymakers need to know more about – and quickly. What is needed is an approach that rapidly detects the earliest smoke signals, supports verification with contextual information and additional evidence, and informs more timely policy responses. Global Pulse is developing an innova-
tive framework for combining traditional indicator-based approaches to tracking development with new kinds of real-time data to pick up anomalous changes in behavior. This involves analysis of anonymous and aggregated digital data to detect abnormal patterns in populations’ behavior, and combining it with data that provides context for multi-dimensional analysis, cross-validation and verification. Such contextual data includes: (a) macro level information (e.g., macro-economic indicators); (b) trend data for key socioeconomic indicators (e.g., MDG data); (c) real-time information on specific contextual variables (e.g., weather, price data); and (d)
finD out more online: For more information about RIVAF, including full descriptions of the parameters and objectives of the projects and their findings, go to our projects page at www.unglobalpulse.org/projects. Download the RIVAF Partners Final Report at www.unglobalpulse.org/research/reports-andresources
Harnessing innovation to protect the vulnerable
finDing Digital signals
How do you find indicators in big data? How do you know which of those signals are relevant enough to warrant further investigation? These questions are at the core of Global Pulse’s research.
“multidisciplinary collaborative projects offer great opportunities for discovering patterns that will lead to new insights useful for impact monitoring and development”
- Miguel Luengo-Oroz, Data Scientist for Global Pulse
can social media mining add Depth to unemployment statistics?
partner: sas (www.sas.com)
news awareness & emergent information monitoring
partners: complex systems institute of paris (isc-pif), formism, ifris, cams, crea, inra-sens.
n the past year Global Pulse has teamed up with some of the world’s leading research firms and institutions to engage in an ongoing series of research projects designed to test the feasibility of real-time data collection and analysis to find new indicators of population well-being. Results from the initial series of projects have underlined several promising approaches to improving monitoring of unemployment and food security. Findings from these projects will help to establish new methodological approaches for analyzing real-time data, identify tools which can provide a faster and clearer understanding of population behavior during periods of stress, and contribute to the development of new proxy indicators for real-time tracking of development. Some of the projects explored ways to produce high frequency data. For example, mobile phones provide a powerful platform to reach people and to get real-time feedback directly from communities. Other projects explored ways to mine, aggregate and analyze online data—such as food prices or news trends—in real time. Global Pulse is also exploring the ability to tap an entirely new source of information: the global conversation, or social media. This type of data can help us “listen” to a populations’ own perceptions of their changing circumstances. The methodologies needed to explore this massive unstructured data from social media sources require the use of the latest analytical techniques, including data mining, machine learning algorithms and automated text processing. Quantifying the perceptions of populations presents a powerful opportunity to understand a population’s behavior in a way that complements existing
methodologies for tracking and monitoring development. “Multidisciplinary collaborative projects between Global Pulse and private companies and academic researchers are a fantastic approach for adapting new tools and methods for development. This innovative ecosystem offers great opportunities for discovering patterns that will lead to new insights useful for impact monitoring and development,” says Miguel Luengo-Oroz, Data Scientist for Global Pulse. An overview of some of these exciting collaborations follows:
How does combining social media data with traditional data sources amplify understanding of the experiences of the unemployed? In collaboration with SAS, this project analyzes conversations from the open social web for early signs of stress among the unemployed in order to better understand their coping strategies for dealing with their unemployment. Another goal is finding out how the unemployed perceive their status, e.g. whether they are collectively optimistic or worried about the future. For this purpose we explore correlations between quantitative indictors produced with the latest text analysis methodologies and official unemployment statistics.
This multi-partner project analyzes thousands of news items related to food security issues in French language media (national and local). It explores how news around particular issues gains salience, evolves over time and answers questions such as: Is an issue (e.g. biofuels and food security) attracting more or less attention with time? Do discussions around any issue (observed at a given time) stem from how past related issues are framed, or do they emerge distinctly? An online interface allows users to visualize and map key themes and news entries and shows what issues related to food security are of greatest concern in the media at any given time.
global snapshot of well-Being: exploring the potential of global mobile surveys
partner: Jana (ww.jana.com)
comparing global prices to local products: real-time e-pricing of bread
partner: pricestats (www.pricestats.com)
twitter & perceptions of crisis-related stress
partner: crimson Hexagon (www.crimsonhexagon.com)
Global Pulse is working with Jana to test the potential of quickly deployed mobile surveys, over SMS, in multiple countries. The objective is to complement and strengthen our agility in the way we do household surveys, reaching thousands of responders in a few days to provide global snapshots. This methodology tests the feasibility of conducting longitudinal surveys over time.
finD out more online: To follow the progression of these and other projects, visit our projects page at www.unglobalpulse.org/projects
Inflation of the cost of basic food items disproportionately affects those who are struggling financially. This project, a collaboration with PriceStats researchers, entails the construction of a daily “bread price index” for six Latin American countries to show how a tool might be created to identify real-time changes in the prices of essential food items. The tool measures daily retail prices through online supermarkets and combines this information with the availability of products to create the daily price index. This has the potential to help us detect excessive price volatility on people’s wallets in real-time.
How can micro-messaging services—particularly, Twitter data—be used to provide analytic insights about how people perceive issues related to food, fuel, housing, and the economy? In collaboration with Crimson Hexagon, we measured the degree to which cost pressures around these core items are discussed in online conversations. The research analyzes trends in these topics in conjunction with keywords such as “afford,” showing how the volume and topics of the conversations change over time reflecting populations concerns.
10 gloBal pulse
Harnessing innovation to protect the vulnerable
A research platform that combines observation, social networks and big data to identify crisis impacts in real-time.
POSSIBLE OUTCOMES POSSIBLE OUTCOMES
IMMEDIATE ACTION IMMEDIATE ACTION
cally analyze evidence attached to hunches. It will also stimulate knowledge sharing and debate by recognizing when discrete groups of experts are working on the same issues and informing them of each other’s existence. “HunchWorks is like simultaneously being connected to the smartest, most informed people you know and the smartest, most informed people you don’t know,” says Sara Farmer, Chief Platform Architect for Global Pulse’s technology team. In several ways, HunchWorks will virtually recreate the working dynamics of a Pulse Lab, the in-country innovation centers Global Pulse plans to facilitate in the next five years, where diverse experts work to combine early evidence from a variety of sources to support more agile and targeted policy action. (For more, see Pulse Labs on page 16 of this publication)
open source DeVelopment
a researcH scenario
A development professional working in a Government ministry, UN agency or research organization notices signs of deteriorating social conditions in a vulnerable community. As she gathers content from household surveys, online news, social media, and satellite imagery, she grows concerned that she may be witnessing the start of a harmful longer-term trend. So she starts to document her observations and gather evidence. She also starts to reach out to other credible observers in the region to find out if they are seeing anything similar. Responses are mixed and her contacts are limited to friends within the local agency and a few local community figures that she has met. She has seen this type of pattern in the data before. She has a hunch that the trend is potentially very damaging, but due to the informal, unverified nature of her observations and the limited feedback of colleagues, she knows that it will be difficult to make the case this early to her superiors for immediate action on behalf of her agency. She knows there must be others observing similar signs who could help gather more evidence if they knew what to look for. She also recognizes that there are experts with deeper subject matter expertise who could review her findings. But how can she identify and engage these people quickly enough to stay abreast of unfolding events on the ground?
FURTHER RESEARCH FURTHER RESEARCH
DISMISS HUNCH DISMISS HUNCH
After a potentially destructive trend is identified through observation, research or first-hand accounts, a hunch is posted with a description, location data and any relevant evidence.
The hunch is shared with trusted networks on the HunchWorks platform. Users from within these networks provide feedback on the hunch and evidence posted. They are also able to contribute additional evidence.
Users are able to evolve their collective understanding through ongoing discussion, a growing body of evidence and analytical tools built into HunchWorks.
If there is sufficient consensus that a hunch has been confirmed, tools built into HunchWorks will allow users to take appropriate action e.g. bundle evidence into reports and activate networks for advocacy.
a new tool
The Global Pulse technology team’s answer to this problem is HunchWorks, a social network tool for gathering evidence, analyzing data and peer review all at once. HunchWorks is based on the recognition that all too often, by the time we have hard evidence of what is happening, it’s too late to act. Yet after a crisis, the experts inevitably point to bits of fragmentary evidence that was
available early on but was so inconclusive at the time that no one was willing to sound the alarm publicly. Using HunchWorks, the researcher could have quietly composed a hunch outlining her concerns. After attaching relevant evidence, she would have shared it informally through the system, where it would be visible only to a trusted network of expert colleagues in relevant communities of practice. Her colleagues might in turn have brought
in additional evidence supporting or refuting her conclusion, provided feedback about her observations, and suggested other experts to invite into the conversation. In effect, HunchWorks is the world’s first social network for hypothesis formation, evidence collection, and collective decision-making. Researchers who use HunchWorks are part of a self-selecting virtual peer community in which their connection is a deep common interest in and expertise
on a topic, e.g. food security or health processes, and their respect for each other’s work. A community of experts may have members from different government ministries, UN agencies, disciplines or expertise. They may contribute empirical evidence from scientific research or anecdotal or observational evidence from field work. Or they may help facilitate collaboration, whether it’s building systems or moderating debate. HunchWorks includes computational reasoning features that will automati-
Collaboration is also integral to the creation of HunchWorks itself as the tool is being developed as open source software with support from a cadre of volunteer programmers, designers and testers. Dedicated volunteers meet weekly at Global Pulse’s New York offices for “code jams” where they develop and test components and work out kinks in the system. The code jams have also enabled Global Pulse to build a database of expertise, as many of the volunteers have day jobs at technology companies. “The open source method allows us to develop technology that can be useful to everyone, gives us access to expertise that we wouldn’t normally have and helps us develop the local tech community,” says Farmer. n
finD out more online: See www.unglobalpulse.org/technology/hunchworks
12 gloBal pulse
Harnessing innovation to protect the vulnerable
s the recent global financial crisis unfolded, it became clear to in-country decision makers that they did not have a real-time understanding of how the crisis was uniquely affecting their own populations. What was needed was the ability to quickly—and as accurately as possible—get continuously updated snapshots of how populations were being impacted so that policy responses could be targeted to their most current situations. Traditional data collection methods like door-to-door household surveys that can take months or even years to complete are woefully inadequate for this task. Finding out today how a community began coping with a crisis two years ago (or even two months ago) could mean it is too late to institute the changes aimed at preventing longer-term damage. Global Pulse was created from the understanding that what is lacking in the world of development planning lies in the world of real-time analytics that is already being employed by the private sector. Companies use real-time data to help them understand the changing needs of their customers and respond by designing more agile business platforms. However, applying real-time analytics and digital data to development challenges is relatively uncharted territory. It is not yet clear which of the many and varied new data sources will prove most valuable for tracking changes in human well-being, nor is it clear which analytical methodologies and technology tools will be required to transform this data into actionable information. Accordingly, Global Pulse has elected to adopt a joint exploration approach in which interested Governments partner at the country level with UN agencies to establish a network of innovation centres, or Pulse Labs, that will develop the approaches, tools, public-private partnerships and hu-
man capacity required to harness real-time data for policymaking. Global Pulse is focused on building a small, highly integrated network of ten labs in the next five years. So far the Pulse Lab track has established Pulse Lab New York and is in the process of developing Pulse Labs in Uganda and Indonesia.
incubators and innovation spaces—with the twist that they connect to a global network of United Nations’ needs and expertise,” says Christopher Fabian, Global Pulse Advisor on Innovation (ad interim). It is clear that the most important success criterion for Global Pulse is
“pulse labs are the heart of the global pulse initiative, and we need government support to bring the vision to full life,”
- Makena Walker, Global Pulse Partnerships Adviser
By mining digital data for correlations with traditional statistical indicators used to track development, Pulse Lab staff will work to identify patterns that were being left behind in digital data—in real time—as communities struggle with the impacts of crises or benefited from government programmes. Once identified, these patterns will be investigated thoroughly as potential real-time proxy indicators of well-being that could be monitored on an ongoing basis to support decision makers. Pulse Lab New York, which functions as a research and development space and a facilitator for partnerships with multinational private sector data providers, spearheads Global Pulse’s innovation in real-time monitoring and leads the design of the technology platform and tool kit design. It is currently serving as the knowledge hub for Global Pulse’s work, ensuring that as other labs develop they will be able to access and build on each other’s innovations. “Labs provide a physical space for active learning, sharing and, importantly, creating solutions in collaboration with host governments. These instruments are modeled on the success of private sector and academic Member State participation in the Pulse Labs track. “Pulse Labs are the heart of the Global Pulse initiative, and we need government support to bring the vision to full life,” says Makena Walker, Global Pulse Partnerships Adviser.
pulse laBs in action
Pulse Labs are networked, country-level innovation spaces where government experts and UN agencies can experiment with trusted partners to develop methods and tools for real-time impact monitoring, collaborative analysis and decision-making.
14 gloBal pulse
With guidance from Government stakeholders, Pulse Labs will focus their research efforts on addressing critical information gaps in understanding the needs and vulnerabilities of at-risk populations. After an initial process of identifying existing data, public and private sector sources of relevant digital data, and key partner organizations, Pulse Labs will pursue a focused research agenda based on retrospective analysis. This process consists of analyzing anonymized and aggregated digital data generated by communities over the past few years (e.g., mobile phone calling data, money transfer records, online news, social media content) and comparing it with robust statistical evidence of what these communities were experiencing at the household level during the same period (e.g., household survey data, unemployment figures, food price fluctuations, disease outbreaks).
Harnessing innovation to protect the vulnerable
How pulse laBs work
Pulse Labs operate as hubs where government and UN country teams can work together with partners such as data analysts, academia, open-source technologists, the private sector and civil society to explore ideas and to design and develop tools that can be used to recognize signals in data exhaust.
In order to support ongoing research and contribute to future capacity for real-time monitoring, Pulse Labs will cultivate strong relationships with a variety of local stakeholders through events, campaigns, and other forms of outreach. Partnerships with promising private sector data providers such as mobile phone carriers and microfinance institutions will be established in ways that allow these organizations to share analysis of their data without compromising either individual privacy or corporate competitiveness. Community leaders will provide insight into what context-specific changes occur in local communities impacted by crises and suggest ways
experts from the private sector. Pulse Lab teams will coordinate virtually with one another as a network for sharing the technological and methodological innovations they each produce. While still in its early stages, the experience of Pulse Lab Kampala is instructive. In December 2010, two innovations officers were posted in Kampala to do the initial scoping for the lab. UNICEF has an active innovations unit in Uganda, and offered to host the pre-launch phase of Pulse Lab Kampala. The Global Pulse team has worked to develop partnerships with the government and private sector, understand information gaps and priorities, identify potentially valuable data
“We’ve learned so much about what it takes to apply the global vision locally. It will certainly make establishing other pulse labs a lot easier.” More recently, the scoping phase for Pulse Lab Jakarta started in October of 2011. The UNDP country office in Jakarta served as the facilitator. In partnership with the Government of Indonesia, the Lab is scheduled to start operations in June 2012. The Lab’s objectives are aligned to the United Nations Partnership Development Framework covering 2011-2015. Two proof-of-concept research projects—one on mobile data, the other on Twitter—should be completed by July 2012.
with guidance from government stakeholders, pulse labs will focus their research efforts on addressing critical information gaps in understanding the needs and vulnerabilities of at-risk populations.
to engage citizens at the grassroots level in the process of reporting these changes when they first appear. UN agencies and national institutions will identify existing programmes that could benefit from incorporating analysis of real-time digital data. Open source software developers will assemble, translate, and enhance the technology toolkit required for mining digital data and providing actionable data visualizations for government decision makers. Secondments from Government ministries and UN agencies will comprise a significant fraction of the team, as will researchers from academia and
PRIVATE SECTOR AND CORPORATIONS
FOUNDATIONS AND NGOs
sources, and develop a strategic plan for the launch of the Lab. The Lab will collaborate with key government departments such as the National Planning Authority (NPA) of the Ministry of Finance, Planning and Economic Development, the Prime Minister’s Office, the Uganda Bureau of Statistics and other sectoral institutions. The private sector, including two mobile phone carriers, has expressed interest in partnering with the Lab. Other key stakeholders include Makerere University and several social enterprises. “The Ugandan experience has been an eye-opener,” says Makena Walker.
OPEN SOURCE TECHNOLOGISTS
Throughout this process, Global Pulse will collectively put together a handbook or “cookbook” for establishing a Pulse Lab, which will include mappings of possible partners, staffing needs and operations objectives. This will be a living document that can be shared widely to encourage and strengthen similar initiatives in countries and agencies that are not directly involved in Pulse Labs. The cookbook will capture best practices and lessons learned, therefore the knowledge base—developed from ongoing inputs from the Pulse Lab network—will be available to other interested countries to jumpstart the development of their own crisis impact monitoring capacities. n
finD out more online: To learn more about Pulse Labs, go to our website www.unglobalpulse.org/pulselabs. Also check out our multimedia page to watch videos from Uganda. For information about establishing a Pulse Lab, contact email@example.com
16 gloBal pulse
Harnessing innovation to protect the vulnerable