Photo credit: Essam Sharaf

On Tuesday, January 25, 2011, Egyptians from all walks of life took the streets and public squares of cities and towns in the largest general strike in that nation’s recent history. The Egyptian revolution was born. In the heady weeks that followed, US pundits and politicians alike lamented the US government’s slow-footed response to the unfolding ‘crisis’ in North Africa.

For all the billions the United States spends on intelligence, the revolution was a surprise. And now that is having consequences.

A Reuters news analysis from February 2011 summarized what was becoming a common theme then and is now accepted wisdom: the Obama administration was “caught off guard” by the North African revolutions.

Both President Barack Obama and Secretary of State Hillary Clinton have been criticized for being slow to grasp the scale of the upheaval in Egypt where tens of thousands of people have protested for days to demand the ouster of President Hosni Mubarak, a long-time U.S. ally.

By 2011, the United States had provided the Egyptian military with billions of dollars in annual security and humanitarian aid for many years, and the Egyptian intelligence services were said to have shared information freely with their American counterparts. So how could they have missed something this big? In February 2011 analysts pointed the finger at the Internet. Again, Reuters:

American and European officials also acknowledge that the spread of social media — Facebook, Twitter and texting — and the leaking of sensitive U.S. documents by the WikiLeaks website were other factors that made it difficult for analysts to predict what might happen and when.

Micro-blogging sites like Facebook and Twitter, the argument went, enabled strike organizers and ordinary people to share information at such a scale and in such a manner that they eluded government monitoring. There was simply too much data, and it was impossible to sort it properly in order to be able to predict what would happen next.

It didn’t take long for the US Director of National Intelligence’s far out research arm, the Intelligence Advanced Research Projects Agency (IARPA), to start looking for a solution to this “problem.”

In August 2011, IARPA released a solicitation for contractors to apply for funds to help the agency develop an “Open Source Indicators” (OSI) program that would allow the US government to “beat the news” by predicting the future. The OSI project aimed to take public research tools that for decades had been used to surveil the spread of disease, and apply them to the study of human politics. IARPA wanted a computer program that would alert the US military and intelligence community to the next revolution before it happened.

From IARPA’s initial solicitation:

Many significant societal events are preceded and/or followed by population-level changes in communication, consumption, and movement. Some of these changes may be indirectly observable from publicly available data, such as web search queries, blogs, micro-blogs, internet traffic, financial markets, traffic webcams, Wikipedia edits, and many others. Published research has found that some of these data sources are individually useful in the early detection of events such as disease outbreaks. But few methods have been developed for anticipating or detecting unexpected events by fusing publicly available data of multiple types from multiple sources.

IARPA’s Open Source Indicators (OSI) Program aims to fill this gap by developing methods for continuous, automated analysis of publicly available data in order to anticipate and/or detect significant societal events, such as political crises, humanitarian crises, mass violence, riots, mass migrations, disease outbreaks, economic instability, resource shortages, and responses to natural disasters. Performers will be evaluated on the basis of warnings that they deliver about real-world events.

In short, US intelligence agencies are trying to develop computer algorithms to detect social and political uprisings.

Should we be concerned? After all, the government claims that all data inputs into the system will come from publicly available sources, and assures us that the program will not track specific individuals or “fund research on US events”

But a source close to the project tells me that once the program is operational, the US military intends to feed the system much more than just “open source” information, as the project’s title promises. Furthermore, my source believes that while data on 21 Latin American countries will be used to develop and test the program, the US military intends to use the final product to monitor the United States. 

Extraordinary claims require extraordinary evidence, and one confidential source may not be enough for some people. But really, is it such an extraordinary claim?

After all,

• Multiple NSA whistleblowers have independently confirmed that the military vacuums up most Internet and mobile phone data in the United States, all without warrants in violation of the Constitution. If their claims are true, the head of the NSA, General Keith Alexander, intentionally misled Congress about the agency’s surveillance capabilities.

• We have seen a recent blurring of jurisdictional lines between domestic and foreign, and increasingly fuzzy distinctions between police and military.

• Events over the past year have demonstrated the federal government’s interest in closely monitoring the Occupy Wall Street movement, and the lengths to which law enforcement at all levels will sometimes go to attack speech and assembly rights.

In light of such facts, it’s not clear we can put a lot of faith in the government when it says that it won’t use this tool to attempt to predict uprisings in the United States. The US security establishment has a record of lying and breaking the law.

My source who is connected to the OSI project told me that the government contracting world is always flush with military and intelligence funds, and that they throw lots of money at projects that never go anywhere. But this project carries an enormous amount of weight, the source said, and is coming from people who seem connected to the top. “The principals seem really excited about this one,” the source said. “They are really pushing for results, and fast. They are very excited about this. And sure, the test projects focus on Latin America, but they will inevitably use this to target the US.”

Why does this matter? If this is true, what’s the worst that could happen?

Imagine that the OSI program becomes a great success at predicting future events, and the NSA adds all of its electronic information into the system to keep an eye on economic, political and social indicators in the United States. What’s the big deal?

For one thing, even if the the OSI program proves to be a complete failure at predicting social movements, it could still become another rationale for gathering and combining information about Americans.

But it goes beyond that. Here’s a useful thought exercise:

What would the United States and Egyptian military have done had they known that the revolution boiling over in Tunisia was about to hit Cairo, Alexandria and Suez? It seems distressingly easy to imagine that the US government would have helped the Egyptian regime try to prevent the uprising using any means possible, whether by pre-emptively imprisoning key activists, offering piecemeal reforms to quiet unrest, or shutting down Internet organizing networks in advance of the demonstrations.

Had the program succeeded in predicting the revolt, and had the US government interfered to try to stop the Egyptian revolution from taking flight, that nation’s first Presidential elections in decades might not have happened last week. And here in the United States, there is enough historical evidence of security agencies engaging in surveillance and disruption of dissenting social movements for us to be concerned about how such a program might play out.