The Federal Big Data Factory
By: Jesse Cryderman
When Edward Snowden, a former contractor for the National Security Agency (NSA), alleged in early June that AT&T, Sprint, Google, Skype,
Facebook, and many other companies had violated the public trust by participating in a secretive effort aimed at capturing and recording all communications data, he ignited the hottest
story (so far) of 2013. Since then the saga
has unfolded like a Hollywood blockbuster, complete with an offer of asylum for Snowden from Russia, further information leaks, geopolitical posturing, intense media scrutiny, and a growing fear of government espionage.
There’s just one difference: This is all real. Very real.
Communications service providers (CSPs) are stuck in the middle. On the one hand, they want to remain transparent with their customers. On the other hand, they certainly have the tools they need to capture all communications, and many have reportedly signed letters of nondisclosure with the NSA in order to prevent discussion of such tools.
Many serious questions have arisen in light of these developments: Just how big is the United States government’s Big Data factory? What types of data can be collected and stored, and how long can they be stored? And in response, how has this story impacted the businesses of Big Data, communications and security?
The allegations
Much ink has been spilled describing PRISM, the so-called cybersurveillance program that grants the US government direct access to telecommunications network cables, enabling the NSA to funnel off all packets of data that originate or terminate with foreign actors who have been targeted for potential terrorist associations. Since June additional information has come to light, specifically in regard to domestic surveillance and the types of programs used by the US, and it’s particularly worrisome because the US Constitution ostensibly protects American citizens from such surveillance.
Ragtime is the domestic surveillance project in the US, and there are four variants, according to official documents. Of those four, the most disturbing is Ragtime-P—the P is short for “Patriot,” as in the USA Patriot Act—which grants investigators warrantless access to private communications between US citizens. The information that’s gathered is then allegedly stored for five years in a database named Marina.
In July the existence of a program called XKeyscore was revealed by Edward Snowden and confirmed by retired US Air Force general Michael Hayden, who was head of the NSA from 1999 to 2005. XKeyscore grants wholesale visibility into private communications; some of its capabilities are illustrated below in figure 1.
Figure 1
XKeyscore is still very much alive and well. Although the postings have since been removed, top defense contractors SAIC and Raytheon had job openings listed for XKeyscore engineers at the beginning of August; a screenshot of one such posting was captured by The Washington Post. The program is also widely deployed throughout the world, as illustrated in figure 2.
Figure 2
