SUBSCRIBE NOW
IN THIS ISSUE
PIPELINE RESOURCES

Where does big data go when it dies (and why should we care)?

By: Jesse Cryderman

According to researcher and tech author Bernard Marr, every two days humanity creates as much data as it did from the beginning of time to 2003. Every minute of the day, Facebook users share nearly 2.5 million elements of content and WhatsApp users share around 350,000 photos. And the world is far from fully connected. When the next billion Internet users come online, these numbers will only skyrocket. Fold in the data generated by connected machines and the Internet of Things (IoT), and we're talking lots and lots of zeros. Pick your favorite data point: today we are literally swimming in data, and the pool grows deeper every week. And while much attention has been focused on the capture, storage, and manipulation of massive data sets, less frequent are stories about data effervescence and "data death". As our lives become increasingly digital, data burial grounds are just as important as cemeteries.

Where does data go when it dies?

Today, communications service providers (CSPs) increasingly turn to third-party applications and cloud services to enhance their offerings for both consumer and business customers and compete with their rivals. These services can be as simple as mobile backup, productivity tools like Evernote, or more complex device monitoring solutions like CarrierIQ. Each one of these services creates data records, and one of the biggest fears related to web-based service companies is that the company will die and take their data with it. Manufacturers of traditional software can go belly-up without it immediately affecting a product's usability, and hardware still works even when it's obsolete (this author still uses his archaic Microsoft Zune and Sony Dash). Cloud-based and web services are different, and if the cloud app evaporates, it sinks customers, too. Just ask anyone who has lost a whole day's worth of notes while using Evernote. 

 

Some companies make the best of a bad situation when such a service goes offline. In these cases, the company is able to shut down in an orderly way and inform users of the impending outage prior to nuking the servers.  The large digital service enterprises manage this well, for the most part. Yahoo, for example, shut down Yahoo Photos in favor of Flickr, but gave users plenty of time to offload their pictures, or move them to another service. When HP shuttered the cloud backup system Upline, it likewise gave users fair warning.

But what about smaller companies, especially companies who rely on cloud infrastructure themselves? These businesses may not be able to keep their servers up when their companies begin to fail, or they might not have servers at all!

When it comes to the death of data, companies need to think about creating a living will for their digital businesses. Where does the data go when a service dies? Is it effectively cremated, or does it reside in cryo-storage? One interesting concept is source code escrow. A startup company called Pogoplug gained some coverage a few years back with this cool and relatively simple solution. Pogoplug put the source code for its servers in escrow, and created provisions that would release the code to an open-source library in the event that the company went belly up. In this manner, the service could be “re-animated” in a cloud environment by a third-party. 



FEATURED SPONSOR:

Latest Updates





Subscribe to our YouTube Channel