The trick with this segment of the BDA "layer" is to choose a pragmatic vendor who can consolidate the key elements of the data management infrastructure from this open source market and bring it together in a secure and cost effective way. The most important aspects of this are (1) choosing and bundling the right mix of open source technologies; (2) productizing data collectors to as many critical data sources you have and making sure these sources are coordinated from release to release; (3) scrubbing and hosting the data in a clean manner which is analytics ready; and (4) integrating the infrastructure into the northbound applications and visualization capabilities. 

One of the drivers towards new infrastructure is the fact that Hadoop can provide a lower cost storage option than traditional data warehouses, particularly once SQL on Hadoop is available, which is the hottest area of Hadoop innovation right now. Most organizations analyze only about 12% of the data they hold, so the race is on to get faster, interactive SQL working well on Hadoop.  This doesn’t mean that the traditional data warehouse will cease to exist, but the CIO’s challenge is to cap the spend in the expensive data warehouse and relational database technologies, as more and more data needs to be retained and analyzed in a cost-effective manner.  Hadoop technologies are five to fifteen times less expensive for this function.

The Money

The top layer contains the actionable analytical applications and discovery capabilities. It is just a simple fact that machines can recognize patterns and trends that humans may miss. This does not mean that the machine is set to replace the human. Machine learning and Big Data technologies are designed to give timely advice and prompt users and systems to make smarter decisions.  

Analytical benefits span across three main domains for service providers. Of course there are additional uses, but the main areas that provide material business and bottom line performance improvements are the following:

1. Dynamic marketing and segmentation to allow CMOs to better promote the right services to the right people at the right time. 
2. Care and business performance for the optimization of the self-care portal, commerce e-shop, and call centers to lower call handling time and optimize call center operations. 
3. Network optimization for better connectivity experience while optimizing infrastructure investment.

Across all of these categories, there are a wide range of customer experience use cases which can analyze thousands of actual data points or KPIs to materially improve Net Promoter Score.  

This top layer brings new requirements that touch all service provider organizations, but has more profound impact on the CMO behavior and processes. CMOs in the past would be charged with championing the company brand and driving interest in the products and services that it produced. This would typically be executed through large, expensive and creative campaigns. Marketing now is depending on IT to supply contextual information about its customers so that the CMO organization can improve targeting and rare customer interaction effectiveness. This means that the CMO and the CIO will need to spend a lot more quality time together.

In 2012, Gartner made a prediction that at the time raised many an eyebrow. It forecasted that by 2017 the CMO would spend more on technology than the CIO. Although a transition to this degree is unlikely, the marketing department will clearly play a greater part in such technology decisions.

The Results

In conclusion, there are three layers of execution that need to be addressed to realize the full potential of Big Data. To date the programs have been fragmented and too much focus has been on the infrastructure and platforms. It is clear that, given the specific challenges in telecoms, more emphasis needs to be given to the quick, efficient extraction and ingestion of data and then the value added applications that need to be applied to drive the real value across all three of the vertical analytics application segments mentioned above. Only then will we even begin to start talking about applying these technologies to real financial benefits.