The Unstructured Data Structure
By: Nancee Ruzicka
In a 2013 report, FICO cautions “80% of big data is unstructured.” If that’s the case, then why are we spending so much time and energy on structured data? The answer, it seems, is that the data currently most valuable to service providers is structured. Service providers collect data from the network for important tasks like billing, revenue assurance, policy management, service assurance, and fulfillment. For the majority of service provider operations, structured data captured from customer transactions and network elements reveals everything they need to know. Structured data is continuously collected and saved, so data bases and warehouses are readily available to be searched and analyzed to uncover trends and spot opportunities. In fact, service providers are required by law to capture and store customer transaction data and they often maintain many years’ worth of data for each customer.
While revenue assurance and fraud management rely almost exclusively on transaction data and network factors to analyze user behavior, find lost revenue, and determine the sources of fraudulent activity, accurately analyzing that same data to determine marketing behavior and evaluate the customer experience requires the inclusion of unstructured data. Traditional solution providers, such as cVidya, that use customer transaction and network data for fraud management and revenue assurance, are now expanding into the marketing analytics space. This new line of business builds upon existing solutions; however, these vendors also recognize that access to the unstructured data relating to customer behavior and attitudes is critical to service providers as they define CEM (Customer Experience Management) strategies to reduce churn and increase up sell. Other vendors, such as AsiaInfo, are providing "contextual awareness" to integrate both static and dynamic data sources to provide service providers with valuable customer insights.
About CES 2014
Pipeline continues its legacy of bringing together the world’s leading service providers and technology innovators this fall at The 2014 COMET Executive Summit. This exclusive event
gathering Pipeline journalists, Industry Advisory Board (IAB) Members, and key solution providers will be an intimate symposium to shape the editorial direction of Pipeline, gather
priceless input from executive-level service provider experts, and create lasting industry relationships.
Pipeline’s IAB is an exclusive group of service provider and analyst executives who have long-term relationships with Pipeline and have played a role in Pipeline programs, editorial
direction, and provided content over the last decade. This year, Pipeline opens the doors to provide an opportunity to engage directly with a broad cross section of experts who
evaluate, recommend, and purchase communications and entertainment technology (COMET) products and services. Multiple levels of participation provide your company with an exclusive
networking opportunity, tailored to your goals and budget.
The COMET Executive Summit will bring together executives from the world’s leading service provider and technology companies, in a flexible format that is filled with unprecedented
networking opportunities designed to build relationships that can be carried forward to solve issues facing service providers today. Some of the topics planned for discussion
include:
- Networking
- Customer experience management (CEM)
- Big data & analytics
- Exploring cloud offerings
- Enabling new business models
- Delivering and assuring digital services
- Network evolution & virtualization
- Leveraging content
- Network security
For more information, visit
www.pipelinepub.com/info/comet/2014_comet_summit.php
Incorporating unstructured data is producing results. Several service providers report success in identifying customers likely to churn by using advanced analytics and integration with unstructured data. One of these service provider identified 75% of the customers that were likely to churn by targeting only a specific 10% of its total customers as indicated by the analysis. That segment is now the focus of offers and support designed to improve their unique customer experience.
Complex events require complex solutions
One of the most difficult challenges service providers face is data volume. Service providers collect massive amounts of data and that in and of itself can be one of the biggest challenges facing them today. Vodafone India, for example, collects 32 billion records per day and the largest service providers regularly collect 40 million records per second!
Much of the data that is readily available to service providers is structured. Data about billions of transactions, millions of customers, thousands of devices and hundreds of products are all captured or calculated in a consistent manner. However, what service providers are missing, and what is becoming valuable to marketing and product management functions, is the unstructured data regarding perception, problems, and desires being expressed by customers during calls to customer care, online during chat sessions, or in online and social networking communities like Facebook and Twitter. There are also additional clues that can be found in videos and promoter index scoring that could be valuable to a service provider if properly decoded and analyzed.
Processing billions of user transactions, correlating and distributing that data to multiple OSS and BSS solutions, ensuring partner settlements, eliminating fraud, and improving the customer experience are all drivers for implementing big data management solutions like Hadoop. But there’s more to it than data storage and computing power. Managing the volume of events, processing customer transaction records, and correlating unstructured data requires complex event processing that collects and analyzes every scrap of data available via network elements, probes, and partner elements.
