big data

big data

Contents 

Part I 

General 1 Strategic Applications of Big Data

Joe Weinman 

2 Start with Privacy by Design in All Big Data Applications

Ann Cavoukian and Michelle Chibba 

3 Privacy Preserving Federated Big Data Analysis 

Wenrui Dai, Shuang Wang, Hongkai Xiong, and Xiaoqian Jiang 

4 Word Embedding for Understanding Natural Language: A Survey Yang Li and Tao Yang Part II Applications in Science

Part II 

Applications in Science 

5 Big Data Solutions to Interpreting Complex Systems in the Environment 

 Hongmei Chi, Sharmini Pitter, Nan Li, and Haiyan Tian

 6 High Performance Computing and Big Data

Rishi Divate, Sankalp Sah, and Manish Singh 

7 Managing Uncertainty in Large-Scale Inversions for the Oil and Gas Industry with Big Data 

Jiefu Chen, Yueqin Huang, Tommy L. Binford Jr., and Xuqing Wu

 8 Big Data in Oil & Gas and Petrophysics 

 Mark Kerzner and Pierre Jean Daniel 

9 Friendship Paradoxes on Quora

 Shankar Iyer 

10 Deduplication Practices for Multimedia Data in the Cloud 

 Fatema Rashid and Ali Miri 

Chapter 1 Strategic Applications of Big Data

by Joe Weinman

Introduction

For many people, big data is somehow virtually synonymous with one application— marketing analytics—in one vertical—retail. For example, by collecting purchase transaction data from shoppers based on loyalty cards or other unique identifiers such as telephone numbers, account numbers, or email addresses, a company can segment those customers better and identify promotions that will boost profitable revenues, either through insights derived from the data, A/B testing, bundling, or the like. Such insights can be extended almost without bound. For example, through sophisticated analytics, Harrah’s determined that its most profitable customers weren’t “gold cuff-linked, limousine-riding high rollers,” but rather teachers, doctors, and even machinists (Loveman 2003). Not only did they come to understand who their best customers were, but how they behaved and responded to promotions. For example, their target customers were more interested in an offer of $60 worth of chips than a total bundle worth much more than that, including a room and multiple steak dinners in addition to chips. While marketing such as this is a great application of big data and analytics, the reality is that big data has numerous strategic business applications across every industry vertical. Moreover, there are many sources of big data available from a company’s day-to-day business activities as well as through open data initiatives, such as data.gov in the U.S., a source with almost 200,000 datasets at the time of this writing. To apply big data to critical areas of the firm, there are four major generic approaches that companies can use to deliver unparalleled customer value and achieve strategic competitive advantage: better processes, better products and services, better customer relationships, and better innovation.

Better Processes

Big data can be used to optimize processes and asset utilization in real time, to improve them in the long term, and to generate net new revenues by entering new businesses or at least monetizing data generated by those processes. UPS optimizes pickups and deliveries across its 55,000 routes by leveraging data ranging from geospatial and navigation data to customer pickup constraints (Rosenbush and Stevens 2015). Or consider 23andMe, which has sold genetic data it collects from individuals. One such deal with Genentech focused on Parkinson’s disease gained net new revenues of fifty million dollars, rivaling the revenues from its “core” business (Lee 2015).

Better Products and Services

Big data can be used to enrich the quality of customer solutions, moving them up the experience economy curve from mere products or services to experiences or transformations. For example, Nike used to sell sneakers, a product. However, by collecting and aggregating activity data from customers, it can help transform them into better athletes. By linking data from Nike products and apps with data from ecosystem solution elements, such as weight scales and body-fat analyzers, Nike can increase customer loyalty and tie activities to outcomes (Withings 2014).

 Better Customer Relationships 

Rather than merely viewing data as a crowbar with which to open customers’ wallets a bit wider through targeted promotions, it can be used to develop deeper insights into each customer, thus providing better service and customer experience in the short term and products and services better tailored to customers as individuals in the long term. Netflix collects data on customer activities, behaviors, contexts, demographics, and intents to better tailor movie recommendations (Amatriain 2013). Better recommendations enhance customer satisfaction and value which in turn makes these customers more likely to stay with Netflix in the long term, reducing churn and customer acquisition costs, as well as enhancing referral (wordof-mouth) marketing. Harrah’s determined that customers that were “very happy” with their customer experience increased their spend by 24% annually; those that were unhappy decreased their spend by 10% annually (Loveman 2003).

Better Innovation

Data can be used to accelerate the innovation process, and make it of higher quality, all while lowering cost. Data sets can be published or otherwise incorporated as part of an open contest or challenge, enabling ad hoc solvers to identify a best solution meeting requirements. For example, GE Flight Quest incorporated data on scheduled and actual flight departure and arrival times, for a contest intended to devise algorithms to better predict arrival times, and another one intended to improve them (Kaggle n.d.). As the nexus of innovation moves from man to machine, data becomes the fuel on which machine innovation engines run. These four business strategies are what I call digital disciplines (Weinman 2015), and represent an evolution of three customer-focused strategies called value disciplines, originally devised by Michael Treacy and Fred Wiersema in their international bestseller The Discipline of Market Leaders (Treacy and Wiersema 1995).

From Value Disciplines to Digital Disciplines

The value disciplines originally identified by Treacy and Wiersema are operational excellence, product leadership, and customer intimacy.

Operational excellence entails processes which generate customer value by being lower cost or more convenient than those of competitors. For example, Michael Dell, operating as a college student out of a dorm room, introduced an assembleto-order process for PCs by utilizing a direct channel which was originally the phone or physical mail and then became the Internet and eCommerce. He was able to drive the price down, make it easier to order, and provide a PC built to customers’ specifications by creating a new assemble-to-order process that bypassed indirect channel middlemen that stocked pre-built machines en masse, who offered no customization but charged a markup nevertheless.

Product leadership involves creating leading-edge products (or services) that deliver superior value to customers. We all know the companies that do this: Rolex in watches, Four Seasons in lodging, Singapore Airlines or Emirates in air travel. Treacy and Wiersema considered innovation as being virtually synonymous with product leadership, under the theory that leading products must be differentiated in some way, typically through some innovation in design, engineering, or technology.

Customer intimacy, according to Treacy and Wiersema, is focused on segmenting markets, better understanding the unique needs of those niches, and tailoring solutions to meet those needs. This applies to both consumer and business markets. For example, a company that delivers packages might understand a major customer’s needs intimately, and then tailor a solution involving stocking critical parts at their distribution centers, reducing the time needed to get those products to their customers. In the consumer world, customer intimacy is at work any time a tailor adjusts a garment for a perfect fit, a bartender customizes a drink, or a doctor diagnoses and treats a medical issue.

Traditionally, the thinking was that a company would do well to excel in a given discipline, and that the disciplines were to a large extent mutually exclusive. For example, a fast food restaurant might serve a limited menu to enhance operational excellence. A product leadership strategy of having many different menu items, or a customer intimacy strategy of customizing each and every meal might conflict with the operational excellence strategy. However, now, the economics of information— storage prices are exponentially decreasing and data, once acquired, can be leveraged elsewhere—and the increasing flexibility of automation—such as robotics— mean that companies can potentially pursue multiple strategies simultaneously

Digital technologies such as big data enable new ways to think about the insights originally derived by Treacy and Wiersema. Another way to think about it is that digital technologies plus value disciplines equal digital disciplines: operational excellence evolves to information excellence, product leadership of standalone products and services becomes solution leadership of smart, digital products and services connected to the cloud and ecosystems, customer intimacy expands to collective intimacy, and traditional innovation becomes accelerated innovation. In the digital disciplines framework, innovation becomes a separate discipline, because innovation applies not only to products, but also processes, customer relationships, and even the innovation process itself. Each of these new strategies can be enabled by big data in profound ways.

Information Excellence

Operational excellence can be viewed as evolving to information excellence, where digital information helps optimize physical operations including their processes and resource utilization; where the world of digital information can seamlessly fuse with that of physical operations; and where virtual worlds can replace physical. Moreover, data can be extracted from processes to enable long term process improvement, data collected by processes can be monetized, and new forms of corporate structure based on loosely coupled partners can replace traditional, monolithic, vertically integrated companies. As one example, location data from cell phones can be aggregated and analyzed to determine commuter traffic patterns, thereby helping to plan transportation network improvements.

Solution Leadership

Products and services can become sources of big data, or utilize big data to function more effectively. Because individual products are typically limited in storage capacity, and because there are benefits to data aggregation and cloud processing, normally the data that is collected can be stored and processed in the cloud. A good example might be the GE GEnx jet engine, which collects 5000

data points each second from each of 20 sensors. GE then uses the data to develop better predictive maintenance algorithms, thus reducing unplanned downtime for airlines. (GE Aviation n.d.) Mere product leadership becomes solution leadership, where standalone products become cloud-connected and data-intensive. Services can also become solutions, because services are almost always delivered through physical elements: food services through restaurants and ovens; airline services through planes and baggage conveyors; healthcare services through x-ray machines and pacemakers. The components of such services connect to each other and externally. For example, healthcare services can be better delivered through connected pacemakers, and medical diagnostic data from multiple individual devices can be aggregated to create a patient-centric view to improve health outcomes.

Collective Intimacy

Customer intimacy is no longer about dividing markets into segments, but rather dividing markets into individuals, or even further into multiple personas that an individual might have. Personalization and contextualization offers the ability to not just deliver products and services tailored to a segment, but to an individual. To do this effectively requires current, up-to-date information as well as historical data, collected at the level of the individual and his or her individual activities and characteristics down to the granularity of DNA sequences and mouse moves. Collective intimacy is the notion that algorithms running on collective data from millions of individuals can generate better tailored services for each individual. This represents the evolution of intimacy from face-to-face, human-mediated relationships to virtual, human-mediated relationships over social media, and from there, onward to virtual, algorithmically mediated products and services.

Accelerated Innovation

Finally, innovation is not just associated with product leadership, but can create new processes, as Walmart did with cross-docking or Uber with transportation, or new customer relationships and collective intimacy, as Amazon.com uses data to better upsell/cross-sell, and as Netflix innovated its Cinematch recommendation engine. The latter was famously done through the Netflix Prize, a contest with a million dollar award for whoever could best improve Cinematch by at least 10% (Bennett and Lanning 2007). Such accelerated innovation can be faster, cheaper, and better than traditional means of innovation. Often, such approaches exploit technologies such as the cloud and big data. The cloud is the mechanism for reaching multiple potential solvers on an ad hoc basis, with published big data being the fuel for problem solving. For example, Netflix published anonymized cu

Today, machine learning and deep learning based on big data sets are a means by which algorithms are innovating themselves. Google DeepMind’s AlphaGo Goplaying system beat the human world champion at Go, Lee Sedol, partly based on learning how to play by not only “studying” tens of thousands of human games, but also by playing an increasingly tougher competitor: itself (Moyer 2016).

Value Disciplines to Digital Disciplines

The three classic value disciplines of operational excellence, product leadership and customer intimacy become transformed in a world of big data and complementary digital technologies to become information excellence, solution leadership, collective intimacy, and accelerated innovation. These represent four generic strategies that leverage big data in the service of strategic competitive differentiation; four generic strategies that represent the horizontal applications of big data.

click here for next chapter 1.3