You are here

Sharpen your Big Data reflexes

By Ashley Stirrup, Talend and Bart Gardeyn, Aprico Consultants

National Hockey League (NHL) goalies have cat-like quickness, quite literally, according to the smart folks at Sport Science that track such things. They have detailed how an NHL goalie is able to respond to a shot in a staggering 1/10 of a second, which is about the same reaction time as a cheetah. This incredible feat is achieved based on highly attuned reflexes or an almost automatic response to a puck hurtling toward them at 80mph.

Looking at big data analysis today, most companies are striving to be faster, but what they really need to become is NHL goalie quick. This is particularly true when it comes to customer interactions. With customers, every moment truly matters. Therefore, the time it takes a company to turn information on the customer (e.g. website navigation) into action can be the difference between making a sale or losing a customer.

Unfortunately, examples of companies with these fine-tuned reflexes are far and few between today. It’s historically been just folks like Netflix, Google and Amazon – massive organizations with the large IT budgets and teams. These organizations have been able to maintain a competitive advantage by delighting customers and almost anticipating their needs with things like predictive search results and personalized offers or recommendations.

Another company, an e-commerce giant in Europe, is using its ability to convert massive volumes of data and streaming information into real-time action in order to tackle the major challenge of shopping cart abandonment. It is estimated that this year alone, $4 trillion worth of merchandise will be left in carts. Using real-time data crunching, the company is able to predict with 90% accuracy when a customer is going to abandon a shopping cart and then send real-time offers to help close the sale.


Recently, for the first time, some key technology ingredients came together to allow a far broader range of companies to create the supply chain necessary to support real-time Big Data analytics. For example, Apache Spark, an open standard for flexible and very fast in-memory data processing, can be coupled with other software tools to give you all the pieces necessary for any company to efficiently and cost effectively move as quickly as an NHL goalie’s glove hand.

This is a substantial market change that has broad consequences. The value proposition for companies is easy to understand. Knowing what their customers were doing last month or week is interesting; however, it’s infinitely more meaningful to understand real-time behaviors. And, better yet, to be able to respond to customers’ needs in an instant and fundamentally transform their experience for the better. 

While e-commerce is an obvious fit for real-time customer insight and management, there is no shortage of compelling use cases across virtually every market segment – from fraud detection in the financial sector to optimizing IoT-equipped wind turbines in the field based on current weather conditions (ed. note: Internet of Things). The healthcare industry is also ripe for being reshaped by the availability of real-time Big Data. Whether you look at diagnosis or research and development, faster access to more data points will help drive better results.

Spark-powered data integration, coupled with Spark-enabled analytics, is a game changer. Companies with the speed and agility of an NHL goalie will absolutely have a significant market advantage. Beyond the obvious opportunities around revenue optimization and customer relations, just about every aspect of business – from manufacturing and supply chain management to human resources – can benefit from instant decision-making.