During the 2015-16 English Premier League soccer season, underdog Leicester City surprised many by capturing the title for the first time ever. But going into the next season, the team struggled. Coaches and managers were challenged to understand why the team was not dominating as it had the year prior.

Soccer is a complex and strategic game, and common statistics rarely tell the full story of how a team succeeds and fails. But working with STATS LLC, a sports data company, and employing machine learning tools and a plethora of data, Leicester City coaches were able to gain more insight into the team’s performance.

"The assumption was they were great offensively, but the data actually showed otherwise. It was actually their defensive prowess that was crucial to their victory," said Patrick Lucey, chief data scientist at STATS. "Their organized defensive structure allowed them to reduce the quality of their opponents’ chances across the different scoring methods. The data highlighted their disruptive game that made them one of the most difficult teams to attack against."

Armed with data, Leicester City coaches and managers honed those strengths and refocused the team. 

Data is not just for sports teams. Many refer to data as the new oil and there is much talk about how to apply Big Data insights to the enterprise. But many organizations struggle to implement Big Data programs. Looking at how sports teams are using Big Data could provide enterprises insight into how Big Data can help them find their own winning formulas.

The magic in the data

Lucey believes that because data has already disrupted the status quo in sports, businesses can learn a lot from the sports industry when it comes to determining how to leverage Big Data.

For example, by structuring the millions of data points within video footage, STATS can model how athletes react to others on the field by looking at thousands of prior instances as models — giving them the ability to test new plays through machine learning. 

"In sports, the data is big, but it’s granular, so that allows us to model specific applications in context," said Lucey. "It allows teams to answer very specific questions."

For example, in terms of soccer, given a specific shot, what’s the likelihood the team will score?

"The data can tease it out, then we can come to understand the data we have and find the context that leads to those very specific answers," said Lucey. "It’s a tremendous time saving tool that also helps teams and coaches do their jobs better."

Another sports data company, Sportradar, is helping National Football League teams better analyze large data sets with an analytics tool it calls radar360. The web-based, statistical analysis application is currently used by all the NFL teams and by NFL media properties to analyze a combination of in-game statistics, historical stats dating back to 1920 and post-game subjective statistics.

"It provides NFL teams the capability to slice and dice data using endless combinations of filters and scenarios to very quickly gain deep analysis and insight on their team or on their opponents," said Ashok Balakrishnan, senior vice president of product management and technology.

Business Big Data challenges

Clearly, businesses are interested in Big Data and the competitive edge it can provide them. Last December, AtScale found that 97% of businesses expect to do "as much or more" with Big Data over the next three months. Two-thirds of the more than 2,550 respondents said they see Big Data as "strategic," while only 19% said they consider Big Data experimental.

The market is expected to grow rapidly and by 2020, IDC projects revenue from Big Data and analytics to reach more than $203 billion.

Launching an effective Big Data program is not an easy undertaking. Companies often struggle to get enough data to draw reasonable conclusions. But STATS, for example, has collected sports statistics for the last 35 years, so it has had an easier go with its programs because it has all that data at its disposal already.

Even when a company does have enough data, it often remains siloed. "It is difficult to extract meaning or patterns from data as each dataset is categorized differently," said Adnan Mahmud, CEO of LiveStories.

While datasets are usually formatted similarly (i.e. in Excel) they are often like different languages and need to be translated. Once the datasets are translated into the same "language," patterns and important insights are illuminated.

Another challenge to using data is the amount of time it takes to clean and find data.

"Most people spend 80% of their time finding and cleaning the data, leaving little time for exploring and sharing," said Mahmud. "Moreover, even when the data is available, like through the Census, it can be very difficult for non-technical people to use."

Finally, demand for Big Data analytics skills is going up, while a deficit on the supply side is driving salaries higher.

"Companies need people that can take the data they have and turn it into useable information," said John Reed, senior executive director at Robert Half Technology. "What kind of info do we have? How do you package that up in a way to deliver that back to the business to help them in making business decisions?"

Big Data engineers — those personnel often tasked with figuring out what to do with the data companies collect and how to use that data to advance enterprise capabilities and offerings — will see one of biggest non-executive salary jump among all IT professions this year, according to the 2017 Robert Half salary guide. 

Practical applications

Overcoming Big Data obstacles is worth the effort. For businesses, drawing new insights from vast data troves can help them stay more clued into customer bases. And provided they can hone in on customer wants and needs, those customers are likely to keep coming back.

Evidence shows businesses are currently using Big Data and analytics in a number of ways, from identifying risk and fraud to creating product insights organizations can act on. Business Intelligence is the number one workload for Big Data, with 75% of respondents planning on using BI on Big Data, according to AtScale. 

Though the potential for Big Data is enormous, even in the sports industry the application of Big Data is still very much in its infancy.

"Even after multiple years, we are still just scratching the surface in terms of what knowledge we can derive from all this data," said Lucey.