Big Data challenges hampering market research initiatives

JANUARY 6, 2012 – When it comes to conducting market research for product development or advertising purposes, the advent of social media and internet data technologies has led to the onset of Big Data. This term, which refers to the massive volumes of names, numbers and other factors that can be mined for business insight, has the potential to improve companies’ operations and enable them to work smarter and more efficiently.

Yet being able to harness the power of Big Data brings its own challenges – including defining what it is. A survey from Connotate found that while 49 percent of data aggregation managers think it is “all of the external and internal web-based data available for business intelligence, another 21 percent said they were not even sure what the term was referring to.

In an article on the study, eMarketer noted that the latter figure proves Big Data’s “status as something of a new, unfamiliar concept for many companies.”

Connotate’s research also revealed that companies are struggling to overcome other obstacles, perceived or otherwise, including having enough human resources to gather and assess the data (45 percent). In addition, 44 percent said they were concerned they would be overwhelmed by the sheer volume of information and be unable to effectively apply it toward business strategy.

While larger companies seem to be ahead of the curve in terms of adopting Big Data, mid-market groups that employ 100 to 10,000 people have been “surprisingly lackluster” when it comes to integrating web data into their decision-making processes and business intelligence. 

“Big Data goes beyond technology and is an HR challenge for corporate America,” said Connotate CEO Tom Meyer. “While it is important that organizations devote resources to Big Data, employees must be freed from the information fire hose so they can concentrate only on the information that is relevant to their tasks.”

Indeed, company leaders are taking a bigger interest in the data, with 27 percent of executives playing a role in how their organizations collect data and set their budgets.

While a small group – 4 percent – said they spent $10 million or more on data aggregation every year, a larger proportion (58 percent) reported spending approximately $500,000 on web data aggregation. Out of the surveyed enterprises, representing companies of 10,000 employees or more, 70 percent engaged in competitive monitoring through web data. Just 46 percent of mid-market organizations could say the same.

In a recent column for Marketing Pilgrim, Frank Reed also analyzed Connotate’s findings.

He discusses the “big headaches” that come from Big Data, especially for organizations that are running marketing campaigns with a heavy content element. The marketing department may take comfort in the fact that they’re publishing “great content,” but without hard numbers (supplied by Big Data) to prove to the C-suite that these efforts are developing a return on investment, the program will lose its funding, fast.

“What is happening is that businesses have more data than ever at their disposal but it’s not manageable, thus it can almost seem useless,” Reed writes. “It’s like having a huge uncut diamond but having no way to refine it to the point that it realizes its true market value.”

Reed thinks this is a handicap that companies cannot afford to ignore. Not being able to make sense out of the huge quantities of data a business is collecting means it will be basing important decisions on small amounts of (sometimes inaccurate) information.