Market research and analytics can sharpen business intelligence

DECEMBER 8, 2011 – In addition to giving a company more insight into how it can promote its goods and services to customers, conducting market research and analyzing big data can also increase business intelligence and help organizations determine how they can improve their operations.

According to a study from EMC Corporation, while many companies have collected vast amounts of unprocessed information – known as big data – just one-third of responding groups said their employees have the skills to “effectively use new data to assist their business decision-making, gain competitive advantage, drive productivity growth, yield innovation and reveal customer insights.”

The researchers surveyed nearly 500 data science professionals in the U.K., U.S., India, China, Germany and France, and found that while the supply of data pulled from medical imaging, smart grids, surveillance, mobile sensors and other sources has “exploded,” the number of people capable of analyzing that data has not kept up.

Additionally, EMC discovered that the trend is expected to continue in the coming years, with 65 percent of respondents saying the demand for data science talent will continue to exceed the available human resources for the next five years.

Two-thirds of respondents said they struggle to base business decisions on the data they collect, and the reasons cited include a lack of budget/resources, a dearth of training and tools/technology, as well as having the wrong organizational structure for the task.

Perhaps more worrisome is the finding that just 38 percent of the BI analysts and data scientists surveyed said their companies learned more about their customers through data.

“Making sense of big data is a combination of organizations having the tools, skills and more importantly, the mindset to see data as the new ‘oil’ fueling a company,” stated Andreas Weigend, the head of the Social Data Lab at Stanford University. “Unfortunately, the technology has evolved faster than the workforce skills to make sense of it and organizations across sectors must adapt to this new reality or perish.”

As Eric Knorr warned on InfoWorld, companies also need to ensure they are ready to capitalize on the “big data bubble.” He said with all the data that will be available through smartphones and social media, enterprising businesspeople will be able to create new services and companies tailored to specific consumers’ buying habits.

He noted that the biggest challenge to the economic recovery was a lack of demand, not an overload of debt. “Crunch on the behavior patterns of hundreds of millions of individuals long enough and you’ll get the ultimate big data outcome: knowing what any consumer wants to buy at any given second,” Knorr wrote.