Growing investment in big data initiatives still slow in showing value
Big data is getting bigger, but big results may not have arrived quite yet. According to a recent survey of big data professionals, enterprises are investing significantly in big data initiatives.
Conducted by business analytics leader SAS, the survey reports 83 percent of respondents call their company’s investment “moderate” or “significant.” However, many of these projects are still in the early stages: Just 33 percent say that they have derived value. Another 49 percent say it’s just too soon to tell.
“The good news is that large organisations understand the importance of big data solutions and are taking steps in the right direction,” said Matthew Magne, Global Product Marketing Manager for Data Management at SAS. “But while the trend is encouraging, many of these companies still have a long way to go; data access and quality remain challenges.”
Respondents generally believe that big data will provide their company with a competitive advantage. At the same time, they point to challenges: data integration, complexity and integrity, as well as unstructured data capture and management. Other issues are associated with culture, skills and access to appropriate technology.
Data access
For many organisations, data access belongs to the “data crunchers” – data scientists and business analysts. Only 13 percent have self-service access without IT supervision. About a third of respondents have instant access to the data they need. But 28 percent say it can take up to a week, 8 percent a month, and 3 percent a month or more.
Just under half of companies surveyed have a chief data officer. Others in IT are typically in charge of the organisation’s data.
Moving to the cloud
Cloud adoption has been slow – just 19 percent of organisations are accessing data in the cloud. The rest continue storing data on-site. Hadoop is on the rise – 56 percent of those actively engaged in a big data initiative have purchased a Hadoop-centered solution. Of those, 87 percent have implemented it. The most popular Hadoop distribution is Cloudera at 44 percent, followed by Hortonworks at 29 percent. Another 40 percent are considering Hadoop to replace an existing data warehouse. Six in ten companies have increased spending on data infrastructure over the past two years.
“The movement to cloud storage is still somewhat guarded, even among those already entrenched in big data. Other research indicates that the biggest concern with cloud storage is security,” added Magne.
Machine learning and artificial intelligence
About one in five companies have implemented machine learning or artificial intelligence (AI) technology. Another 23 percent have tried machine learning or AI on an experimental basis. The largest segment, 42 percent, are exploring AI solutions but have yet to invest.
“Nearly all respondents are seeing traction from their data-driven projects, with about half saying they have many data success stories,” said Magne. “Overall, the survey shows that big data initiatives are making progress and continuing to gain traction.”
SAS surveyed business and technology personnel at large US-based organisations who are familiar with their company’s big data initiatives.
