Are you taking these three vital steps to successfully prepare your business data from sources such as ERP, CRM, accounting systems, databases, and even Excel?
Before I go into what the steps are, let’s take a quick step back, and I’ll hit you with a couple of ‘did you knows’…
Did you know that 80% of data analysts’ time is spent preparing data? And only 9% of time is spent analysing data? In terms of what this means for businesses, did you know that eight billion hours per year are wasted on manual data preparation across Europe? And that €55 billion is spent – or wasted, I would say – on manual data preparation every year?
Incredible numbers. But when you look at what data analysts are faced with, it’s little wonder. They’ll be handling myriad data sources: business systems and structured data sources like ERP, CRM, HRM; databases like SQL, Oracle, and DB2; files like the omnipresent Excel and CSV. Their challenges will be heightened with the need for near-instant reporting, a common semantic layer, data governance and integration of all this data from all these sources.
Manual data preparation is made even more arduous when you consider that – of the multiple types you might already be dealing with – some will be on-premise, others in the cloud and some a hybrid of both. Some will be recently-installed systems, and some will be legacy (which, perhaps, only certain members of staff or expensive external consultants know how to handle).
It’s the 21st century and, to my mind, only an automated, software-based approach to data preparation will tackle all of these hurdles and – furthermore – tackle this dramatic overspend and waste of time that’s going in so many businesses.
So, my three vital steps for successful data preparation?
-
Automate your data management
The benefits of automating data management still remain largely untapped for businesses of all sizes and sectors. For this step, I’d advocate a move to pre-packaged software to automate the management of data across multiple sources, be it ERP, CRM, databases or any combination of them. Such software automatically accesses and unifies data from multiple apps and sources of any type, regardless of whether they’re in the cloud, on-premise or a hybrid combination of the two. And they employ familiar ‘wizard’-type interfaces that use familiar business terminology to make data modeling as easy as possible.
-
Automate your data integration
Why automate the data integration stage? Quite simply, because then nothing is missed, regardless of what level of business- or department-specific customisations have been made to the data. Such customizations might hitherto have turned data warehouse projects into lengthy, manual processes or deter attempts to unify data at all. Regardless of geography, file type, size or user-customized attributes, the best data management automation software has features including data profiling and recommendations, automatic merging of tables, multi-threaded and/or parallel migrations and incremental loading and updating.
-
Ensure data governance is underpinned by technology, not policy
Data governance is one of the most important considerations of any business and one which transcends technical functionality, lying at the heart of corporate best practice, customer trust and compliance with legal directives such as GDPR. And, as with so many other things, all of this is on its strongest, most rigorous footing if underpinned by technology, rather than people, process or policy.
I’ve found that data management automation software can control, audit and log all aspects of data governance and security. Everything from password and access control to reconfiguring mismatched data definitions.
The best software in this field, while being pre-packaged, should include enterprise-class controls, be able to set security permissions, regulate access and protect data at any level of granularity. Software can control data governance at four separate levels: user, resource, warehouse data and cell data. Data lineage can be automatically governed, too, with separate history tracking, auditing, and monitoring.
Who else is talking about this?
I’m not the only one preaching the automation message. Bloor Research would agree that an automated, self-service software-based approach is the key to efficient data preparation. “It allows (all business units) to prepare and perform ad hoc analyses without being reliant on IT,” they explain. “There are also major benefits for IT, not just in being able to monitor what users are doing but also in reducing IT workload.”
Also pointing the way to automated, self-service data preparation and data management software is GCN. They list 10 steps to speed the journey from data prep to insight and – point three is simply to “use technologies and methods to increase repeatability so that scripts, workflows, and other
elements can be reused.” And, to expand on their fifth and sixth points, their recommendation is to “evaluate new technologies enabling non-technical users to prepare data themselves” and “integrate self-service data preparation with self-service Business Intelligence and visual analytics.”
What about some real-life examples?
Ascom Network Testing has nearly 40% market share in the wireless communications drive-test market and more than 650 customers in 140+ countries. Data preparation is the ultimate responsibility of Ash Aly, Senior Global ERP Manager who oversees 13 business units worldwide, all maintained within a Microsoft Dynamics AX landscape. They’ve automated data management — using pre-packaged software – to reduce data preparation times and, ultimately, reporting times.
Over at Bluefish Technologies, we find a company heralded as the fastest growing smart card supplier in the world. They manufacture more than 300 million SIMs per year and have offices across EMEA, Asia-Pacific and Latin America. Prior to automating, Bluefish took a manual, time-consuming approach to accessing, unifying and preparing ERP data for budgeting and credit risk management.
Now, implementation of pre-packaged data management software has successfully automated data preparation and other data management processes for Bluefish. As Thomas Hoffman, the company’s Group Finance Director summarizes, “we are able to make decisions immediately, based on information that was previously impossible to access.”
So, the bottom line?
Having looked at everything from GDPR to the world’s biggest manufacturer of SIMs automating their data management processes, I hope this article gives you food for thought… And I hope you might be a little closer to answering my opening question – “Are you taking these three vital steps to successfully prepare your data?” My view is simple, with software, the answer can be a simple yes. Software-based automation of data preparation conquers all these challenges and delivers trust — in resulting reports and BI analysis, time savings – more time on analysis, less time on manual integration! – accuracy (removing the potential for human error) and governance.
About the Author
Garth Laird is CEO of ZAP. Born and raised in Auckland, New Zealand, Garth’s been with ZAP for more than 10 years. Previously COO, Garth took the position of President & CEO in March 2016 and, as IDC reports, “successfully struck up a strategic partnership with Sage… the arrangement sees ZAP embedded as the data management platform for the Sage ERP portfolio.”
Garth has overseen the development of ZAP’s data management software and, most importantly, he’s overseen its adoption by customers in almost 50 countries, from mid-size businesses to Fortune 500 and FTSE 100 organisations.
