Google ‘Is there such a thing as too much data’, and the resounding answer will likely be yes.
Data overload is a common problem that businesses face. Whilst data can be a company’s strongest asset and its unique point of differentiation, too much data can also overwhelm decision making – especially with businesses gathering data faster than they know how to use it. A study found that 73% of business leaders admit that both the sheer volume and lack of trust in the data available to them has caused decision paralysis. What’s more, nearly all decision makers surveyed (92%) believe that a growing number of data sources does not equate to success.
When you look at it like this, one might be able to understand why this paradox might tempt business leaders to look for easier, shorter-term solutions that can solve their immediate data challenges. But the reality is that no organization will be able to reach a perfect state of data overnight. This is why it’s important to get the foundations for managing constant flows of data in place. Think of it like this: It’s about having the right system architecture, data cleansing and data quality in place to derive as much value from the data as possible.
Assigning swimming lanes to your pool of data
Each business’ pool of data is continuing to get deeper and wider – constantly being added to with more internal and external data. Data and analytics teams are looking to sift through these pools, to understand its formula and direction. Data teams want to redirect some of the pools into different sized pools within their establishment, making the volume of data in the first pool less overwhelming, to help them categorize it in a way that is more understandable for human eyes. However, the result is that products and services often work in isolation from the organization’s larger dataset. That means the organization will lack oversight into what data they have, where it is and, most importantly, what it means. This means that while an organization has a whole swimming pool full of useful data, there is a general lack of trust in how it can be used to make business decisions.
Instead of building different sized pools, CDOs should work to build lanes across one pool. That way, rather than keeping data separated in a variety of pools, disparate data can be managed in open sight. Out of 97% of organizations that invest in data initiatives, only a quarter report successfully becoming a “data-driven organization” (New Vantage 2022). Yet managing data is essential to the success of an organization, particularly nowadays.
And the biggest problem that they continue to have is unifying disparate data at scale. Internal and external data, all of which refers to real people and businesses, need to be operating in the same plane, so that we can paint a clear and conclusive picture of what’s going on. That’s where Entity Resolution comes in.
Enter: Entity Resolution
Entity Resolution (ER) is the ability to resolve a 360-degree view of individuals, businesses and products. ER software can be used across industries and sectors, making as much of a difference for public sector organizations and government agencies as it can for financial institutions and private banks. At the core of ER is eliminating duplicate records and combining records that belong to the same person or entity. ER does this by comparing and combining data from multiple systems, producing the most accurate match possible. This allows an organization to create meaningful and accurate descriptors of each person or entity from a vast dataset with ambiguities and siloes.
Using banks as an example, ER software can process all of a customer’s transactions, and bring them into one picture. A network of transactions can not only provide insight into the behaviors of a customer, but it can explain relationships they have with other internal and external data sources. Where a collection of swimming pools will put a bank’s data into siloed boxes that are difficult to understand, ER brings each customer to the surface.
Traditional master data management approaches require data transformation exercises to bring the data to light, and even then, it doesn’t have the same context as ER creates. For example, MDM will typically focus on providing a single customer view and will therefore have no view on non-customers, such as suppliers or transacting parties. Furthermore, it will be highly strict in its matching and would miss some of the weaker connections. These are critical for detecting criminal activity.
The power of context in decision intelligence
The word continuously mentioned when dealing with data is context. While data can inform business leaders’ decisions, they can’t trust the decisions they’re making without the context to support. No decision can stand on its own, particularly as decisions become more complex – according to Gartner, 65% of executives say the decisions they are making are more complex and 53% are facing increasing pressure to justify their decision making. Applying context to an existing customer, for example, would mean gaining insight into not only that customer relationship when they sign up for a new product, but to give us insight into the relationship that the customer might have with people, organizations and places all of which could be relevant to inform decision making.
To keep the example within banking, contextual monitoring allows banks to see a complete view of a customer’s transactions and flag any transactions that breach the norm. However, contextual monitoring also has the unique ability to spot risk from opportunity. For example, with contextual monitoring, a bank can distinguish when a customer is taking out a large sum deposit for a new office building, rather than assuming that a change in financial behaviour means that they are committing a financial crime. This is enhanced when the bank can identify the network of interactions associated with this large transaction.
A wider and richer view of a customer allows a bank to assess whether the customer is in a high-risk geography, as well, or understand whether a customer is associated with negative news. The bank can then expose the risk and elevate the alert to be dealt with, as it has the full picture to prove why the transaction is raising alarms. This level of decision making is key in a banking landscape of increased financial crime, though holds true for decision making across all industries too.
Maximizing AI and ML for decision making
All this tells us that AI and automation are big powerhouses of decision making in the new age of decision intelligence. However, what it really does is empower humans to make better informed decisions and trust their judgement in an increasingly complex landscape. Automating manual, high-volume operations allows organizations to maximize cost savings and improve efficiency.
That said, transparent models mean that each decision, backed by AI insights, can be explained with full visibility for security and regulatory requirements. Generative AI co-pilots are particularly useful in this context in which organizations are concerned about AI lacking in transparency, otherwise known as ‘black box’ thinking. However, with generative AI, organizations can interrogate the machine’s recommendations, asking why and how each alert came about. And it can only do so effectively if it has constant access to the swimming lanes of data flowing through the pool.
That’s why organizations should look to Decision Intelligence platforms that sit across the whole organization’s data – with no concealed or incorrectly parceled pockets. Building a platform that supports deployment options for public or private cloud but that are attainable and explainable, will allow for more intelligent decision making across the organization.
About the Author
Jamie Hutton is CTO of Quantexa. When it comes to making the right decisions for your organization, more is possible with the right data, in the right context. Quantexa’s Decision Intelligence Platform gives customers the ability to understand their data by connecting siloed systems and visualizing complex relationships. The result is a single view of data that becomes their most trusted and reusable resource across the organization. Quantexa helps customers establish a culture of confident decision making at strategic, operational, and tactical levels to mitigate risk and seize opportunities on their path to building efficient and resilient organizations.
Featured image: Adobe
