At this point, we are all very familiar with the idea that “every company is becoming a software company.”
But the meaning behind this is rooted in the fact that innovation in the digital age – application development – is a driving force in creating new business and gaining a competitive advantage. The speed of deployment and the quantity of innovative features in a new application have a direct correlation to business success.
If applications are the currency of the new economy, then development teams are the market makers. However, these teams continue to be misunderstood, mismanaged, and marginalised inside both large and small companies. And companies pay a tax when they fail to understand the nature of the work developers do, or provide a safe and productive environment for them to do it.
Defining the DIRT
The innovation tax, like income tax, is real. Taxed organisations see their pace of innovation suffer as people and resources are locked into maintaining rather than innovating.
Another way to define this tax is DIRT. Why? Well, it’s rooted in data (D), because it so often springs from the difficulty of using legacy databases to support modern applications that require access to real-time data to create rich user experiences. It affects innovation (I), because your teams have little time to innovate if they’re constantly trying to figure out how to support a complex and rickety architecture. It’s recurring (R), because it’s not as if you pay the tax (T) once and get it over with. Quite the opposite. DIRT makes each new project ever more difficult because it introduces so many components, frameworks, and protocols that need to be managed by different teams of people.
It’s clear that technology leaders recognise this tax and immediately grasp the degree to which it’s caused — or cured — by their data architecture. Data is sticky, strategic, heavy, intricate — and the core of the modern digital company, and modern applications have much more sophisticated data requirements. Obviously, there is more data, but it’s more complicated than that: Companies are expected to react more quickly and more cleverly to all of the signals in that data. Legacy technologies, including single-model rigid, inefficient, and hard-to-program relational databases, just don’t cut it.
If this were rare, it wouldn’t be such a big deal. But large enterprises can have hundreds or thousands of applications, each with their own sources of data and their own pipelines.
Over time, as data stores and pipelines multiply, an organisation’s data architecture starts to look like a plate of spaghetti. The variety of technologies, each with their own frameworks, protocols, and sometimes languages, makes it extremely difficult to scale, because every architecture is bespoke and brittle. Teams spend their precious “flow” hours doing integration work instead of building new applications and features that the business needs and customers will love.
In many cases, the innovation tax manifests as the inability to even consider new technology because the underlying architecture is too complex and difficult to maintain, much less understand and transform. It’s clear that organisations are ready for a new approach.
Start in the DIRT
Start by getting a better understanding of just how DIRT might be holding your team’s back. Do your developers have trouble collaborating because the development environment is so fragmented? Do schema changes take longer to roll out than the application changes they’re designed to support? Do you have trouble building 360-degree views of your customers? And if so, why?
By starting digging in the DIRT you will be able to get a better understanding of all your applications and their data sources. This will also give you an understanding of what it would take to move your data onto an application data platform. That is not to say this will be easy and it may seem like starting again from scratch, but technology leaders have worked on problems like this before, so they know what progress looks like. And what success means for the organisation. Start by figuring out your Innovation Tax bill and you will be on the right path.
About the Author
Mark Porter is the Chief Technical Officer (CTO) of MongoDB, where he is responsible for crafting the long-term technology roadmap and vision for the company. Prior to MongoDB, Mark worked at Grab, Southeast Asia’s super app that provides everyday services such as ride-hailing, food, package, grocery delivery, mobile payments and financial services to millions of people. Mark has also worked at AWS, NewsCorp and Oracle. He has been professionally coding since he was 16 years old and founded and ran his own electronics services integration company.
Featured image: ©maxxyustas