EverQuote was founded in 2011 as an online insurance marketplace that would connect consumers directly with insurers
Our use of data and analytics helps make insurance simpler, more affordable, and personalized, saving both consumers and providers time and money.
A Data-Based History
From early on in the company’s history, we understood the value of running a data-driven organization. We leveraged data and analytics from our start as a spin-off from Cogo Labs to build up a large insurance industry user base. Cogo’s reliance on data analysis to make better, more accurate business decisions was a huge driver for our approach to the industry.
Originally, we operated by utilizing a MySQL cluster, various Python services, direct connections to Excel, and a custom online analytical processing (OLAP) interface that used point-and-click tools for ease-of-use.
Because the use of data and analytics was burned into our company’s DNA, it was just understood as the way we did things. Our reliance on data and analytics gave us a competitive advantage and paved the way for our transition to self-service data analytics.
Improving the Process
As we looked to democratize the use of data and analytics across a growing organization, the one aspect we needed to ensure was that the tools and solutions we used were able to be utilized by everyone from employees to customer support, to internal operations. We knew our existing data architecture and custom tools were not scalable and would not be easy to teach to new users.
We made the decision to shift our extract, transform and load (ETL) data process to open-source tools like DBT and Apache Airflow. We had also been using Databricks for many years but were finding that data preparation was still taking hours.
Existing users were also reporting that the current workflow was slowing them down and had not evolved well as the organization had grown. Our custom tools were limiting our ability to increase our use of machine learning.
We soon realized that we needed to streamline our entire data architecture in order to make data immediately accessible. As an organization whose success was built on data and analytics, improving and expanding our data tools the right way was critically important.
Finding the Right Tools
EverQuote formed a data analytics team that would help our entire organization strengthen its data use. The starting point was to promote data literacy across the team, educating them on how to strategically use data, while also supporting awareness and repetition of best practices as they used it.
As a part of our initiative to streamline our data access and analysis tools, the company turned to Snowflake. We decided to make Snowflake become the source of all of our data. For this to be successful, we also needed to finally leave our custom OLAP interface behind and adopt a solution that would work well with Snowflake, while making our data accessible and user-friendly.
This led EverQuote to AtScale, whose unified semantic layer helped us consolidate our data and produce shared insights across the organization, regardless of where that data was stored. By adopting AtScale’s semantic layer, we were soon able to realize the self-service approach to data analytics that we wanted. Our company now counts more than 85 databases and 1500 tables on Snowflake that are also accessible by business users in Tableau and Excel and are connected to machine learning workflows.
The combination of AtScale and Snowflake truly made self-service analytics reality for us. In the past it took a month or more for our organization to make new data accessible – and even longer for us to add new users to the mix. With our revised structure we expect to bring this down to just a couple days.
Lessons Learned
The importance of having one core, consistent location for your company’s data cannot be understated. This enables the democratization of data and analytics use to business users to be a smoother process, while also making it easier to onboard new users. An important part of this process is ensuring that the technology you choose also complements each other well. Scalability is so important to a growing company that you don’t want to be in a situation where you must reinvent the wheel every so many years.
Remember – self-service is a continuous process, especially as an organization grows. Continuous training and knowledge sharing is a critical part of getting self-service and democratized data correct. We were lucky as the company’s core was built around data and analytics from the start – but not every organization will have that luxury. Establishing a data team, improving overall data literacy and teaching best practices will go a long way to ensuring a successful initiative.
About the Author
Kwan Lee is EVP, Data & Analytics, at EverQuote leading efforts on their direct-to-consumer agency engineering. He is passionate about scaling systems and people. For the last decade he has been helping product engineering teams and sales organizations transform to agile and data-driven organizations. He has diverse industry experience in consumer electronics, telecom, venture capital, edtech, insuretech, fintech and scaling engineering teams. His research interests include how to drive better human decisions and social behaviors through data. He has Ph.D. from MIT and B.S. from Cornell University.
Featured image: ©Quardia
