Benchmarking Quantum: My Qubits are Better Than Yours

When it comes to computers, technologies for bits are not created equal.

This is apparent in a quick stroll through your local electronics store. Photographers might pick up an SD card, while gamers might pick up disks for their home console. Meanwhile, in the next aisle over, a back-to-school shopper will be purchasing the latest Macbook with “solid-state drive.” Ultimately every photo, video game, or final report boils down to a list of 0s and 1s. But how those bits are stored—whether in a magnetic groove, an electronic circuit, or another format entirely—depends entirely on the particulars of the application.

Finding the right qubit for the right job

Today, we are still in the early stages of quantum computing so it’s hard to believe we may someday need to make these kinds of choices: which type of qubit (quantum bit) is right for which job? But in the next decade – as quantum computers come to market – we expect to see a similar breakout occurring. Key to all of this will be the ability to objectively benchmark performance against claims being made.

Just as there are many different ways to store a traditional bit, there are many types of qubits. today’s frontrunner technologies include qubits built from atoms, ions, photons, superconductors, and ions. Each of these technologies scores differently in terms of key metrics like speed, scalability, reliability, interoperability, ruggedization, and cost. But critically, no qubit type wins in all categories. In fact, many of these metrics oppose each other—for example, the fastest qubits also tend to be the least scalable.

Borrowing from a rich tradition of benchmarking

In the absence of a single dominant qubit type, we should instead benchmark the capability of each qubit type to support different use cases like machine learning, quantum dynamics, optimization, sensing, or material science. Such benchmarks, which are finely tailored to concrete applications, will drive the industry’s preferences and demand for specific quantum hardware models. Just as in traditional computing, we expect that a diversity of qubit types will emerge that will each drive different software applications.

Designing an effective and useful suite of benchmarks is a challenging task, but luckily we can borrow from several principles from benchmarking of classical computers over the last three decades.

As enterprise leaders begin mapping out their quantum strategies, performance benchmarking must be a key component to any long-term plan. Hardware vendors will be battling for market share, and there will be no shortage of options. Objectively understanding the performance each one provides will give enterprise leaders clarity to make the best choices possible. Moreover, high-quality benchmarks also empower leaders to cut through the hype and let data drive purchasing decisions.

A yardstick towards the holy grail: quantum error correction

While early applications of quantum computing are expected to unlock speedups for niche use cases, the broadest market applicability is gated on a key enabling technology: quantum error correction (QEC). This technology, which enables qubits to behave perfectly, is the principal aim for many of the leading quantum hardware vendors. The first qubit types to demonstrate QEC will be well poised to capture latent industry demand for applications that require far more reliable quantum computers than what we have today.

In this spirit, quantum benchmarks ought to include quantum error correction itself as a yardstick for progress. In our own work, we found that QEC “stresses” very different parts of a quantum computer than other applications do. For example, a critical component of QEC is the ability for software to adapt itself on the fly (or in technical terms, “feedforward measurement”). This component is unique to QEC and currently an area where hardware vendors need to make continued progress.

In this sense, setting appropriate benchmarks—informed by real-world industry demand—can accelerate progress toward tomorrow’s quantum computers. Just as benchmarks have both chronicled and influenced the design of traditional computers, benchmarks will play an important role in quantum computers by matching qubits to applications. Let’s work to make it as easy as a walk through the electronics store.


About the Author

Pranav Gokhale is VP of Quantum Software at ColdQuanta. ColdQuanta is a global quantum technology company solving the world’s most challenging problems. ColdQuanta harnesses quantum mechanics to build and integrate a range of quantum computers, sensors, and networks. From fundamental physics to leading edge commercial products, ColdQuanta enables “quantum everywhere” through our ecosystem of devices and platforms.

Featured image: ©Vchalup

more insights