Colorado’s governor says he is using data to guide the state’s response to the coronavirus. But the numbers are incomplete.

As Gov. Jared Polis stood before the television cameras this week to announce the statewide stay-at-home order in response to the coronavirus pandemic, he fielded a question about when mandated isolation would end.

“We will make that call based on the real data that we receive,” he said.

For weeks, Polis has tackled this pandemic like the tech entrepreneur he used to be, consuming large amounts of data to inform his decisions. This week, he mentioned that the state is looking at traffic data and cell phone data to determine the effectiveness of stay-at-home orders. And the Colorado Department of Public Health and Environment maintains a data website providing daily tallies of coronavirus cases across the state.

Despite this numbers-heavy approach, Colorado’s response has been hindered by an incomplete understanding of some figures. What it means is that the state — like many others across the country — is struggling to get an accurate projection for the spread of COVID-19, the disease caused by the new coronavirus.

There are inconsistencies in the numbers CDPHE provides to the public every day on the pandemic. For instance, the state has included some people who are suspected of being infected but have not actually tested positive for COVID-19 in its tally of confirmed cases.

There is a lag of sometimes days between when local health departments report new cases or deaths and when they appear in CDPHE’s numbers. Cases that were reported by local health officials in Rio Grande County on Monday didn’t appear in CDPHE’s total until Thursday; a case in Costilla County reported Thursday morning did not appear in CDPHE’s update that afternoon.

And the state has not been posting information about the total number of tests being run by private labs, meaning researchers looking at Colorado’s data don’t know the ratio of positive to negative tests.

Many of these problems can be traced to the nation’s lack of a plan for widespread COVID-19 testing, state officials say.

“That lack of information has certainly made it more challenging to contain outbreaks throughout our state,” said Scott Bookman, CDPHE’s incident commander for the coronavirus response and the head of the department’s testing lab.

Bookman said Colorado has tried to make up for that by looking at statistical models developed in other states and countries “to try to understand what the future may hold for Colorado.”

But Colorado still needs to know some basic numbers to develop reliable models here. And, partly because of testing and partly because of how new COVID-19 is, it’s not clear that it does.

Building a COVID-19 model

Kathryn Colborn — Ph.D., masters of science in public health — is an associate professor in the Department of Surgery on the University of Colorado’s Anschutz Medical Campus. But her background is statistics and, specifically, statistics on infectious diseases.

She’s worked on a number of research projects for mathematical modeling of malaria. She can recite figures for the infectiousness of measles and other diseases from memory. And now, she and a team of colleagues are working on developing a model for the spread of COVID-19 in Colorado. They hope the model will be able to predict when the pandemic will peak here and how many critical-care hospital beds and ventilators will be needed to handle that swell.

The team uses the data released every day by CDPHE, the same numbers available to the public. But most crucial for their model are two numbers not available in those figures.

The first is a number that is written in equations as R₀, which Colborn pronounces “R naught,” like “are not.” It’s a calculation of how infectious any given virus is — essentially, how many people a single infected person is expected to pass the infection on to. The second number is the total number of people who have COVID-19 in Colorado right now. 

Both of those numbers currently require some guesswork.

Colborn and her colleagues are assuming the new coronavirus has an R₀ of around 3 to 4, based on research published elsewhere. As for the total number of reported cases in Colorado, Colborn said the team isn’t using the number published by CDPHE.

“It’s hard to track the reported number of infections and believe those numbers right now,” she said. “We’re not testing everybody.”

Instead, the team is using the number of hospitalizations. Relying on research published by a team in England on the percentage of COVID-19 cases that require hospitalization, Colborn and her teammates looked at the reported number of hospitalizations in Colorado and calculated an estimate of the total number of cases here: About 3,000 to 4,000.

That’s far above the state’s official tally of 1,430 as of Thursday night. But it is in line with what Gov. Jared Polis and others have been saying, that Colorado likely has many times more COVID-19 cases than have been reported.

From that overall number, Colborn and her teammates can use studies in other countries to arrive at estimates of the disease’s spread in Colorado, the likely peak for infections and the number of critical-care beds and ventilators that will be needed at hospitals. (Colborn said the team is still refining the model and is not ready to release its projections.)

But even small changes in the number of hospitalizations and R₀ can ripple throughout the calculations, causing potentially meaningful swings in the final numbers. And that, in turn, could have a significant impact on the state’s planning and readiness.

For instance, CDPHE reported Thursday night that there are 184 people hospitalized with COVID-19. But hospitals have been warning that the true numbers are likely higher — and hidden by the lack of testing. More people hospitalized would mean a bigger number of people infected in Colborn’s model and could mean a faster spread of the virus.

There’s also a question of whether the CDPHE data is weighted unevenly. Colorado currently has outbreaks in nine nursing homes or long-term care facilities. But state officials won’t say where those are or how many people have been infected in those outbreaks. If they are contributing disproportionately to the state’s hospitalization and death totals, it throws off Colborn’s model.

And then there are unknowns around R₀. What if that early research that helped set the number is wrong? But, more importantly, R₀ effectively drops with social distancing. That’s why Colborn said the stay-at-home order is a wise decision, even if the state’s statistics are questionable.

“This is really positive, and we’re trying to factor those things into our model to say, ‘Where are we at right now, and where are we going to be in three or four weeks into this path?’” she said.

Defending the data

Despite acknowledging their limitations, state health leaders have generally defended their figures.

Earlier this week, an asterisk popped up on CDPHE’s COVID-19 data website next to the total number of confirmed cases. The fine print reads: “Positive cases include people who tested positive, as well as cases where epidemiological investigation has determined that there is a high likelihood that an untested individual has COVID-19 due to their symptoms and close contact with someone who tested positive for COVID-19.”

In an emailed statement, CDPHE said those untested confirmed cases are “epidemiologically linked” and make up only about 5% of the total number of cases, “a small number that doesn’t change our projections.”

“This is a standard case definition, and we’ve been reporting cases that way for a number of weeks,” CDPHE said in the statement.

The delay in including deaths that are reported by local public health agencies is the result of investigations to make sure those deaths are really from COVID-19, Bookman said.

But there is no dispute that, in order to stop COVID-19 and allow life to return to normal, Colorado will need a lot better data.

“We have been so disappointed by the lack of testing supplies.” Polis said this week. “This is so frustrating.”