REGIONAL— Details of the Minnesota COVID-19 model used to inform decision-making about a range of coronavirus mitigation strategies contain a thread common to all such models being bandied about in the media these days: uncertainty.
“I will remind you there is lots of uncertainty in inputs,” MDH health economist Stefan Gildemeister said Friday during a videoconference with members of the statewide press. “Every assumption, every data point, every research piece that underlies this has some uncertainty associated with it, and that affects the outcomes on the back end.”
Decision-makers, media, and citizens all want to know when COVID-19 infections will peak, when stay-at-home orders can end, and business and life can start returning to normal. Which prompts many of us to latch onto specific dates and specific numbers that appear in various models. But that’s not what epidemiological models are designed to do, according to state health officials, particularly when data and knowledge about the virus and its impacts change daily.
“They are not about specific point-in-time estimates,” Minnesota Department of Health Commissioner Jan Malcom said. “They are about directional changes.”
While various scenarios cover a specific period of time, the main things researchers look for are shifts in trends as variables are manipulated. When those changes indicate a decrease in overall deaths or extending the time frame for peak hospitalizations, those trends can be factored in with other information policymakers use to make decisions.
Malcom emphasized that the Minnesota-specific model developed by University of Minnesota researchers and MDH is only one tool among many Gov. Tim Walz uses to make decisions about mitigation strategies.
“He’s also being informed by national guidance from the CDC, in particular what we’re learning from other states, a lot of additional epidemiological data from MDH, as well as direct dialogue with the health care system,” Malcom said.
Models differ significantly
A model developed by the Institute for Health Metrics and Evaluation at the University of Washington has garnered increased attention in recent weeks for its more optimistic predictions on the course of the pandemic, including its predictions that much of the country is already beyond peak demand for medical resources as a result of the COVID-19 outbreak. That model predicts peak use of medical resources in Minnesota as early as April 28.
That contrasts sharply with Minnesota’s model, which places the state’s peak need for all medical resources as occurring in late June to mid-July.
Gildemeister said it’s hard to compare the models because IHME uses different assumptions, data, and time frames than Minnesota’s own health experts.
“We could spend an hour on why that model is different from Minnesota’s model,” he said. “They have made some incredibly optimistic assumptions, optimistic in regards to what mitigation is actually in place, optimistic with regards to the extent to which death data is accurate. I think the biggest difference between (models) is that they’re predicting out only four months.”
The Minnesota model, by contrast, spans 30 weeks, not 16, and the scenarios have been projected well beyond that.
The original IHME model assumed a scenario in which extreme social distancing measures would remain in place across the country through August, a scenario not mirrored by daily news accounts of varied and changing strategies across states. IHME officials have said they are now assuming social distancing continues through the end of June, which still be optimistic given that President Donald Trump and others are pushing for lifting of restrictions in many states by May 1.
The most recent IHME model, according to their website, relies on death rate data from 17 areas in Asia and Europe. While Minnesota’s model originally had similar data to work with, researchers have now been able to adjust those rates using Minnesota-specific data.
Armchair statisticians’ calculations based on total numbers of deaths or tests administered fall far short of the sophisticated details and scope that computer-driven models employ, and tiny changes in a specific input variable can yield dramatic differences over time.
For example, in a world of data that is evolving and changing rapidly, what number should the model use for the number of people who get infected by each person with COVID-19? Choosing two rather than four doesn’t make too much of a difference in a few days, but over six months, a year, and 18 months the differences are in the thousands. Current research suggests a target used by the Minnesota model, 3.87, but researchers work with a level of uncertainty from 2.5 to 4.7 persons infected by any one individual.
Now add in such things as the amount of time the virus is latent in a person and the number of days a person can infect others, the number of days patients are in the hospital, the length of time a patient spends in an ICU, the increased death rate when ICU beds are at capacity and can’t take patients who need them, and the increased death rates for people with one or more underlying health conditions, all of which carry some level of uncertainty and are subject to change, and the complexity of modeling potential COVID-19 outcomes becomes glaringly apparent.
Gildemeister said incorporating additional data, including that specific to Minnesota, has led to differences from when the model was first developed. Some of those changes are that the virus spreads faster than initially believed, but hospital stays are fewer and shorter.
Most significantly, the number of predicted deaths in the state has dropped from an initial estimate of 50,000 to 20,000 to 22,000.
When one journalist suggested 22,000 seemed high compared to the lower-than-projected number of deaths in New York City, Commissioner Malcom responded.
“We’re really not going to try to respond to point-in-time estimates,” she said. “This is the peak of wave one for New York. I’ve heard other epidemiologists very concerned that New York will see waves much larger than what they’ve already seen in the coming months. We have to help people understand this is not a one-time event, it isn’t that you hit the peak and everything goes back to normal. This is going to be with us in a really challenging way, in multiple ways, until there are treatments and a vaccine. That’s just the hard reality we have to factor into this planning and these policy decisions.”
Gildemeister noted that the situation in New York was significantly different from conditions in Minnesota, and re-emphasized that models are part of what needs to be considered when trying to address mitigation.
“Our job is to provide data-driven evidence for decision makers,” he said. “There are trade-offs with every decision. The burden of decision lies with people who have been elected or who have been appointed.”
Malcom agreed, and noted that data from Minnesota hospitals modeling their own capacity paints a more bleak picture.
“This underscores why we would never use this model as the only driver of the decision,” she said. “This is one of many data points. The other thing we’re looking at very closely is what hospitals are telling us about their capacity, their confidence level in how quickly they can add capacity. Their own models, as the governor has said, are much more pessimistic than this one. The hospitals’ models are suggesting that the time is really critical because they think peak ICU demand will be greater than these numbers.”