Modelling has been around for decades. In epidemiology, a model is a projection of possibilities that could arise given a certain set of circumstances; a set of equations about how a disease could spread.
Vital statistics: models are playing an important role in the response to the COVID-19 pandemic.
It is important to understand the aim of the model. For example, a model might be created to assess the trajectory of a disease. It is then used to address specific questions: what might happen if you pull a certain lever – like social distancing, or wearing masks. How might that impact mortality rates?
Models are about projecting. They are not about instantaneous control.
Who creates models?
Models are generally created by an expert team. Some people specialise as modellers and their training is often in mathematics and/or statistics. They will also often have experience in epidemiology.
Others may join the team because of their specific expertise in the subject of the model. Many of these people will also have existing affiliations and may draw on these in the formation of the team. In the case of modelling relating to COVID-19, the team will also include experts in infectious diseases.
How are COVID-19 models created?
By their nature, models must make assumptions. Each of these assumptions – or characteristics – should be supported by published data. This data must specifically relate to the model you are working on. In terms of say, transmission probability, you would look at the papers that inform that one characteristic. But this could be just one component of that whole model.
Early COVID-19 models that predicted tens of thousands of deaths in Australia were based on assumptions of what the outbreak would look like if there were no interventions.
In the case of COVID-19, there was a lot of uncertainty in the early days. We didn’t know very much about how the disease was spread, who was susceptible, how the movement of people would affect the spread, what the infection fatality rates would be, what rates of testing we would have, and so on. There were so many unknowns. It takes time for data to be appropriately collected. As time passed, we were able to access more data and modify parameters so that models could be improved.
Modelling can be complex. It is difficult to work out how individual parameters are affecting the outcome and how multiple parameters interact with each other.
Are models peer-reviewed?
Anyone can write a model. If you go on Facebook or Twitter you will see models being created by a wide variety of players. These are not generally peer-reviewed. The models published in medical journals, and those used by decision makers (in the Department of Health for example), usually go through thorough review processes.
Complex business: Professor Amin says COVID-19 modelling that showed tens of thousands of deaths in Australia was based on a scenario where there were no interventions to stop the spread.
There is a push for the availability of open data to enable review of model assumptions and methods and to ensure that reported outcomes can be reproduced. In the context of COVID-19, models used by decision-makers and scientists will be reviewed and updated as more robust information becomes available.
What about the criticisms of inaccurate modelling?
It is important to understand the paradigms upon which a model was created. Models that are considered in hindsight by some people to be wildly inaccurate may not be so. For example, there are early COVID-19 models that predicted tens of thousands of deaths in Australia. These were based on assumptions of what the outbreak would look like if there were no interventions. That meant no lockdowns, no shutdown of travel, no border closures. In other words, what would it look like if we did nothing?
Of course, the reality was that we were never going to do nothing. These models were designed to show a worst-case scenario and then to suggest what would happen if we changed certain parameters, such as closing the borders or implementing lockdowns.
Also, every model has bounds of uncertainty. Models are built on assumptions and all of those assumptions have some level of uncertainty. So a model is never going to be so accurate as to be able to say this exact event will happen on this exact date. Models generally estimate a chance – a probability – of something happening. It’s not a certainty, it’s an informed estimate.
Professor Janaki Amin is Head of Department, Health Systems and Populations, Faculty of Medicine Health and Human Sciences at Macquarie University.
