Decoding the data: how we are tracking Covid-19

Confirmed cases, daily growth rate, fatality curves – every day the headlines bring an overwhelming number of new numbers on the Covid-19 pandemic. But which matter most when understanding Ireland‘s place in the global crisis?

Here, the Business Post breaks down the meaning behind the key metrics and what they can and can’t tell us.

Confirmed cases: Underestimates, time lags and location indications

- Estimated time lag since transmission: 1-21 days

- Pro: Broadest and most immediate metric

- Con: Dependant on testing levels

“Confirmed cases” accounts for those who have been tested and found to be positive. This is often the first figure to be cited in news reports, but also potentially the most unreliable.

Confirmed cases are entirely reliant on the level to which a country is testing its population. The more tests carried out, the more cases that are likely to be detected.

Some countries are testing only those who present with severe symptoms, capturing only the tip of the iceberg. Others may seem to have high levels of cases but they are including mild and moderate cases. This is a positive when trying to manage the outbreak; the more people we know have it, the more we can make efforts to trace and limit its spread.

Because testing levels will vary widely from country to country, it can make international comparisons difficult.

When taken purely as a domestic measurement, confirmed case numbers still come with many caveats.

Depending on testing levels, confirmed cases are likely to be a smaller percentage of actual cases given that the 80 per cent of people with no or mild symptoms are less likely to be tested.

When testing criteria changes, it alters the way we interpret cases over time. Irish testing criteria became stricter on March 25, meaning results from tests that were returned on March 28 or 29 onwards were from a different sample size.

This could change again as testing capacity grows, meaning our linear understanding of how case numbers are growing needs to be adjusted yet again.

Even if there were consistently high levels of testing, there is still the issue of the time lag when interpreting the results. A test result can take about two to three days before the swab results return from the lab. If it had already taken three to four days for a patient who had symptoms and passed the criteria for a test to get swabbed, and it had taken between one and 14 days since the virus was transmitted to them before they showed symptoms, that means a potential time lag of up to three weeks.

That means that the “confirmed cases” we see today could be for people who caught the virus three weeks ago. If we are looking to see the impact of social distancing or lockdown on case numbers, we would need to add that potential time lag into our considerations.

However, compared with the time lag of hospitalisations, intensive care admissions and deaths, confirmed cases are still the most immediate and broadest available metric for understanding the scale of the spread of the virus before relying on estimates or predictive models. It can also be used as an indicator of “where” the virus is present, with the Department of Health releasing county level data that is unavailable for any other metrics so far, such as hospitalisations or deaths.

ICU admissions: admission criteria, capacity limits and time delays

Estimated time lag since transmission: 12-17 days

Pro: Proven reflection of scale of crisis

Con: Growth rate stagnates when ICU reaches capacity

While data that relies on confirmed cases depends on a limited sample size because of testing and can differ greatly country to country, ultimately the scale of the spread will be directly proven with the number of patients admitted to intensive care units (ICU).

Finding a reliable average based on other countries’ experience can be tricky for a number of reasons, however.

ICU admission rates already vary from country to country; in the US the existing bar for admission is a little lower than in Europe.

In countries in crisis, admissions are complicated by capacity. This is because, at the peak of a crisis, those who need intensive care and those who are admitted could become two different numbers.

Admissions in countries with high existing capacity will continue to rise in proportion to the spread. In those that reach capacity, the peak of the crisis won‘t just result in more ICU admissions – it may prevent them.

If the crisis gets so bad that the criteria for admission grows more stringent, as we have seen in some parts of Italy, then we would see ICU admissions stagnate or even fall as patients struggle to be admitted. The wrong reading of this data could give false hope when in fact hospitals are still in the throes of the crisis.

Conversely. some countries will more effectively expand ICU capacity as the crisis peaks.

Other factors such as demography apply. Different demographics mean that in some countries, say where there is an older population or a higher prevalence of high-risk underlying conditions, more people will get so ill that they end up in ICU.

In terms of using ICU admissions to assess the overall scale of the virus spread, the time lag also needs to be taken into account.

One estimate is that the commonest time frame from virus incubation to symptom presentation – five to seven days – to acute deterioration requiring ICU care – seven to ten days – could result in patients not being admitted to ICU until 12 to 17 days after transmission.

This means the number admitted to ICU will peak about two weeks after the peak in cases.

Fatalities: country curves, case fatality rates and country comparisons

Estimated time lag since transmission: 13-31 days

Pro: Most consistent reliable metric

Con: Time lag of up to a month

The number of deaths is the main constant when assessing the impact of Covid-19.

A time lag since transmission needs to be taken into account. Patients may not die until weeks after transmission.

We also need to exercise caution when comparing fatality rates from different countries. Again, demographics will have an impact on the mortality rate. Healthcare capacity will also have an influence.

We must also consider the possibility that countries count deaths differently. Some may miss early deaths if unprepared, others may fail to consistently count Covid-related deaths in the peak of the crisis. In Italy, deaths in nursing homes were not counted toward the overall death toll, for example. Others may give incorrect official numbers, as in Britain.

Case fatality rate

Case fatality rate (CFR) – a crude calculation of percentage of deaths out of the number of confirmed cases – is somewhat useful but is yet again affected by variable testing levels.

The Financial Times data team recognised this as the “‘iceberg” problem. Where Germany tests widely, including moderate cases, the case fatality rate was 1.3 per cent as of April 2. Deaths are calculated as a proportion of a much larger sample of confirmed cases. In Britain, where severe cases are tested, the sample is smaller and biased toward the most sick – the tip of the “iceberg” – making the case fatality rate appear to be 8.7 per cent.

Country comparisons

Data analysts have rushed to compare countries’ fatality curves, both measuring total deaths and daily totals.

Based on models developed by the data journalist John Burn-Murdoch, these curves aim to chart a version of the “curve” that experts have warned we need to flatten. While a flat line indicating a plateau is the goal in the “totals” chart, the most recent version charting “daily fatalities” is useful to show daily growth as well as a recognisable flattening of the curve.

The cumulative number of cases is used instead of controlling for population – that is, the number per million – because cases are understood to spread through a population at the same rate in the early phase of the outbreak, regardless of the country’s size.

A useful analogy is a forest fire, when the blaze breaks out in one area it will spread at the same rate regardless of how big the whole forest is. If the virus becomes more widespread in a population – like fires spreading in multiple areas – a “per million” comparison could become more useful. Population densities in major cities or how susceptible a population is based on demographics – like how dense or dry the forest is – is another metric that could be useful.

Nations can compare their curves to the countries that resemble their demographics, response strategies and estimated phase in the pandemic, and so on. Through that comparison, they could estimate their course to a similar country that is further along in the pandemic than them.

The below graph, for example, indicated that Ireland is unlikely to end up with a death toll as high as Spain or Italy but that it is still possible for it to replicate China or Canada‘s path.

Growth rates: speed of spread, exponential growth and predictive models

Predictive modelling may be necessary for government‘s worst-case scenario planning, but otherwise it should be treated with caution. One reliable metric is average growth rates.

By using the percentage increase in new daily case numbers and averaging it over the past week to control for peaks and lows, we can calculate the average growth rate and use it to estimate future trends.

This is exponential growth: the worst case scenario for the virus’s spread through a population where it grows consistently at the same rate. The metric can also be applied to ICU admissions and fatalities.

Distant predictions are likely to be inaccurate because it is subject to too many variables in the interim, such as the effect of social distancing measures. Shorter-term estimates, however, are still useful.

Of course, as the pandemic goes on, growth rates may slow for another reason. The growth rate of ICU admissions could slow because of capacity. Confirmed cases may slow down if the scale increases to a level where testing can’t keep up.

Updating our understanding

Our understanding of Covid-19’s spread changes daily as new research, strategies and resulting impacts are observed throughout the world. So too will our measurements of progress.

Keep track of the new numbers on the Business Post’s Covid-19 Tracker, updated with the latest numbers daily.

Updates to graphics and key metrics for measurement will also be updated in this article and on the dashboard. If you have feedback or specific requests, -mail rlavin@businesspost.ie.