A cost benefit analysis of 5 extra
days at COVID-19 alert level 4
COVID-19 is both a health and an economic crisis for New Zealand. How it handles the health crisis affects the
economy, and how it handles the economic crisis will affect New Zealander’s health in the future.
Looking at health or economic costs in isolation risks presenting a skewed picture of the overall situation. This note
brings together estimates of health and economic costs into one model.
In preparing this note, my intent is to show a technique with a concrete example, rather than to critique a specific
decision made by the Government.
Such models rely on the assumptions that drive them. I have estimated many parameters in this model, based on
available data and literature.
The model’s results might be sensitive to changes in some parameters, and insensitive to others. (By sensitive I mean
that a plausible alternative value for a parameter might tip the model’s results towards supporting different policy
conclusions.) I supply a sensitivity analysis to find which, if any, parameters the model’s conclusions are sensitive to.

Decision point on Monday 20 April – to extend or not to
extend?
1.

On Monday 20 April Cabinet met to decide whether to change New Zealand’s COVID-19 alert level. Later
that day the Prime Minister announced a 5-day extension to level 4, to be followed by two weeks at level 3
before a further decision about moving levels would be made.

2.

Cabinet’s reasoning behind the extension was expressed in cost-benefit terms. The Economist magazine
reported that:
… in pursuit of elimination, [the Government] recently announced a five-day extension of New Zealand’s strict
lockdown. The extra short-term cost … will give “much greater long-term health and economic returns”.1

3.

The author has not seen the analysis that informed the Government’s statement. However, to make that
conclusion, one needs to calculate both the “extra short-term cost” and the “long-term health and economic
returns”, and to compare the two. This, in turn, requires a marginal cost-benefit analysis that incorporates
both health and economic costs. This note presents such an analysis.2

4.

This note accepts the world as it was on Monday 20 April. It limits the analysis to information sources that
were (or should have been) available to decision makers on that day. It is thus an ex ante analysis, even
though performed late.

5.

New Zealand had been at alert level 4 for 25 days. New case numbers appeared highly responsive to the
introduction of social distancing measures, with a lag of approximately two weeks. Daily new cases had
dropped from 89 on 4 April to an average of 10 for the preceding four days.

https://www.economist.com/asia/2020/04/23/what-next-for-countries-that-are-nearly-covid-free. See also
https://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&objectid=12326157.
1

2 This note was developed independently from the analysis by Layard et al. that addresses a similar question for the United Kingdom
(http://cep.lse.ac.uk/pubs/download/occasional/op049.pdf). That analysis quantifies some costs and benefits not included in this analysis. It could usefully inform
further refinement of this note.

New Zealand Productivity Commission
(With minor corrections, 2 May 2020)

2 May 2020

1

 6.

The Government faced a choice between two alternatives:3
•

Option SL3 (straight to level 3): Move to level 3 on Thursday 23 April.

•

Option X5D (extra 5 days): Stay at level 4 for five additional days before going to level 3 on Tuesday 28
April.

7.

I can estimate the marginal costs (C) and benefits (B) of X5D over SL3. This type of analysis can (and should)
inform such decisions.4

8.

I will calculate net benefits from adopting X5D over SL3. This is (BX5–C X5D) – (BS3–CS3). Or equivalently,
(BX5– BS3) – (C X5D–C SL3).

9.

A positive result means that the net benefit of X5D exceeds that of SL3. A negative result means that the net
benefit of SL3 exceeds that of X5D. Apart from the sign on the estimated result, magnitude matters too, as
does sensitivity testing to provide a range of estimates of the net benefit.

10. I can omit any benefit or cost that would occur in both scenarios. This makes the analysis much more
straightforward.
11. The effective reproduction number (Reff) of SARS-CoV-2 was known to be below 1 on 20 April (the Prime
Minister reported it as 0.48 on that day).5 So, virus spread was under control. It was also known that if Reff
moved above 1 at some time in the future, level 4 measures (or those used successfully in other countries)
could be reintroduced to bring Reff back below 1. The risk of an uncontrolled spread in New Zealand was thus
effectively zero, so uncontrolled spread is an inappropriate reference point for this analysis.
12. The 20 April decision in no way constrains choices beyond the announced period (ie to 11 May).
13. The benefits (and costs) from alternative SL3, relative to alternative X5D, must arise directly from the
differences between the alternatives.

3 Clearly, there were myriad other possibilities, including a longer extension of level 4 and an immediate drop to level 2. To be tractable, marginal analyses need to
artificially restrict the option set.
4 This type of marginal cost benefit analysis is also appropriate for determining whether or not a specific measure should be included in the bundle of measures that
make up an alert level.

The source of the 0.48 number is not clear. My calculations suggested an Reff of 0.14 on 20 April. That analysis, however, was neither publicly available nor widely
communicated within government.
5

New Zealand Productivity Commission
(With minor corrections, 2 May 2020)

2 May 2020

2

 The benefits of a 5-day extension over no extension
14. Due to delays between infections and new cases, any health benefits will arise after the extension has expired.
15. Health benefits of an intervention are typically measured in quality-adjusted life years (QALYs). “Qualityadjusted” refers to the quality of life during the period lived, relative to that of an “average” healthy person of
the same sex and age.6
16. SARS-CoV-2 infects people of all ages, but morbidity (illness) and mortality (death) are highly concentrated
in those with pre-existing health conditions and the elderly. The average age of people dying from COVID19 in Italy is 79.5.7
17. A 79-year-old New Zealander might on average expect to live another 10.7 years.8 Life expectancy for those
with pre-existing health conditions will be lower however, as will life quality. If such people lived 5.4 years
on average with 70% life quality, then a premature death from COVID-19 would equate to a loss of 3.7
QALYs. This is significantly higher than an estimate based on the actual pattern of deaths to date in New
Zealand.
a.

The majority COVID-19 deaths in New Zealand have occurred in rest homes, and more specifically
in people with dementia.9 Average life expectancy for a current rest home resident with severe
dementia is roughly 1.2 years.10 The disease burden estimates in the Global Burden of Disease 2013
study estimates that life quality for someone with severe dementia is no more than 55%.11 This
suggests a QALY loss of 0.66 from the death of a rest-home resident with severe dementia.

b.

The average life expectancy for rest home residents without severe dementia is 42.5 weeks.12 Life
quality will vary depending on the specific health conditions that residents live with. Assuming an
average life quality reduction of 30%, this suggests a QALY loss from the death of a rest-home
resident death of 0.60.

18. Blakely, Baker and Wilson used 5 QALYs for a COVID-19 death in New Zealand.13 Wilkinson uses an
implied value of 6.9 QALYs.14 In this analysis I will use a value of 5. (I also conduct sensitivity analysis using
3 and 7.5.)
19. I used a similar process to estimate QALYS for non-hospitalised COVID-19 cases and recovered cases
following hospitalisation. For the former, I will use the average of the disability weights for mild and
moderate cases of an “infectious disease, acute episode”, ie, 0.029 QALYs. For the latter, I will use the
disability weight for a severe case, ie, 0.133 QALYs.

6

Quality-adjustment can also refer to the differences in quality of life for a specific person under differing treatments for the same condition.

https://www.cebm.net/covid-19/global-covid-19-case-fatality-rates/. I estimate the average age of the 14 people who had died from COVID-19 in New Zealand (as at
20 April) to be 84 years. This estimate is unreliable due to the small numbers involved, so I prefer he Italian average.
7

https://www.stats.govt.nz/information-releases/new-zealand-cohort-life-tables-march-2019-update. The median life expectancy for 79-year-old males is 9.9 years.
For females it is 11.5 years. Some data suggests that males are more susceptible than females to COVID-19, this would bias average life expectancy towards the figure
for males. However, New Zealand has more females of this age than it does males, which would bias the figure in the other direction. Absent further data, I have used a
simple average.
8

9

Eight of the 14 deaths as at 20 April were residents of the Rosewood Rest Home, which specialises in dementia care.

According to Ministry of Health figures reporting in https://www.stuff.co.nz/national/health/108004835/median-length-of-stay-in-a-rest-home-for-someonereceiving-govt-funding-is-just-17-years, the median length of stay for those with severe dementia entering residential care in New Zealand was 2.4 years in 2017/18.
The median lengths of stay for those in dementia units and normal rest home facilities was 1.8 and 1.7 years respectively. On average, a current resident will have already
been in the rest home for half of these periods.
10

https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(15)00069-8/fulltext. Severe dementia has a disability weight of 45%. Residents may have other
health conditions that further lower their life quality. I ignore that possibility here.
11

The average period spent in rest home case is 85 weeks according to https://www.stuff.co.nz/business/money/69112545/rest-home-stay-can-cost-over-80000. This
suggests an average remaining life expectancy for current resident of 42.5 weeks.
12

13

https://blogs.otago.ac.nz/pubhealthexpert/2020/03/23/the-maths-and-ethics-of-minimising-covid-19-deaths-in-nz/

https://nzinitiative.org.nz/dmsdocument/618-quantifying-the-wellbeing-costs-of-covid-19-pdf. This value calculated from the spreadsheet at
https://nzinitiative.org.nz/assets/Uploads/Modelling-value-of-health-costs-of-Covid-19.xlsx.
14

New Zealand Productivity Commission
(With minor corrections, 2 May 2020)

2 May 2020

3

 20. International evidence suggests that 5% of confirmed cases need hospitalisation, and 1% of cases overall
result in a death.15 Using 5% and 1.1% (the estimated figure for New Zealand as at 20 April), one new case of
COVID-19 leads to the loss of an average 0.087 QALYs.16
21. New Zealand had 1440 total COVID-19 cases up to 20 April. This equates to cumulative loss of 126 QALYs.
This figure does not contribute directly to the marginal analysis, as it describes the past, which is common to
options SL3 and X5D.
22. Direct health benefits from X5D would arise if the 5-day extension led to fewer COVID-19 cases over time.
Whether it does or not hinges on the Reff value that emerges with level 3 measures. This was a significant
unknown for New Zealand decision makers at the time of the decision. I consider three possibilities that,
between them, cover all eventualities.
a.

Reff averages 1.0 with level 3 controls. With SL3, the average number of new cases would stay at
approximately 10 per day. A 5-day extension of level 4 (X5D) should drive that number down to 5.17
Once level 3 starts, new cases would stay constant at that level. X5D thus offers a future benefit
(compared to SL3) of 2738 cases over the next 18 months.18

b.

Reff averages above 1 with level 3 controls. This affects both options SL3 and X5D. New case
numbers would climb quickly once level 4 controls were relaxed under either option, resulting no
doubt in the re-imposition of those controls before new case numbers reached previous highs of
around 90 a day. The 5-day extension of level 4 might do nothing more than delay its re-imposition
by a week or two. Should that be the case, the total cases expected from options SL3 and X5D over
time would be similar, though their timing may vary. Assuming it takes 5 days (at level 4) for new
cases to drop from 10 to 5, and then 10 days (at level 3) to climb back to 10, the future benefit of
X5D over SL3 is 96 cases over the next 18 months.

c.

If Reff averages below 1 with level 3 controls, then New Zealand is most likely to be looking at a
scenario like that playing out in South Korea.19 New cases in that country have stabilised at around
28 per day or 0.54 per million population, over the preceding fortnight. (New Zealand’s rate was
about 9 times higher over the same period.) If NZ achieved the same “stable” rate as South Korea,
then it would expect an ongoing average of 2.7 cases per day. Any benefit of X5D over SL3 is the
result of reaching this stable rate more quickly. Say that took 8 weeks under SL3 but only 4 under
X5D. Then the future benefit would be 159 cases avoided over the next 18 months.

23. This table outlines the benefits of X5D over SL3, over the next 18 months. QALYs are calculated as 0.082
QALYs per new case avoided. Deaths are calculated assuming a case fatality rate of 1.1%.20
Possibility
a
Reff = 1.0
b
Reff > 1
c
Reff < 1
Used for subsequent analysis

New cases avoided
2738
96
159
2738

QALYs gained
239
8
14
239

COVID-19 deaths avoided
30
1
2
30

24. Australia and other countries are reducing the numbers of new cases with various mixes of control measures
corresponding to New Zealand alert levels ranging from 2 to 4. This makes it highly likely that Reff will
remain below 1 in New Zealand with level 3 controls.

15 The case fatality rate varies significantly by country, reflecting (among other things) differences in the classification of COVID-19 cases, criteria for testing, and in the
classification of causes of death. Within one country it appears to fall over time. See https://ourworldindata.org/coronavirus#what-do-we-know-about-the-risk-ofdying-from-covid-19 for a fuller discussion.

5% is the hospitalisation rate used by New Zealand modellers. See https://cpb-ap-se2.wpmucdn.com/blogs.auckland.ac.nz/dist/d/75/files/2017/01/Supression-andMitigation-Strategies-New-Zealand-TPM-1.pdf and
16

17

With level 4 controls, and after an initial lag, the daily number of new cases has been decaying with a half-life of (very roughly) 5 days.

I use 18 months (ie, 20 October 2021) as the forecast period, assuming that as an average projection for the development and deployment of an effective vaccine. I also
use 12 and 24 months in the sensitivity analysis.
18

19

This is also the case if Reff is below 1 at alert levels 1 or 2, and New Zealand drops to those levels in future.

As at 20 April, the case fatality rate for New Zealand was 1.09% according to https://ourworldindata.org/grapher/coronaviruscfr?country=AUS+NZL+OWID_WRL.
20

New Zealand Productivity Commission
(With minor corrections, 2 May 2020)

2 May 2020

4

 25. So, the most likely benefit from X5D is 159 cases avoided (ie, 14 QALYs gained or 2 COVID-19 deaths
avoided). Across the possibilities examined, the largest estimated benefit is 2738 cases (ie, 239 QALYs or 30
COVID-19 deaths avoided, though this is less likely.
26. A reasonable approach would be to estimate the relative probabilities of (a), (b) and (c), and calculate a
weighted average of the three results. However, as the calculated benefits vary little across the three
possibilities, for this analysis I will assume the outcome with the largest estimated benefit. This corresponds
with Reff averaging 1.0 with level 3 controls.
27. I cover the direct and indirect health costs arising from level 3 and level 4 measures below.

The costs of a 5-day extension over no extension
28. The direct economic costs of the 5-day extension come from the restriction of economic activity. Treasury
calculate a 15% loss of economic output at level 4 relative to level 3.21 This particular 5-day extension is
atypical, as it covers a weekend and a public holiday. Assuming 60% economic output on a Saturday (relative
to a normal weekday), 40% on a Sunday and 20% on a public holiday, these particular 5 days are equivalent to
3.84 days chosen at random.22 So, I will model a 3.84-day extension.
29. New Zealand’s GDP for 2019 was $310bn. The direct effect on GDP of X5D over SL3 is thus a loss of
$489m. Direct effects, however, are not the whole story as GDP does not bounce back immediately after the
cause of a slowdown is removed. I will refer to this as a “slow bounce-back effect”. Many things contribute to
the slow bounce-back, including business failures, loss of customer confidence, and the erosion of staff,
creditor and brand loyalty. The longer the suspension of normal business operations, the longer it takes to
recover.
30. Across scenarios 1–4 in the Treasury’s modelling, direct effects account for on average 33% of the total
nominal GDP loss over 4 years.23 In other words, for every $1 of production lost from direct effects, a further
$2 is lost due to indirect effects. Indirect effects include the slow bounce-back, along with other effects such
as reduced trade from a drop in world GDP.
31. I assume in this model that the slow-bounce back effect adds an additional 50% over direct costs (ie, a
multiplier of 150%). This is equivalent to saying that a restriction of economic activity for four weeks creates
a further two weeks of costs, spread out over time. (I also conduct sensitivity analysis using 125%, 175% and
200%.)
32. The direct health costs of the 5-day extension arise from two sources: (a) the psychological effects of social
distancing and added financial insecurity on the whole population; and (b) the lack of access to normal health
services (eg, elective procedures and primary health consultations) under current conditions. These are in
addition to the loss of production (GDP effects) above. Specifically, for source (b), these undelivered health
services are still being paid for (by Government), so they contribute to production as measured by GDP.
These costs should be accounted for in a marginal analysis.
a.

The psychological effects of social distancing and added financial insecurity across the entire
population are not likely to be small. Anxiety and depressive disorders, for example, are responsible
for the annual loss of an estimated 61 284 quality-adjusted life years in New Zealand.24 If social
distancing measures at level 4 led to a 50% increase in such disorders that lasted, on average, twice
as long as the lockdown itself, then I estimate a QALY loss of 840 per 5 days of level 4 lockdown.
Level 3 relaxes social distancing rules a little and allows for the reopening of some workplaces. If

https://treasury.govt.nz/sites/default/files/2020-04/c19-4265378-t2020-973-economic-scenarios-v3.pdf . The ANZ bank have calculated a similar proportional loss
(https://www.anz.co.nz/content/dam/anzconz/documents/economics-and-market-research/2020/ANZ-QEO-20200422.pdf).
21

The proportions of economic activity on weekend days and public holidays are my assumptions. They are, however, more reasonable than assuming either 0% or 100%
of normal economic activity on non-weekdays. These proportions could be refined using employment or consumer spending data.

22

23

I omitted scenario 5 as it assumes a different decline in world GDP than do the other four.

See https://www.moh.govt.nz/notebook/nbbooks.nsf/0/F85C39E4495B9684CC257BD3006F6299/$file/health-loss-in-new-zealand-final.pdf. That study reported
in terms of disability-adjusted life years (DALYs). These are sufficiently close to QALYs for the purpose of this analysis. That study estimated 50954 DALYs for 2006. I
have scaled this to 61284, adjusting for population growth.
24

New Zealand Productivity Commission
(With minor corrections, 2 May 2020)

2 May 2020

5

 the increase at level 3 was only half that of level 4, then the anxiety-related health cost of X5D
would be 420 QALYs.
b.

Some, but not all normal health services will resume at level 3.25 With level 4, only approximately
50% of hospital capacity was in use, and many other health services were under-utilised.26 To
demonstrate, I will assume that this increases to 60% at level 3. Ten percent of health spending
amounts to a health loss of $1.27 million over the 5 days of the level 4 extension (equating what
New Zealand spends on health with the health value of that spending).

33. These assumptions are indicative rather than conclusive, so the resulting costs need to be treated with
caution. The costs are real, however, so it would be misleading to omit them. Accordingly, they are included
in the totals below.

Comparing costs and benefits
34. GDP and QALYs are at one level, incommensurate. However, it is common in public health and in economics
to convert QALYs to dollars. If is reasonable to convert from QALYs to dollars (and present this as a % of
GDP), then it is also reasonable to convert the other way. The logic behind this two-way conversion goes
like this. If one argues that a country should be willing to spend say $33000 to avoid the loss of a QALY, then
not having $33000 destroys the country’s option to avoid the loss of a QALY.
35. This is not an ethical judgement that the two are somehow equivalent and can be traded off against each
other. Rather, it is a recognition that the two are interdependent – a healthier population results in a
healthier economy, and a healthy economy results in a healthier population.
36. Richer countries tend to spend more per capita on health than poorer ones.27 Part of the reason is that health
is labour intensive, and labour is more expensive in richer countries. A further reason is that richer countries
can buy health treatments that may be simply unaffordable in poorer ones.
37. A lower future GDP will restrict the incomes of New Zealanders – individually and collectively. This has
corresponding implications for what New Zealanders can afford to spend on maintaining and enhancing their
health.
38. While the actual conversion factor can be controversial, plausible alternatives fall within a range. If a society
was willing to spend its entire resources to avoid the deaths of the entire population, then the implicit
conversion factor is 1 QALY = per-capita GDP. As under normal circumstances people require more to live
than can be satisfied by health-related spending, this represents an upper limit. A lower limit could be
obtained by dividing the GDP of a country by its total health spending – this is around 9.5% for
New Zealand. This would imply a conversion factor of $5950 per QALY in New Zealand.
39. Blakely, Baker and Wilson suggest using per-capita GDP as the value of a QALY. This is currently roughly
$62 607. However, they used a somewhat lower figure of $43 000 for COVID-19 analysis.28 Treasury
recommends using $33 000 in its CBAx guidance.29 Wilkinson suggests that a better proxy is national
disposable income per capita ($52,500 for the 2019 calendar year).30 These estimates fall within the upper and
lower limits discussed above. In this analysis I used Treasury’s recommendation. (The sensitivity analysis
includes results for the $43 000 and $62 607.)
40. The following table summarises the costs and benefits.
X5D (against the counterfactual of SL3)

Value ($m)

QALYs

Refer to paragraphs

See https://meras.midwife.org.nz/wp-content/uploads/sites/4/2020/03/National-Hospital-Response-Framework-4pm.pdf,
https://www.health.govt.nz/system/files/documents/pages/health_and_disability_services_at_alert_level_3_-infosheet_0.pdf and https://www.health.govt.nz/ourwork/diseases-and-conditions/covid-19-novel-coronavirus/covid-19-current-situation/health-and-disability-services-alert-level-3.
25

26

https://www.rnz.co.nz/news/national/413398/covid-19-new-zealand-hospitals-prepare-to-handle-virus-patients.

27

https://drewsmithblog.com/2018/01/which-countries-put-the-highest-value-on-human-life-and-health/.

28

https://blogs.otago.ac.nz/pubhealthexpert/2020/03/23/the-maths-and-ethics-of-minimising-covid-19-deaths-in-nz/

29

https://treasury.govt.nz/publications/guide/cbax-spreadsheet-model-0

30

https://blogs.otago.ac.nz/pubhealthexpert/2020/03/23/the-maths-and-ethics-of-minimising-covid-19-deaths-in-nz/

New Zealand Productivity Commission
(With minor corrections, 2 May 2020)

2 May 2020

6

 Benefits
Deaths and illness avoided
Total benefits (original units)

Total benefits (equivalent units)

8

239
239
239

18–26
34–39

Costs
Direct (GDP loss)
Indirect (slow bounce-back of GDP)
Psychological
Health services paid for but not delivered
Total costs (original units)

-489
-245
-420

28–29
29–31
32(a)
32(b)

Total costs (equivalent units)

-1
-735
-749

-420
-22 692

34–39

Net benefits (equivalent units)

-741

-22 453

34–39

41. The net cost of X5D (an extra 5 days at level 4) over SL3 (straight to level 3) is large whether measured in
dollars or QALYs. Using Treasury’s $33 000 figure for the value of a QALY, the net marginal cost is $741m.
This is equivalent to 22 453 QALYs lost.
42. Alternatively expressed, X5D results in, at best, 239 QALYs gained (including the value of 30 COVID-19
deaths avoided). This benefit comes at a substantial cost: $749 million (or equivalently, 22 692 QALYs lost).

Other costs and benefits
43. This analysis does not include incidental health benefits of the lockdown. Road deaths, sports injuries and
other accidents have all dropped with level 4 controls (and they might be expected to rise with level 3
controls). Such health benefits, however, are not costless. People value their freedom to travel, play sport and
take part in other activities that come with a higher risk of personal injury. Under normal circumstances,
society accepts that these activities should be allowed – implicitly assuming they have a net benefit. (Society
bans other activities that do not meet this criterion.) Incidental drops in accident-related health costs should
not be treated as an unambiguous benefit in a cost-benefit framework.
44. There are other costs and benefits, such as the impact on natural capital, which are harder to quantify and
potentially require a higher number of assumptions regarding potential impacts. These are not included in
this analysis. (See appendix for further information.)

Sensitivity analysis
45. I re-ran the model with changed parameters, as per the following table. The net benefits calculated by the
model do not appear overly sensitive to these parameters.

2%
0.5%
3
7.5
12
24
43000
62607
125%
175%
200%
10%
20%
2.5%
7.5%

Net value Net QALYs
($m)
gained
-741
-22453
-737
-22334
-744
-22533
-743
-22514
-738
-22378
-744
-22533
-738
-22374
-743
-17274
-746
-11921
-619
-18748
-863
-26159
-986
-29865
-496
-15042
-986
-29865
-741
-22461
-741
-22446

New Zealand Productivity Commission

2 May 2020

Parameter

Base
model Alternative

Base model
Case fatality rate
QALY per
COVID-19 death
Time to vaccine
(months)
Dollar value of
QALY
Indirect GDP
effect multiplier

1.1%
5
18
33000
150%

Loss of GDP, level
4 vs level 3

15%

Cases hospitalised

5%

(With minor corrections, 2 May 2020)

COVID-19
deaths
avoided
30
55
14
30
30
20
40
30
30
30
30
30
30
30
30
30

Net value Net QALYs
($m)
gained

-0.5%
0.4%
0.3%
-0.3%
0.4%
-0.4%
0.2%
0.7%
-16.5%
16.5%
33.0%
-33.0%
33.0%
0.0%
0.0%

-0.5%
0.4%
0.3%
-0.3%
0.4%
-0.4%
-23.1%
-46.9%
-16.5%
16.5%
33.0%
-33.0%
33.0%
0.0%
0.0%

COVID-19
deaths
avoided

82%
-55%
0%
0%
-33%
33%
0%
0%
0%
0%
0%
0%
0%
0%
0%

7

 Conclusions
46. This analysis concludes that SL3 (straight to level 3) outperforms X5D (an extra 5 days at level 4) on all
criteria considered. The net cost of X5D is $741m. This is equivalent to 22 453 QALYs lost.
47. Alternatively expressed, X5D results in, at best, 239 QALYs gained (including the value of 30 COVID-19
deaths avoided). This benefit comes at a substantial cost: $749 million (or equivalently, 22 692 QALYs lost).
48. The decision point to extend level 4 is in the past, so this analysis cannot inform it. However, this technique
could be applied to inform the next decision point (11 May), when the Government most likely faces the
decision to stay at level 3, or to shift to level 2 or back up to level 4. Similarly, the Government will face
decisions in the future on lifting or changing border controls. This analytical technique could also be
appropriate for assessing intra-alert level decisions, for example including or excluding specific measures in
an alert level.
49. Leading up to the next decision point, the relevant information needs to be collected to inform that decision.
For example, updates and improvements to Treasury’s economic modelling would improve CBA modelling
of COVID-19 related decisions.

Contacts
Author: Dave Heatley, New Zealand Productivity Commission
Additional modelling and research: Judy Kavanagh and s9(2)(a)
External peer review: Joanne Leung and Dieter Katz
Internal (Productivity Commission) peer review: Gail Pacheco, Andrew Sweet, Murray Sherwin and Geoff Lewis.
Further information: Judy Kavanagh or Dave Heatley

Appendix – Peripheral issues and other concerns
The “elimination” goal
50. What about the Government’s stated goal to “eliminate” the virus? The World Health Organisation is yet to
issue a technical definition of elimination for COVID-19. Looking at its definitions for the elimination of
other infectious diseases, such a definition would include four criteria: (a) zero incidence of transmission; (b)
in a defined geographical area; (c) over a defined time period; and (d) in the presence of a well-performing
surveillance system.31
50A. The main uncertainty is with criterion (c) – the required time period. Twelve months appears to be the norm
(eg, measles, rubella). A year with zero transmission of a disease would offer a very high level of confidence
that COVID-19 was no longer present. However, a year is too long to inform adaptive management during a
pandemic.
50B. As at 20 April, New Zealand was yet to have a single day with no new cases. However, a single day would
offer no confidence that elimination had been achieved, as the incubation period of COVID-19 is at least 5
days. This suggests that elimination (under any reasonable definition) could not have been achieved before
the end of the 5-day extension to level 4 (27 April).
50C. A useful concept for adaptive management might be “medium-confidence elimination”, defined as “28 days
with zero new cases”. (I suggest 28 days as this is roughly two virus reproduction cycles.)

See https://www.who.int/neglected_diseases/resources/NTD_Generic_Framework_2015.pdf and https://www.who.int/wer/2013/wer8809.pdf?ua=1. Elimination
is not the same as having 100% confidence that the virus will not re-appear in a country. Infections may be undetected because of under-reporting, false positive tests, or
because they are asymptomatic. These factors also make border quarantine measures fallible. Elimination would leave the population susceptible, so a significant subset
of current virus control measures would need to remain in place.
31

New Zealand Productivity Commission
(With minor corrections, 2 May 2020)

2 May 2020

8

 51. Elimination (under either option) cannot be progressed unless possibility (c) in paragraph 22 applied (ie, Reff
< 1 with level 3 controls). So, an elimination case must be a subset of possibility (c).
52. It would be extremely unlikely that X5D would be sufficient to make a difference between elimination of the
virus and a failure to eliminate. A 5-day extension could change the starting point (say 5 cases/day rather
than 10) for level 3. In an “optimistic” scenario in which X5D led to medium-confidence elimination in 9
weeks and SL3 led to medium-confidence elimination in 14 weeks, its benefit would be 280 cases avoided (ie,
24 QALYs gained or 3 COVID-19 deaths avoided).

The value in reducing uncertainty over a future return to level
4
53. A couple of reviewers suggested that an extra 5 days reduced the uncertainty for Cabinet about whether the
big reduction in case numbers over the 10 days before 20 April would be sustained in following days, and
gave Cabinet comfort that they would not have to order a return to alert level 4. I agree that reducing
uncertainty had a benefit for the decision makers, but I am less clear this translates into a benefit for
New Zealand as a whole.
54. Following the logic presented in this note, the only circumstances under which Cabinet would “have to order
a return to alert level 4” was if the effective reproduction number Reff with level 3 controls turned out to be
greater than 1. But, if that turned out to be the case, then the extra 5 days at level 4 offered little “comfort” to
Cabinet – it would only delay, not prevent, a return to level 4.
55. One way to deal with uncertainty is to assign probabilities to outcomes. This note considers three outcomes:
Reff > 1, Reff = 1 and Reff < 1. While the analysis suggests the third outcome is the most likely, it actually uses
the second outcome in the subsequent analysis, on the basis that it offers the highest health benefits. By
making this choice, the analysis more than compensates for the uncertainties faced by decision makers.

The value in waiting until “gold standard” tracing capability
was in place
56. Reviewers also speculated as to whether the 5-day extension was chosen by decision makers because contact
tracing capacity was not yet at the “gold standard” required to “safely” move to alert level 3 on 23 April.32
57. I question whether the health system really needed to gear up to be able to deal with 1 000 cases per day
under alert level 3 as recommended by the Verrall Report.33 The report noted that “In March the workload
of PHUs [Public Health Units] exceeded their capacity to conduct rapid contact tracing on occasion, even
though case numbers were less than 100 per day”. Granted, tracing was occasionally overwhelmed at
90 cases per day in March. Even so, the less-than-gold standard tracing capability was sufficient to rapidly
reduce new cases down to approximately 10 per day under alert level 4. It seems extremely unlikely that
tracing capacity would become quickly overwhelmed after a shift to level 3 on 23 April.
58. The incubation period of COVID-19 is roughly 5 days, so any additional infections that occurred after the
move to level 3 would not have shown up until 5 days after the change of level. So, the earliest date that
implementation of the “gold standard” was (possibly) needed was 28 April, not 23 April.
59. I can see no clear benefit from delaying the move to level 3 until “gold standard” tracing capacity was in
place.

Does an extra 5 days really matter for business?
60. One reviewer asked whether the marginal cost of an extra 5 days at level 4 was material for businesses. This
might not matter for a business that faced a fixed cost of restarting production, and whose costs of interest,
rent of premises, and wages were being met by government subsidies (or deferred through other government
32 National MP Michael Woodhouse cast doubt on contact tracing ability: "Now unless they're able to fix that, the government won't be able to have the confidence that
they can lift out of level 3". https://www.rnz.co.nz/news/political/414707/contact-tracing-capacity-a-mathematical-impossibility-michael-woodhouse
33

https://www.health.govt.nz/system/files/documents/publications/contact_tracing_report_verrall.pdf

New Zealand Productivity Commission
(With minor corrections, 2 May 2020)

2 May 2020

9

 programmes). However, many businesses would not meet those criteria, and government support may cover
just a fraction of their actual costs. Further, deferred costs still contribute to insolvency tests, and many
businesses are already financially stressed. So, for some businesses at least, the extra 5 days will tip them into
insolvency. Such business failures have direct effects on their owners and staff, but also indirect effects on
other firms that they buy from or supply to. Business failure imposes an externality on other businesses,
which in some cases can cascade.
61. For businesses, the longer the lockdown, the more decay of relationships with investors, creditors, customers
and staff. The lower the likelihood that the firm’s suppliers and business customers are still in operation. All
this needs to be “patched up”, or substituted for, before their pre-lockdown state can be regained. The
business costs of lockdowns will thus vary with the length, as well as the severity, of lockdown.

Private and social costs
62. One reviewer noted that, according to the media and his own straw polling, there has been generally strong
support for the 5-day extension. How can that be if the cost is so large? The reviewer noted that many of his
friends and acquaintances have continued to work or are retired, and the extra 5 days has cost them little,
while they receive extra assurance that they remain safe from COVID-19. Those continuing to get paid while
not working may be in a similar position. The wage subsidy has driven a wedge between private returns and
social returns under lockdown.
63. More generally, economic theory would assume that individually rational agents do not internalise infection
externalities they impose upon others when disease is transmitted. A paper by Bethune and Korinek found for
the United States that private agents perceive the cost an additional SARS-CoV-2 infection to be
approximately $80k, whereas the social cost including infection externalities is more than three times higher,
at approximately $286k.34
64. The analysis in my paper takes a societal perspective.

The four capitals and the living standards framework
65. My analysis covers two of the four capitals in Treasury’s living standards framework. Human, and financial
and physical capital are included, while social and natural capital are not (explicitly) covered. This is not an
ethical judgement that the omitted capitals are somehow less important. Rather it reflects my judgement that
(a) current techniques for the measurement of changes to the components of those capitals are
underdeveloped; (b) there are no publicly available nor widely accepted weights to apply to those
components; (c) the dashboards of indicators approach currently used by Treasury and Stats NZ can lead to
problems of both omission and double counting; and (d) there are (albeit imperfect) shadow prices in the
economy for at many of the components of these capitals.
66. As an example, the net environmental effects of X5D are ambiguous. An obvious benefit is fewer greenhouse
gas emissions during the lockdown period, though I might expect some pickup to service pent-up demand
once restrictions are lifted. (Greenhouse gas emissions, to the extent that they are priced in under the
emissions trading scheme, already appear in economic analyses. To avoid double counting, only the unpriced
portion should be included under this capital.) An obvious environmental cost is the effect on native flora and
fauna of the reduction of pest animal control accompanying the lockdown. Even if it was straightforward to
calculate the greenhouse emissions avoided, or the kea, kiwi and kaka lives lost, from X5D, establishing
appropriate weights for a tonne of CO2 emitted vs. a kea life would not be.
67. The loss of economic output that is included in this analysis will reduce the options of current and future
New Zealanders – including their ability to mitigate greenhouse emissions (eg, by purchasing more fuelefficient cars) and protect native birds (eg, by expanding the pest animal trapping network).

34

https://www.nber.org/papers/w27009

New Zealand Productivity Commission
(With minor corrections, 2 May 2020)

2 May 2020

10

 68. In my judgement, the direct benefits and costs of X5D on natural and social capital are likely to be small
relative to the direct benefits and costs on the other capitals. However, the direct impact of X5D on the
financial and physical capital domain is likely to have substantial net costs for natural capital.
69. Similar, but perhaps greater, uncertainties apply to the norms, rules and institutions that contribute to social
capital. From one perspective, mandated social distancing limits freedom of association and freedom of
movement, thus reducing social capital. Taking another perspective, the “we’re all in this together” nature of
the Government and public response are making a positive contribution to social capital. I am unclear how to
quantify such factors individually, and even less clear how they might be appropriately weighted.

Sensitivity analysis
70. A further refinement to the sensitivity analysis would be use Monte Carlo modelling of plausible ranges for
parameter values, rather than point estimates. This would provide a confidence interval around the estimates
of the base model.

New Zealand Productivity Commission
(With minor corrections, 2 May 2020)

2 May 2020

11