Avoiding confirmation bias 

There are "Lies, damned lies and statistics" as the famous saying goes. And I might add ‘… and inconvenient parts of statistics that are best left out’.  A case in point is this graph, which came across my radar from an anti-masker.

The point that the person was making was that the number of positive cases of COVID-19 is not influenced by mask-wearing; it is ineffective.

I happen to think that the question ‘Do masks help stop the spread of COVID-19’ is an important question to ask. It’s one that I have asked myself.

So, the first thing that I want to do is to check the statistics. I download the raw data and perform the same 7 day moving averaging to show the trends. Those statistics are correct.

The next thing is to check the statistics about mask-wearing; that North Dakota had obligatory mask-wearing and business restrictions, whereas South Dakota did not. And it’s here that the author has used a more selective approach. Neither state had a mask wearing policy until November 14. On this date (arrowed on my graph), North Dakota introduced state-wide mask-wearing in public places and some business restrictions. South Dakota did not.

The author of the graph wants the reader to think that the restrictions in North Dakota have been a long-term thing, which have made little difference to the spread of COVID-19. Whereas those restrictions were only introduced near the peak of the outbreak. The graph therefore tells you almost nothing about the correlation between mask-wearing and COVID-19 cases (although some may argue that the reason that rates in North Dakota have now fallen below that of South Dakota is because of mask-wearing).

The full set of statistics does not support the author’s original claim. But, of course, the whole picture is much more complex than this. For example, while there was no state-wide mask-wearing policy in South Dakota some local areas did enforce their own policies (for example the city of Brookings introduced compulsory mask-wearing on 8 September).

It is also important to note that even if there is a correlation between two factors, it does not mean that one factor actually causes a change in the other. Correlation does not imply causation. For example:

There is a clear correlation here but I think that most of us would accept that ice cream does not cause death by drowning. Other factors (such as warmer weather, holidays, numbers of tourists) are likely to influence both ice cream consumption and drowning.

It is important that we all question the world around us and ask things like ‘Do masks make a difference?’, ‘Is this vaccine safe?’, ‘Does social distancing work?’, ‘Is COVID-19 really that bad?’ etc. etc. Scientists ask exactly the same questions. They then try to come up with answers. However, key to the scientific approach is trying to avoid ‘confirmation bias’ – looking for things that support what you already believe, rather than looking at all data completely objectively.

In a free society, we all have a duty to ourselves and others to ask questions. But we must try think objectively as we probe for the truth and ask:

a) What is the evidence?

b) How good is that evidence / can I trust it?

c) What objective conclusions can I draw from that evidence?

When people ask their questions, most carry out step (a) above but do themselves a great disservice by forgetting to carry out steps (b) and (c).

Text: Mark Levesley   @marklevesley    levesley.com

Photo: https://www.nursetogether.com/, CC BY 4.0 <https://creativecommons.org/licenses/by/4.0>, via Wikimedia Commons