Quote of the day

“I find economics increasingly satisfactory, and I think I am rather good at it.”– John Maynard Keynes

Sunday 4 December 2022

The problem with statistics

 Call me geeky but I found this quite fascinating - errors in census data can have a big impact on funding for schools and more. It is one of those things it is useful to have in your toolkit in case you are making a point about policy directed at very specific issues and want to question whether the data the decision is based on is reliable:


Can we trust what the census says?

Crucial policy decisions are based on its data, but it’s only as accurate as the people filling it in: us

Tom Calver
The Sunday Times

When the 2011 census said just 9,000 people lived in Whitechapel, east London, Dr Shlomit Flint Ashery refused to believe it.

Armed with a clipboard and a grasp of Arabic and Bengali, the researcher at Bar Ilan University, Israel, conducted her own door-to-door survey of the ethnically diverse area. She befriended religious leaders in mosques and mothers in local parks. After weeks of work she had recorded the names of 13,000 individuals – 4,000 more than the census thought.

“The census is generally intended for the mainstream population,” says Flint Ashery, “and it tends to be blind to people from other cultures.” Specifically, in Whitechapel, it was not very good at counting households that contained multiple generations, or multiple families – especially among tight-knit migrant communities where distrust of the authorities is high.

Unlike the biblical iteration that required Joseph of Nazareth to return to his hometown, the modern census is a “snapshot” of the day it is filled in
Unlike the biblical iteration that required Joseph of Nazareth to return to his hometown, the modern census is a “snapshot” of the day it is filled in
ALAMY

The census is hailed as the gold standard: the Office for National Statistics (ONS) invests millions in ensuring it is done correctly. Filling it in is mandatory — those who didn’t in 2021 faced a £1,000 fine — and last time 89 per cent did so online. In Scotland, a dismal response rate of 79 per cent has forced an inquiry. Everything from homes, healthcare, schools and even sewer capacity is planned from the information it collects. Can we trust it?

In 1911 the suffragette Emily Davison, later killed by the King’s horse at Epsom, was found “hiding in the crypt of Westminster Hall” by census officials. This was recorded as her address.

Here lies a limitation of the modern census: unlike the biblical iteration that required Joseph of Nazareth to return to his hometown, it is a “snapshot” of the population of where they are on the day it is filled in.

The latest census was conducted in early 2021. Britain was in lockdown, so many of us were not where we usually live. Students who had gone home for Christmas did not immediately return to university; families may have gone to live with grandparents or in second homes.

Officials believe this made the population of some London boroughs seem much less than it usually is. Camden’s population — which the ONS previously thought was rising — nominally shrank by 10,000 over ten years to 210,000. Councillors, fearing the borough could suffer a reduction in government funding as a result, have joined a campaign to find the city’s “lost Londoners”.

The apparent drops are most significant among children, which could have implications for school funding. “Across London, billions of pounds for frontline services are at stake, and our residents will be the ones who suffer,” the Camden council leader Georgia Gould said in the summer.

Perhaps a greater problem is that the census is simply not very good at reaching certain communities. Some 97 per cent of households responded to the census, according to the ONS, up from 94 per cent a decade earlier: Professor Tony Champion, emeritus professor of population geography at Newcastle University, points out that the fact it happened “when most people were bored out of their minds and had plenty of time to complete the form” may have helped.

There are other reasons to treat census figures with a pinch of salt, as another population-wide programme conducted at the same time shows. Some 6.1 million people in their sixties had received a first vaccine dose by the end of 2021 — around 1 per cent more than the total number of sixty-somethings recorded in the census.

The latest census was conducted in early 2021 and recorded the population of England and Wales as 59,597,300
The latest census was conducted in early 2021 and recorded the population of England and Wales as 59,597,300
MATT ALEXANDER/PA

Comprehension has mercifully improved since the first census forms were handed out 200 years ago. When asked of his “relationship” to the head of the household, one Victorian agricultural labour, living in a shed at the bottom of a farmer’s garden, replied “friendly”. Yet even now that most of us can read, some of the categories remain ambiguous. If you are not one of Britain’s two million churchgoers but enjoy some fruits of the Christian tradition, do you mark your religion as Christian? Even ethnicity is, apparently, not that objective. “I changed my ethnicity between 2011 and 2021,” says Professor Edward Higgs, a leading census historian: “I put white Irish this time.”

Censuses are good at tracking change — but not if you change the questions. For last year’s census, the ONS moved “British” to the top of the census form, where “English” had been a decade before.

This was, it says, to make it easier to answer the question — yet in a striking example of “question-order bias”, the number of people identifying as “English” fell from 58 per cent, to 15 per cent. Some of that fall may be a genuine collapse in “English” identity, but we have no way of knowing.

Does any of this matter? It very much can do. For years, the ONS forecast that Coventry’s population would rise by 32 per cent between 2011 and 2031 – twice as fast as Birmingham’s. Yet on closer inspection, it emerged that these projections expected most of the city’s substantial student population to stay in Coventry after graduating (most do not). Thousands of homes for these graduate ghosts that were set to be built — some on greenbelt land — were not really needed.

There is a solution to these census imperfections. Countries like Norway have a joined-up “population register”, where data on taxation, benefits, schooling, health and education — currently separate in Britain — are all linked together in a single database. This approach may not sit well with some privacy campaigners, and the ONS is set to decide next year whether it can adopt a similar approach here.

This most recent census may well end up being the “best and highest quality” so far, according to Champion. Yet as long as it is filled in manually, it can only ever be as accurate as the people filling it in.

@TomHCalver

No comments:

Post a Comment