An incredibly stupid decision on the 2011 census

There is no exaggerating the boneheadedness of this decision:

Tories scrap mandatory long-form census: For the first time in 35 years, the census will not feature a detailed, long form that Canadians are obliged to send back to the government.

Instead, a mandatory short form will go out to everyone for next year's census, with basic questions about how many people live in the household and their ages and genders.

The voluntary “national household survey,” with detailed questions about ethnicity, income and education, will be sent to one-third of homes. That's an increase from the 20 per cent of homes that used to get the mandatory long-form…

It's not often that sample selection bias becomes an issue of national importance, but then again, it's not often that census sampling design is outsourced to drunken monkeys. The people who thought that increasing the number of long forms to be sent out will somehow compensate for making the long-form responses voluntary simply do not understand the gravity of the error they are making.

Suppose that we want to estimate the average height of Canadians, and the following sampling design is proposed:

  1. Find a group of people who are more than 6 feet in height.
  2. Ask them how tall they are.
  3. Take the average of the answers you get.

Those of us who are sentient will see that no matter how big you make the sample, this procedure is fundamentally flawed and will produce worse-than-useless results. If you don't see this, worry not: you may yet have a promising career as Minister of Industry.

There is no reason to believe that response rates will not be correlated with the features we are trying to measure: certain groups will be more likely to answer, others will be less likely. These variations in response rates will generate distorted data. In fact, the problem is worse than the simple example described above: we don't know which groups are systematically excluding themselves from the census.

One of the most pressing policy questions facing Canada is how best to develop our skills and improve upon education attainment levels. We need the best possible data to develop policy, and this measure would provide us with data that are at best badly flawed, and are likely to be completely useless.

The census is an exercise that consumes a enormous amount of time and resources. They shouldn't be wasted in this way.

Update: Here's a more temperately-worded explanation for why this is a bad idea.

55 comments

  1. Stephen Gordon's avatar

    Because there aren’t any similar cases involving StatsCan employees?

  2. Bill Smith's avatar
    Bill Smith · · Reply

    Because there aren’t any similar cases involving StatsCan employees?
    Cases that you know of. We only learned of the CRA cases due to Access to Information requests by some tenacious journalists.
    Given the propensity of many levels of government to stonewall & cover up wrongdoing, the fact that misuse of Statcan data hasn’t been in the news doesn’t mean much.
    In fact, I recall during the 2002 census there was some bizarre mix-up and hundreds of completed census forms were mailed to other people’s residences.
    Now, if Statscan’s (and other government departments that routinely deal with very private information) security procedures were audited in detail, and if bureaucrats who abuse their authority are fined, lose their jobs, and go to jail for a long time, I might agree with you.

  3. Patrick's avatar
    Patrick · · Reply

    Lemme see if I’ve got this straight. You’re so worried about a hypothetical situation for which their is no concrete evidence that you’d just toss out the whole thing despite the obvious benefits? Uh huh.

  4. Bill Smith's avatar
    Bill Smith · · Reply

    Lemme see if I’ve got this straight. You’re so worried about a hypothetical situation for which their is no concrete evidence that you’d just toss out the whole thing despite the obvious benefits? Uh huh.
    Given that this sort of thing regularly goes on in other government departments, even then, it is rarely caught, and even when it is caught, they get a slap on the wrist. So I am concerned about misuse, fraud & theft. Any reasonable person should be.
    If you owned a business that regularly handles large amounts of cash, you would probably take precautions. You would have regular cash counts by different people, strong locks, safes, alarms, security cameras & audits.
    You would likely do these prudent things even if you have never been robbed (either by your employees or by outsiders). If there was a theft, you would want them prosecuted and sent to jail, pour encourager les autres.
    You might even do something like reducing the amount of cash you keep on hand, to mitigate the size of a theft.
    These are the same government bureaucrats who couldn’t account for $15 million of gold at the Royal Canadian Mint. Oddly enough, one of the first things Mint officials did was conduct some opinion polls and hire a PR firm. Eventually, the RCMP determined that no theft occurred, but that the record-keeping practices & internal controls of the Mint had been terrible for many years.
    So yes, I do have difficulty trusting in the competence & honesty of government bureaucrats.
    Given the available evidence, and the lack of consequences for those who misuse government data, I can’t see how any reasonable person wouldn’t be concerned.

  5. Patrick's avatar
    Patrick · · Reply

    You’re becoming incoherent.
    What does an administrative snafu at the mint have to do with the claim the StatsCan employees are incompetent and corrupt? Totally irrelevant.
    And if you think the private sector is any better than the public on privacy, you’re living in a dream world.

Leave a comment