You might remember that I wrote about this last year. Now the Ottawa public school board has some results to talk about. And they are for sure talking about it.
The big takeaway for them is the diversity of the student population. Now, anyone can see this for themselves by following a school bus around (not that I have done this) and watching who gets on. It doesn’t take a genius, or a survey, to observe that.
In fact, this is perhaps a more accurate way to judge the extent of diversity of students on our schools than the survey. Why? Because, exactly as I predicted, the response rates turned out to be too low in order to arrive at conclusions about the whole student body; either at the Board or the school level.
The OCDSB admits that the rate was lower than they had wanted… under 50% for elementary schools and mid-sixties for our high schools. The reason why these low rates are problematic is that when one applies the conclusions to the population and then uses this as information upon which to make Board and school level decisions, they might be using flawed data.
Why? We don’t know who didn’t reply to the survey. The demographic characteristics of non responders might be quite different from those who filled in the questionnaire. In survey parlance, this is called non response bias.
There are ways to adjust for this bias but given that there are no baseline data that one can use to assess the effect that the bias may have had, these adjustments are not possible.
Now, the Board could do something about this and hire a qualified consultant to randomly select a number of schools that represent different areas of the City. Then go into the schools and observe the actual profile of the population; then compare these data to the data obtained through the survey instrument. Then adjustments could be made if required. This is also called data confrontation.
But unless they do this, or something line it, these results aren’t very useful. Indeed, using them may do more harm than good!