We would expect a fair and balanced assessment of research from one of New Zealand’s reputable tertiary institutions.
Unfortunately, I was disappointed with the quality of analysis of international research of Massey University’s 100 page Report, ‘Charter Schools for New Zealand.’
Studies aimed at determining the effectiveness of a policy should include or exclude findings from evaluations based on whether they meet common criteria for high quality research.
These criteria include whether studies use experimental techniques, statistical control for participants’ background characteristics and other factors that could influence the observed effects on participants and sample of a large population over a period of time.
Missing criteria
The Massey report does not list the criteria by which the researchers decided whether to include or exclude studies. In their opinion, they were ‘satisfied’ that the evidence pointed to charter schools having negative effects. On several occasions, the Report selectively cited findings to make it seem like they say one thing, when they mean another. At best, this is negligence; at worst it is a gross misrepresentation of academic research.
I do not have the space to assess every study or example that the authors have cited or written but I will cover a few glaring mistakes or misrepresentations which they have made about the US situation.
The Massey Report’s authors cite the overall finding from the ‘Stanford University CREDO (2009) Study, that 17% of the charter schools examined, “provided superior education opportunities for their students,” 46% produced results that were not statistically different from local schools and 37% provided learning results that were worse than their pupils would have achieved if they had stayed in regular state schools.
The first point to note is that the authors incorrectly state 56% performed no worse when it was actually 46%. Adding the percentages make it 110%.
When discussing which pupils benefited from charter schools, the authors do not note that the CREDO study found modest but statistically significant achievements in reading and mathematics test scores for pupils in poverty and for whom English is a second language.
Facts overlooked
They also fail to note that even though the study detected wide variation in charter schools’ performance.
The CREDO report’s authors did not think this was enough of a reason to deem charter schools harmful or to recommend that every charter school should be closed. In fact, the CREDO report’s authors suggested that policy-makers should try to uncover why some charter schools are effective and the type of policies that have worked in some States so that they can be replicated.
While it is fair to say that studies of average pupil performance have limitations for assessing particular groups of pupils, other research is helping us to learn more and more about how charter schools may or may not make a difference to specific groups of pupils’ learning.
The authors also assess the KIPP (Knowledge is Power Programme) Charter School Programme, which some research and evaluations (accepted by the Massey authors) indicate has been successful at boosting the educational achievement of pupils from low-income families.
Negative questions
KIPP schools also tend to enrol high shares of African-American and Hispanic pupils. The Massey authors put forward a series of questions about whether KIPP schools make a difference to the achievement of lower-ability pupils of these backgrounds, one of which was: “The fact that the roll is almost entirely ethnically based suggests that there are racist overtones: are such groups being targeted for a relatively narrow approach to learning?”
This is quite a disingenuous line of questioning, where one asks negative questions without providing any answers. This particular question ignores how KIPP’s approach to teaching was based heavily on that of Harriett Ball, an African-American teacher who was highly successful in her education of African-American children.
We need to debate the merits of charter schools, rather than engage in an ideologically-polarised debate, if Charter Schools are to be successful at helping our children to learn. For this to happen, we must know whether they are effective.
One of the authors’ last recommendations is to require New Zealand Charter Schools to be subject to accountability measures through the collection and reporting of student-level data on their background characteristics and their prior achievement. This should be encouraged, not only in charter schools but also in all New Zealand state schools.
The above is a highly edited version of an analysis written by Steve Thomas, a Senior Researcher and Media person at Maxim Institute based in Auckland. To read the full text, please visit www.maxim.org.nz
A Massey University report on the subject appears elsewhere in this section.
