In the summer of 2005, it was announced that the government had commissioned this review by Torgerson et al. and that Jim Rose would have the opportunity to draw on it (see the government’s response to the Education and Skills Committee’s report ‘Teaching Children to Read’). The review was published in January 2006. Diane McGuinness’s comments on it can be read in full at www.rrf.org.uk.
Prof. McGuinness writes that the Torgerson et al. review ‘purports to be a reworking of the National Reading Panel’s report in the US (2000) or, more accurately, the reading committee chair’s report of the same data (Ehri et al., 2001)’. ‘The reality is that every statement under the heading “Key Findings” [in the Torgerson et al. review] is incorrect or seriously compromised by the true facts.’ She points out that ‘there are enormous difficulties with meta-analysis research’ of this type and that ‘without great care researchers can combine apples and oranges’, producing studies in which ‘variables are too dissimilar to be combined’. For a number of reasons, ‘studies on remedial tutoring and early classroom instruction cannot be compared directly and certainly should not be included in the same meta-analysis. This precaution was not taken by the NRP committee nor by Torgerson et al.’
Furthermore, Prof. McGuinness comments that the Torgerson et al. decision to include studies that were not peer-reviewed is unacceptable: ‘peer-reviewed material is absolutely paramount to doing research of this type’. She is also critical of the excessive weight attached to ‘random assignment to experimental and control groups’, pointing out that this is not practical ‘given the basic reality of how schools work’. Finally, she criticises the authors for stating that they found 14 studies which included ‘comparisons between synthetic phonics and analytic phonics’. She herself ‘could only find one’.
In private discussion, others have expressed similar concern about the Torgerson et al. review, for example criticising its inclusion of an unpublished study by Skailand dating from 1971. In this study, children were trained on pairs of words such as ‘tap’ and ‘tape’, so 50% of the words were ‘silent e’ words, which are not appropriate for early synthetic phonics teaching.