I’ve been communicating with Hugo Kerr (a retired vet) re. Dorothy Bishop’s blog posting, ’What’s in a name?’ http://deevybee.blogspot.com/2010/12/whats-in-name.html The following paragraph caused me particular problems and I hoped that Hugo, with his medical background, would be able to put into words why it made me feel uneasy. He came up trumps! Thank you, Hugo.
Prof. Dorothy Bishop wrote:
It follows from what I’ve said above, that the boundary between disability and no disability is bound to be fuzzy: most problems fall on a scale of severity, and where you put the cutoff is arbitrary. But in this regard, neurodevelopmental disability is no different from many medical conditions. For instance, if we take a condition such as high blood pressure: there are some people whose blood pressure is so high that it is causing them major symptoms, and everyone would agree they have a disease. But other people may have elevated blood pressure and doctors will be concerned that this is putting health at risk, but where you actually draw the line and decide that treatment is needed is a difficult judgement, and may depend on presence of other risk factors. It’s common to define conditions such as dyslexia or SLI in terms of statistical cutoffs: the child is identified as having the condition if a score on a reading or language test is in the bottom 16% for their age. This is essentially arbitrary, but it is at least an objective and measurable criterion. However, test scores are just one component of diagnosis: a key factor is whether or not the individual is having difficulty in coping at home, work or school
Here is what Hugo had to say:
The problem you have with the piece you sent me is that much of it is perfectly good, even common, sense. The issue is that she applies said sense across from known concepts grounded in observable and concrete realities to improbable ones (like dyslexia, or even perhaps SLI) where they tend to steadily generate consequences which get less and less credible the more work is done with, or upon, them. The opening remarks people make about dyslexia are usually apparently banal enough, but because they are applied to an unlikelihood underpinned by beliefs rather than scientific understandings they tend to grow bizarre when developed.
Many medical conditions (like not spelling particularly well!!!!) come in categories arranged along a spectrum, often alongside many others with overlapping symptomology, as you already know of course. The normal range (ie no clinical indication at all of a need to intervene) can be wide. At the potentially pathological end of the normal range we begin to think in terms of abnormality, but of course there is also a range of abnormality (from let’s wait and see to let’s do something soon, or even let’s do something right now). It’s a smooth spectrum of course in most cases and conditions – and she insists dyslexia is a neurological condition like any other, so it will have to obey this iron law of clinical subjectivity. This frequent, and very often inherent, clinical subjectivity is why diagnosis is an activity which demands such expertise but also such humility. Making a diagnosis is very often, as he admits, a “difficult judgement” and to exactly this extent it demands an expert – one whose expertise lies precisely in the sensitive appreciation of signs and the appropriate weighing and prioritising of these - and the “other factors” she invokes.
It’s interesting that she mentions SLI in this regard. I read a bit about SLI at one time. It was fascinating to observe. Researchers invariably, and subjectively, decided that x% of low achievers would be designated SLI and did so. They awarded a diagnosis on the strength of this statistical placement. A few longitudinal studies, however, followed the same schools over many years and measured language ability at intervals, using the same tools. They continued to ‘diagnose’ the bottom x% as SLI even though very many actual children moved in and out of the category! Many children moved well out of the x%, while many others fell quite a distance into it. This did not shake the researchers’ confidence in their diagnoses. It all reminded me strongly of another condition…
The section you sent is very typical of today’s writing. Having abandoned the old IQ/achievement diagnostic standard (well, in theory; older research which used it is still disgracefully held to be solid) many people now fall back on the bottom so-and-so-much percentage as their criterion, floppy and scientifically disreputable though they would probably, if pushed, themselves even admit that this is. The genetics of dyslexia wallahs in Colorado do this, for example. Olson himself even writes that this makes statistical life so much more rewarding as using the Bell curve of normal distribution permits the fanciest statistics to be deployed, and then she does. It’s completely shameless!
Here’s how I attack this fraudulence in my book
(see Hugo’s chapter on ‘Dyslexia’ http://dyslexics.org.uk/KerrCh8.pdf
'A further means of ‘diagnosis’ of dyslexia and selection of sample ‘dyslexics’ is simply to throw in the sponge, deploy the ‘bell curve’ of reading ability and define those in, say, the lowest 10% as ‘dyslexic’ (e.g. Olson 2006, Paracchini et al 2007). As you will know, the normal distribution curve, or bell curve, is the curve which can be plotted for any attribute which is normally distributed across a population (height is the usual example). The curve looks like a bell, hence the name. Reading ability is normally distributed. If you plot reading ability across the population you get the familiar bell shaped curve. It is easy to select, say, the bottom 10% of such a population from their results on reading tests and consequent place on the curve. As Paracchini et al write ‘RD [reading disability] represents the lower tail of a normal distribution of reading ability found in the general population’ (ibid. 2007 p. 59). Kate Nation (2006 p. 2) reaches the same over-extended classification when she writes about ‘… individuals who are at the low end of distribution – individuals who are reading disabled’. Olson further claims that ‘the positive consequence of the bell curve in reading research is that it allows us to apply powerful statistical methods in our genetic analysis of dyslexia and individual differences that depend on normal distributions …’ (Olson 2006 p. 3).
It may be useful, in certain limited circumstances and in certain rather broad but limited ways, to identify and analyse those in the lowest 10% of reading ability. However, it is not legitimate to claim that simply because they all find themselves in this bottom 10% they must all share any particular characteristic, let alone all suffer from the same syndrome, without further evidence that this is so. We have no evidence as to why these poor readers are in this group. We can guess, though, that there will be many and very various reasons for their poor reading. All we can properly say from contemplation of the bell curve is that they all seem to be poor readers. It is improper to claim more than this on this evidence – especially to claim that membership of the poor readers group per se indicates possession of a neurological deficit – indicates that all these people suffer from dyslexia. We cannot say this with any certainty whatsoever – the reasons for inclusion in this group will be numerous and various. These poor readers do not constitute a group which is reliably homogeneous. For this reason sophisticated statistical and/or genetic analyses and conclusions in respect of ‘dyslexia’ are not appropriate, however tempting the wonderful mathematical potentials of the bell curve and the statistical marvels of normal distribution.
A frivolous example to illustrate this general point: Suppose we set up a driving test whereby a thousand randomly selected people are asked to drive a car across rural Wales between two points 50 miles apart. We measure their performance. (Time taken, number of bumps recorded, frequency of road rage incidents etc.) The results will probably approximate to a bell curve of normal distribution of whatever we have decided to define as ‘driving ability’. Would this statistical fact mean we can consider that the worst 10% of drivers all share the same characteristics, though? Are they all poor drivers for the same reason? Of course not. Some may have been drunk, or high, others may have been partially sighted, others may have been teenage males charged with testosterone, others again may have been elderly and very cautious, some may have driven for years while some may only just have passed their test, some will only just have got off the plane from Australia, some will have been rendered hopelessly nervous by the knowledge that their driving was being tested, for some the route will be familiar while for others it will be completely novel. And on and on. There will be a plethora of reasons for their poor performance, and regarding these drivers as a homogeneous group with a single ‘syndrome’ (dysautomobilia?) will not be valid. Nor will it be particularly useful. It will not reliably reveal much of interest, either to science or to the department of transport. Our findings will not enable us to apply sophisticated analyses to make generally useful policy decisions, in fact, nor to reach any particularly valid conclusions about the drivers themselves.
I hope this answers the main point: for example the “essentially arbitrary but at least objective and measurable criterion” mentioned is a diagnostic chimera. It’s not a diagnostic criterion at all. It may (or may not, in individual cases over time & see my remarks about SLI above) be highly measurable and objective, but that ‘fact’ does not in and of itself give it any further meaning whatsoever. It is not ‘diagnostic’ of anything beyond the potential existence of a problem, or of an apparently loosely quantifiable difference. It does not, is actually perfectly unable to, tell us anything interesting about either causation or remediation. In particular, of course, it does not tell us anything neurological at all. It is possible that there is something neurological behind the finding and common to the group, but it is also possible that there absolutely isn’t.
She writes that “it is common to define conditions such as dyslexia or SLI in terms of cutoff points”. This is to take two steps where only one is justified. For example, liver disease. Patients will present as ‘unwell’. The symptoms will be vague and a little various. Some of the patients – but only some – will have liver disease. The unwellness they present with is not, itself, usually particularly diagnostic. Although perhaps we diagnose the liver disease (loosely, mainly) in terms of cutoff points – a blood parameter or two for example – we do know that liver disease exists in the real world and we have carefully, and objectively, measured the degree of clinical disease against blood parameters in many, many proven cases. We do not ‘define’ liver disease as those blood parameters. We define liver disease as a particular pathological process and degree affecting that organ which we have repeatedly and concretely demonstrated. Our diagnostic tests selectively indicate cases from the population of ‘unwell’ people who make up the tail end of today’s Bell curve of wellness, we can say, on the basis of profoundly grounded understanding of pathology and function.
We have no such background with ‘dyslexia’ – we have no objectively demonstrated syndrome, no satisfactorily demonstrated pathology (at least any pathology accepted outside the dyslexia field); we have not genuinely calibrated any objectively discovered data against its alleged signs. We have no consensus on said signs. It’s a specious taking of properly grounded science from one context into another way too uncertain for it to be validly applied. We are, in effect, diagnosing every case presenting as ‘unwell’ as liver disease because we cannot tell whether they are or they aren’t. Our would-be diagnostic tests do not reliably distinguish those which are from those which aren’t because we are so profoundly unable to describe the pathology underlying our apparent syndrome. We do not know enough about pathology or even function to devise a better test, we do not, in fact, agree on the pathology or even that there is any. As we have no consistent, known pathology, so we have no diagnostic tests. Membership of the group with weak literacy is not a diagnostic test. ‘Unwell’ is a description, not a diagnosis.
I think her final, rather despairing catch-all remark (“test scores are just one component of diagnosis: a key factor is whether or not the individual is having difficulty in coping at home, work or school”) is either an innocent giveaway indicating that the writer subconsciously knows perfectly well that the statistics are being grossly over-interpreted and misused, or a shabby attempt to imply to the reader that a plethora of very perceptive and highly diagnostic data are being collected and examined off-stage, in the wings, when you and I know there are not.