|
> INTELLIGENCE: The mismeasure of many |
Including:
Can intelligence be measured?
What do they mean by intelligence?
Are all intelligence scales whiffy?
And, anyway, who is qualified to measure us?
Illusions of accuracy and precision
Social, cognitive and cultural factors
Is it really so stupid to hate intelligence tests?
See also:
Bell curve: t'aint no big thing…
IQ
IQ: Hereditarian theory of
IQ: Racist theory of
Mensa
Psychometric testing
______________________________________________________________________
Can intelligence be measured?
"Intelligence is the capacity to achieve whatever you would wish to achieve, given no lack of resources, opportunity or motivation."
(The Author)Try measuring that. End of story.
(Hang on a minute, Fitzroy, do I hear you say? You haven't even presented an argument for or against the measurement of intelligence. And how 'bout a couple of pithy rejoinderth and thidethwipeth at famouth pthychologithtths…
Ho hum. You, dear reader, are right. God, do I have to..? Right then, let's give it a go…)
While it is easy in retrospect to find amusement in the efforts of 19th century theorists to infer intelligence from the measurement of brains, skulls or facial features, such methods were taken seriously and employed by early biometricians including Paul Broca [1] and Sir Francis Þ Galton[2]. Such techniques were ultimately worthless, nay crap, but at least offered something solid to measure. More recent theorists have had to satisfy themselves with a hypothetical construct, the nature of which is inferred from intelligence test performance. Charles Spearman (1903) [3] attempted to provide some theoretical credibility for intelligence measurement in his development of the sophisticated statistical technique known as factor analysis in which an underlying quantity of 'general intelligence' is inferred from correlations between a person's scores on range of tasks. But the appearance that something has been measured guarantees neither the accuracy of such measurement nor even the existence of that which is ostensibly being measured. Add to this difficulty the lack of consensus on which abilities are to be considered manifestations of intelligence, and then the complex cultural, experiential and personality dynamics that may also come into play. One may question whether intelligence is measurable in a psychologically meaningful sense any more than it was in Galton's day.
What do they mean by intelligence?
Establishing terms of references should be the first task of the prospective measurer. The Oxford Companion to the Mind provides a number of early operational definitions. For Edwin Boring (1923) intelligence is:
'..whatever it is that intelligence tests measure' [4].
Such circularity is clearly circularity is clearly circularity is clearly circularity is clearly circularity is clearly unacceptable but more useful definitions differ with regard to whether they describe a measurable entity. Intelligence is alternatively:
'..the capacity to acquire capacity' (H. Woodrow) [5];
'..the power of good responses from the point of view of truth or fact' (E.L. Thorndike) [6];
Woodrow's catchall definition and its emphasis on learning ability would at least allow the inclusion of mental qualities familiar to all of us, including common sense, creative talent, wisdom, lateral thought etc. Not just the logical exercises beloved of the Þ Mensa crowd. Thorndike, by his appeal to 'truth or fact' demands the hard objectivity that is necessary for measurement, but within his definition all but semantic knowledge (knowledge of facts about the world) and procedural knowledge (how to do stuff) and logical reasoning (how to reason logically) must be excluded.
Herein lies the first difficulty of measurement: desirable as it might be to allow for a maximum scope of cognitive skills in a test, it is only in 'correct-answer' exercises that one can provide a common yardstick for testees. Neither Terman's (1916) [7] Stanford-Binet 'IQ' test nor Wechsler's (1939) [8] scale can accommodate original or creative responses.
Even within this slimmed-down, fact-and-logic model there are difficulties. Paradoxically, the greater the depth of analysis of logical processes found in later, more comprehensive theories of intelligence such as Sternberg's [9] 1986 'componential' model, which identifies 'learning', 'retrieval', 'transferral' etc. as discrete testable entities, the less intelligence hangs together as an attribute with a unitary identity. Rather like a Gestalt effect where under close examination the parts of a figure exist, but the whole does not. Not even high performance correlations between subcomponents would guarantee common causation or functional unity.
[Editor's note: could you explain that last bit again in plain English?
Well, Sternberg would say we use different aspects of our mental abilities in combination to carry out an 'intelligent' task like solve an IQ puzzle… Ok?
Ed: Ok…
Well he might be right but that fact alone doesn't mean there is an underlying 'intelligence' factor facilitating all three processes. Nor does it allow us to conclude that 'learning + retrieval + transferral = intelligence'
.Ed: Hmm… Is that what Sternberg says, then?
Maybe not in so many words, but I think the second assumption is implicit. I'll have to go and read him some more…
Ed: Perhaps you'd better.]
Are all intelligence scales whiffy?
Actually, no. I even like the approach of Howard Gardner [10] though his method of enquiry bears scant resemblance to the type of measuring scale to be discussed here.
And, anyway, who is qualified to measure us?
Many a cynic would have it that the starting assumption behind the theoretical formulations of the intelligence measurers is that they, the theorists, are brainy sods 'coz they have letters after (as well as in) their names, and if the form of measurement they contrive happens to allocate lower scores to the plebs and thickos beneath them, to the micks across the water, to their dusky-skinned cousins from sunnier climes and to even their own wives, mothers, mistresses, daughters etc, well so be it…
Because whatever their intention, twentieth century psychologists had a terminal compulsion for comparing the supposed intelligence scores of every conceivable sub-grouping of human being: by race, by gender, by age, by sexual bleedin orientation - you name it. And - hey, guess what - it is the white, western educated male who comes out tops every time, the very tribe that coincidentally the psychologists belong to. Though - like it or not - intelligence gatherers in the west have latterly had to make room at the top of the cognitive tree for Asians from countries with western-style economies and education systems.
(Hmm, has any intelligence theorist yet created a test which they themselves were incapable of performing well at..?)
And we may suppose that all too often the researchers or those who fund them have had ulterior motives. Spearman's mentor and inspiration, Karl Pearson [11] , for example, cited the supposed racial inferiority of the Russian emigrant in impassioned letters to the British government to halt their entry into the country. Both Germany and the United States used the poor intelligence test scores of the feeble-minded to justify enforced sterilisation programmes (check out Stephen Jay Gould's 'The Mismeasure of Man') [12]
The question of the supposed heritability of intelligence is discussed in the entries
Þ IQ: Hereditarian theory of and Þ IQ: Racist theory of .
Illusions of accuracy and precision
There are to date many different intelligence scales, and of mixed quality. The best-known is the
Þ IQ test.Were such tests free of their own idiosyncratic flaws, there is a problem shared by all methods that would convert an individual's mental performance to a numeric value. And that is how much intelligence are we talking about? What, if anything, do IQ scores mean in terms of available brainpower or ability to lead a useful or productive life?
One a priory textbook assumption that underlies test standardisation is that:
|
'Like many physical characteristics such as height, intelligence is considered to be normally distributed' (Taylor & Hayes, 1990, p.124) [13 ]. |
But comparisons with normally distributed physical attributes may be ultimately self-defeating. (See also
- its range would be so narrow as to make comparisons trivial, and
- its distribution so tightly clustered as to make accurate rankings of a population nigh-impossible when 'viewed from a distance', ie. measured by the indirect means of an intelligence test. Especially when you can't see the ground…
[Editor's note: You have lost me again. What is all this statistical gubbins supposed to tell us?
I am trying to say that the difference in intelligence between someone with a high IQ and someone with a low IQ has no meaning beyond the context of the IQ test. Bear with me…
Ed: Hmm, well there might be a few cuts before the final draft…]
Politicians and marketing professionals are famously fond of the 'gee-whizz' graph: a figure-massaging trick that is used to mislead the unwary voter and hapless consumer.
|
Fig 17a. A politician's 'gee-whizz' graph, illustrating the fall in unemployment since a hypothetical government came to power. |
|
Fig 17a. True fall in unemployment since hypothetical government came to power. |
The numerical values shown on a gee-whizz graph are likely to be fairly precise, but at the same time giving the honest punter an inaccurate impression of the quantities involved. The distortion lies in the scaling of the axes such that the location of zero relative to visible data is hidden.
And we are at least dealing with what is known as ratio level data in that zero has a true value (ie. no new jobs), and two million new jobs would represent exactly twice as many new jobs as one million new jobs.
But a score of 100 IQ points does not represent twice as much intelligence as a score of 50 IQ points, and zero is not merely absent from IQ score-charts, but absent for good reason. It would be psychologically meaningless: nobody outside of the Flat Earth Society has zero intelligence. And without a true zero, the value of each IQ point scored is also meaningless.
A note about 'accuracy' and 'precision': Horologists, for one, are keen to stress the difference in meaning between the terms. A timepiece may measure passing microseconds with precision but, unless set to the correct time, the device will not be accurate. An inaccurate watch will still measure time consistently (a minute has the same length this year as it did twelve months ago), and may measure with high precision. An intelligence test, by contrast, is not merely inaccurate, in that we don't know how much intelligence an IQ score is supposed to represent; it is also imprecise - unless it could be shown that question (1) in an IQ test measures, in theory and in practice, exactly the same amount of intelligence as does question (2). In practical terms, for any given overall IQ score, the testee should be equally likely to solve question (1) as he/she is to solve questions (2), (37) or (99) in that same test.
Test-setters do not even aspire to create this state of affairs as Eysenck, in 'Know your own IQ' is glad to confirm:
|
'Each test consists of a varied series of different types of problems with the easiest at the beginning and the most difficult at the end, although it should not be assumed that the difficulty levels in between have been carefully graded.' (p. 39) -p [14] |
The apparent accuracy and precision of a distribution plot or results table of intelligence scores is a deception greater than that of the gee-whizz graph. All the more so in that the illusion lacks the transparency of the politician's obvious statistical trickery.
(Many of the problems of intelligence testing are the same as those covered more fully in
Þ Psychometric testing. The dubious content of many intelligence tests is examined, somewhat glibly, in Þ IQ: Can psychologists think?)
Social, cognitive and cultural factors
Variability in intelligence, then, may be partially innate - or a number of studies suggest as much - but it is not overwhelmingly so. Test scores show a regular distribution but the range may be meaningless or trivial with regard to the 'amount' of intelligence being measured. Measurers willing to take these difficulties on board (or, perhaps, ignore them) as well as the narrow logic-orientation required to transform performance into scores, may find the real difficulties just beginning.
What social or cultural factors might destabilise a test's baseline or significantly prejudice test results for certain groups of people? The following study offers persuasive evidence that such distortions occur.
The Burakumi people are a distinct ethnic sub-group of low social status in their native Japan where they alway register significantly lower IQ scores than other Japanese. A study by Ogbu (1986) [15] found that after emigration to the United States (where in the eyes of their new host nation they were no different from any other Japanese), their mean IQ scores rose to match those of their compatriots. While it may be difficult to identify which factors caused the Burakumi's former poor performance - low self-esteem seems possible, as does lack of relevant experience - they appear to be environmental in origin.
Even when a test is standardised for specific target groups such that between-group mean scores and variances are similar to those of other populations, there will be further unknown variables operating at the individual level. Each testee will bring to the test their current mood, their level of motivation, their persistence, their hatred of logical puzzles, or perhaps their disposition for reacting positively or otherwise to the emotional arousal invoked by the test situation.
Hebb (1972) [16] identified the role of stress in cognitive impairment: the greater the emotional arousal beyond an optimal level, the greater the disruption to mental performance. 'Exam nerves' are a good example. IQ testing's claim to validity lies in its apparent power to predict academic success, but as social situations go both intelligence tests and examinations can elicit similar emotional responses. There may also be a self-perpetuating effect where those who, through poor IQ test or exam performances, come to see themselves as intellectual 'also-rans'. Like the Burakumi in Japan, perhaps, they don’t see brainy stuff as their bag. Students can lose the will to live after a couple of bad exam grades. Undergraduate suicide is fortunately rare but higher than in other sections of the population and pre-examination anxiety is frequently implicated.
It has become something of a truism to point out that many well-known geniuses were, for whatever reason, underachievers at school or university. Albert Einstein and Charles Darwin come to mind. We do not know how they would have scored on an IQ test, though I suspect their scores would not have been exceptional. Eysenck challenges this assumption:
|
'It is sometimes argued by critics of IQ test that Winston Churchill, to take but one example, was very bad at school, and very slow in acquiring academic knowledge; it is argued from this that he would have done badly on IQ tests, and that his consequent demonstration of high ability disproves the value of tests. Apart from the obvious absurdity of begging the question… …that he would have done badly on tests when no test was in fact administered, this argument breaks down because it assumes he was motivated to acquire school learning.' -p |
In which case why assume that anyone would be motivated to acquire school learning or, indeed, perform well on an IQ test?
But Eysenck is right in one respect: to assume that Churchill was of no more than average IQ, is begging the question. But, as always, the burden of proof lies with the claimant. Without evidence to the contrary, we have no reason to believe that any genius thrown up by history would have achieved a high IQ score. Eysenck, too, is begging the question to imply a link between IQ and genius. People with high IQ's tend to be good at trivial brain-teasers and cryptic crosswords but it has never been shown they have a facility for political leadership, for creating great symphonies or poetry, or for formulating revolutionary scientific theories. If anything, genius is characterised by non-standardised modes of intelligence. Geniuses are, almost by definition, people whose minds work differently from those of their contemporaries and - who knows? - differently, even, from the minds of intelligence test compilers.
(And it is no use citing Sir Clive Sinclair, inventor of the pocket calculator, as Mr I.Q. Genius. He later designed and tried to market the C4 battery car and the battery-powered 'Zike' bicycle - with equally disastrous consequences. Any low-IQ idiot would have told him both were doomed for the museum of futures past where they keep the old film footage of Tomorrow's World and those world-transforming gadgets whose time never came.)
Is it really so stupid to hate intelligence tests?
Motivation must have an important role in performance. For one thing, IQ tests are terminally unsexy and boring as crap. If a person is not inspired or nerdy enough to try hard, or give her best performance in a test, is this a sign of low intellect? While the tester may do nothing to reward the testee's performance, Social Learning theorists have demonstrated that in the absence of external rewards for a task, 'self-reinforcement' can serve as a powerful motivator (Bandura & Perloff, 1967) [17]. There is no way a tester could hope to control for individual differences in self-motivation on the part of the testee.
It should also be remembered that an intelligence test is of necessity timed; does intelligence really require fast cognitive processing? Is not a slow, systematic (rather than impulsive) approach sometimes the intelligent way to tackle real world problems? - or perhaps the more creative, lateral methods proposed by de Bono (1970) [18]. To control for personal 'styles' of intelligence would again prove difficult, if not impossible.
A picture is emerging - I hope - of intelligence as an ill-defined construct that is imprecisely measured on a meaningless scale and in a potentially stressful environment and whose measurement is prone to distortion from cultural, personality or other factors and whose longitudinal consistency is doubtful. If intelligence is measurable, then the difficulties involved in measurement are such that claims of accuracy and precision are questionable, as are projections to beyond the immediate contexts of time, place, and testing procedure. Is intelligence even made of such stuff as lends itself to measurement? In spite of Spearman's best efforts it is doubtful whether it is made of any kind of 'stuff'. But the nativist approach is right in one respect: nature has via evolution endowed humans with a power and versatility in their cognitive abilities that is unrivalled in the rest of the animal kingdom. It is thus demeaning to reduce an individual's abilities and potential to a score that may be used for comparisons with others in the way functionally inert characteristics such as height or weight may be measured.
If people really want to study intelligence, and as it manifests in the real world, might they not be better served by a non-evaluative approach such as Gardner's, or, to borrow an expression from George Kelly's 'Personal Construct' theory of personality, one might try:
|
'..to embrace a person's outlook, rather than measure his performance' [19] |
It might even be argued that - given enough contextual information - every conscious decision that a person makes is an intelligent one for their given circumstances. The common yardstick may be lost, but it would be better understood how, say, inventions or discoveries take place, orchestral masterpieces are written or how the Sistine Chapel came to acquire more than a coat of whitewash.
Boring was probably boring, but certainly wrong. IQ, not intelligence, '…is whatever it is that intelligence tests measure' . Intelligence is something else. Something qualitatively richer, more diverse in its expression and not measurable in any meaningful sense. In the light of problems outlined above, attempts to quantify it may constitute a category error of the first order.
______________________________________________________________________
References:
[1] BROCA, P. (1873) Sur la mensuration de la capacite du crane. In GOULD, S.J. (1981) The Mismeasure of Man. NY: W.W. Norton.
[2] GALTON, F. (1884) Hereditary Genius. London: Methuen.
[3] SPEARMAN, C. (1904) General Intelligence objectively determined & measured. In GOULD, S.J. (1981) The Mismeasure of Man. NY: W.W. Norton.
[4] BORING, E. (1923) Intelligence as the tests test it. In GREGORY, R. (1987) The Oxford Companion to the Mind. Oxford: OUP.
[5] WOODROW, H. (1921) Intelligence and its measurement: a symposium. In GREGORY, R. (1987) The Oxford Companion to the Mind. Oxford: OUP.
[6] THORNDIKE, E.L. (1921) Intelligence and its measurement: a symposium. In GREGORY, R. (1987) The Oxford Companion to the Mind. Oxford: OUP.
[7] TERMAN, L. (1916) The Measurement of Intelligence. In GOULD, S.J. (1981) The Mismeasure of Man. NY: W.W. Norton.
[8] WECHSLER, D. (1939) The Measurement and Appraisal of Adult Intelligence. In ATKINSON R.L., ATKINSON, R.C., SMITH, E.E., & BEM, D.J. (1990) Introduction to Psychology. New York: Harcourt Brace Jovanovich.
[9] STERNBERG, R.J. (1986) Intelligence Applied: Understanding and Increasing Your Intellectual Skills. In ATKINSON R.L., ATKINSON, R.C., SMITH, E.E., & BEM, D.J. (1990) Introduction to Psychology. New York: Harcourt Brace Jovanovich.
[10] Gardner
[11] Pearson, K. Lost reference, but believe me, I read it.
[12] GOULD, S.J. (1981) The Mismeasure of Man. NY: W.W. Norton.
[13] TAYLOR, I. & HAYES, N. (1990) Investigating Psychology. London: Longman.
[14] EYSENCK, H. (1971) The IQ argument. Race, Intelligence and Education. In EYSENCK, H. & KAMIN, L. (1985) Intelligence: the battle for the mind. London: Macmillan.
[15] OGBU, J.U. (1986) The consequences of the American caste system. In ATKINSON R.L., ATKINSON, R.C., SMITH, E.E., & BEM, D.J. (1990) Introduction to Psychology. New York: Harcourt Brace Jovanovich.
[16] HEBB, D.O. (1972) Textbook of Psychology. In ATKINSON R.L., ATKINSON, R.C., SMITH, E.E., & BEM, D.J. (1990) Introduction to Psychology. New York: Harcourt Brace Jovanovich.
[17] BANDURA, A., & PERLOFF, B. (1967) Relative efficacy of Self-Monitored and Externally Imposed reinforcement systems. In BANDURA, A. (1977) Social Learning Theory. New Jersey: Prentice Hall.
[18] DE BONO, E. (1971) Lateral Thinking. London: Ward Lock Education.
[19] KELLY, G. (1955) The Psychology of Personal Constructs. NY: W.W. Norton.