A Thought Experiment in Test Construction
A good starting point for this reexamination is to think about what sort of
activity Binet would have engaged in if he had been a member of a cultural
group vastly different from his own. As a sort of "thought experiment" let
us suppose that a "West African" Binet has taken an interest in the kinds of
knowledge and skills that a child has growing up in his part of the world
would need to master as an adult. To make the thought experiment somewhat
concrete, I will do my supposing about the tribal groups inhabiting the
interior of Liberia, principally the Kpelle people, among whom I have worked
and about whom a good deal of relevant information is available.
Following in the footsteps of his French model, our Liberian Binet would
want to make a catalogue of the kinds of activities that children are
expected to master by their parents and the village elders . People in
rural Liberia make their living by growing rice and other crops, which they
supplement with meat and fish when these scarce commodities can be obtained.
Rice farming is physically difficult work that demands considerable
knowledge and planning for its success, but as practiced by the Kpelle, it
is not a technologically sophisticated enterprise. It is carried out using
simple tools such as a machete to cut the underbrush; fire to burn the dry
bush; vines to tie together fence posts in order to keep out animals, and
slingshots to harass (Gay, 1973). Other aspects of Kpelle material culture
are relatively simple, although in every case the proper use of tools
requires a good deal of knowledge about how the tools are supposed to be
used. There is division of labor among Kpelle adults (men hunt, women do
most of the fishing; men cut the bush on the farms, women plant the seed,
children guard the crops), but far more than is true of contemporary
American, everyone pretty well knows what there is to know about adult
economic activities. There are some specialists (blacksmiths, bonesetters,
weavers) whose work is an exception to the generalization, and study of
their activities would certainly be important.
Of course, there is more to getting through life as a Kpelle than growing
rice or weaving cloth. All descriptions of the social organization of
Kpelle life stress that, as in America, knowledge of the social world is
essential to adult statue (Bellman, 1975). Kpelle people are linked by a
complex set of relations that control how much of the resources available to
the society actually get to the individual.
Faced with this situation, how should our West African Binet proceed?
Should he sample all the kinds of activities valued by adults? This
strategy is almost certainly unrealistic. Even allowing for the possibility
that aspects of technology make it reasonable to speak of the Kpelle as a
"less complex" society than our own, it is very complex indeed. No
anthropologist would claim to have achieved a really thorough description of
even one such society. Moreover, like Tylor, he would have to admit the
possibility that in some respects Kpelle society provides members with more
complex tasks than we are likely to face. Since it is unreasonable in
Liberia, as it is in the United States to think that we can come up with a
test that samples all types of Kpelle adult activities, why not follow
Binet's example and sample an important subset of those activities? From
an anthropological perspective, schools are social institutions for assuring
that adult knowledge of highly valued kinds gets transmitted to a society's
next generation (it must be transmitted, or there would be no later
generations!). While the school is not likely to be a random sample of
life's tasks, it is certainly a convenient place to sample activities that
adults consider important, activities that are complex enough to make it
unlikely that kids would learn what they need to know simply by "hanging
around."
So, our Liberian Binet might decide to search for some institutions in his
society that correspond roughly with the basic goals of schooling in ours.
Not all societies readily manifest such institutions, so that
anthropologists are led to speak of "socialization" as the broadest relevant
category. Fortunately for discussion, in the case of Liberia, he would
undoubtedly discover the existence of institutions called "bush schools" in
the Liberian English vernacular.
There are no detailed accounts of the curriculum of the bush school. The
three or four years that youngsters spend are organized by town elders who
are leaders in the secret societies that control a variety of esoteric
information. This material cannot, on pain of death, be communicated to
outsiders. However, we know enough about aspects of bush school activities
to continue our hypothetical research (Bellman, 1975; Gay, 1973); we know
that youngsters learn to farm, construct houses, track animals, shoot birds,
and carry out a variety of adult economic activities (children live apart
from their home villages in something like a scouting cap during their time
in bush school). They are also interested in the important lore of the
group. This lore is communicated not only in a variety of ceremonies, but
in stories, myths, and riddles. So, let us suppose that our West African
Binet decided to use "successful execution of bush school activities" as the
abilities he wanted to sample.
Again, like Binet, our researcher would not be able to sample all such
activities for his test, nor would he want to. He would not, for example,
want to sample activities that all children knew how to accomplish before
they got to school, nor would he want to sample activities considered so
universally accessible that everyone mastered them well before the end of
schooling. This information would not help him pick out those children who
needed extra instruction. Instead, he would seek those activities that
discriminated among children, activities that some mastered far earlier than
others, and perhaps activities that some mastered only in later life. Once
these Binet-like restrictions had been placed upon the activities selected
for study, our hypothetical researcher could begin selecting tasks on which
he could base test items.
In considering what sort of test would emerge, it is useful first to
consider what activities would be excluded as well as those included.
Cutting brush or sowing rice seed probably would not be the test; everyone
knows how to do that before he or she gets to school. Nor would anyone
spend time explicitly teaching children common vocabulary. However, there
would be explicit instruction in such tasks as constructing houses and
identifying leaves that are useful in different kinds of medicine. There
would also be some mechanism for insuring that the history of the group and
its laws and customs were taught to everyone in the form of stories and
dances. Finally, some children would be selected for specialist roles that
would require special tests (bonesetter, weaver, midwife, blacksmith,
hunter, and so on). These children would receive additional instruction.
Looking at those areas where instruction might be considered important, we
can see many candidate activities for testing. We might want to see if
children had learned all of the important leaf names for making medicine
(Bowen, 1964). Riddles are often important parts of stories and arguments,
so we could test to see how many riddles children know and how adept they
are at interpreting them (Kulah, 1977). The specialties would be a rich
source of test material, especially if we thought that rational testing of
ability to perform like adults would improve the quality of our cloth or
machetes. In short, it seems possible, in principle, to come up with test
items that could perform functions in Kpelle society similar to the way that
Binet wanted to use IQ tests.
Could we carry out such a program of research in practice? There is no
simple answer to this question, but it is useful to consider the obstacles.
For some activities such as naming leaves or remembering riddles, it should
be relatively easy to make the relevant observations because the Kpelle have
already arranged for them: several researchers have described children’s
games that embody precisely these activities (Cole, Gay, Glick, and Sharp,
1971; Lancy, 1977). We could also test people’s skills at constructing
houses, weaving designs, and forging sturdy hoes. However, from a Kpelle
point of view, tests of such skills would not be particularly interesting.
The real stuff of using one’s wits to get along in the world has been
excluded.
This point was made very explicitly by a Kpelle anthropological acquaintance
of mine who was versed in the more esoteric aspects of Kpelle secret
societies and medicine (or magic, according to American stereotypes). We
had been talking about what it means to be intelligent in Kpelle society
(the most appropriate term is translated as “clever”). “Can you be a clever
farmer?” I asked. “No,” came the reply. “You can be a hardworking farmer,
or you can be a lucky farmer, but we couldn’t say that someone is a clever
farmer. Everyone knows how to farm. We use ‘clever’ when we talk about the
way someone gets other people to help him. Some people always win
arguments. Some people know how to deal with strangers. Some people know
powerful medicine. These are the things we talk about as clever.”
In this bit of dialogue we see an emphasis on activities that require social
interaction as the arena where intelligence is an appropriate concept.
(Among the Kpelle and many other nontechnological groups, display of a good
memory for use in discussions is often considered an important component of
intelligence, Dube, 1977) This usage is quite consistent with Binet’s
analysis; it is those activities that differentiate among people in terms of
the way they manipulate information that the Kpelle, like the French, use to
mark intelligence.
However, once we reach this point, we face two important difficulties.
First, the situations that we have selected for our study of Kpelle
intelligence are exceedingly difficult to describe. Second, these contexts
are very difficult to arrange. It is not enough to know riddles, everyone
knows riddles. What is important about riddles is how they are used to get
one’s way with other people. Riddles are a resource to be used in a variety
of social interactions where people’s status’s and rights are at issue.
Consider the first difficulty. Bellman (1978) recounts an occasion when an
elder member of a secret society t old a long story about how he came to be
a high ranking shaman. He followed this (presumably autobiographical) story
with a long riddle, which was also in story form. A novice such as myself
would have no way of figuring out what part of the story was true, and I
certainly would not have responded to the riddle as if its interpretation
depended upon the autobiographical story; the two monologues appear to be
about quite different topics. Bellman succeeds in demonstrating, however,
that the riddle is closely linked to the autobiography. Not only are there
formal, structural similarities (once one understands the basic categories
of the relevant Kpelle belief systems). There is a rhetorical link as well.
The autobiographical story actually represents a bit of self-aggrandizement
by the person who told it. The man is claiming special knowledge and
special power in a convert manner. The riddle reinforces the main point of
the story (which raises the teller above his fellow shaman), giving the
story “logical” as well as “historical” validity. The fact that listeners
are constrained to agree with the riddle also gets them to agree, at least
in part, with the message of the autobiographical story.
By almost any account, this man’s autobiographical account plus riddle is a
clever bit of behavior. It is exactly the kind of thing that our West
African Binet ought to be sampling. But, at precisely this point, our
cross-cultural thought experiment in IQ testing comes apart. As I have
already pointed out, in order to construct a test Binet needed to be able to
select a large number of items. But the “item” we have just described (very
loosely) is not easily constructable. The participants in this scene were
doing social work on each other; the shaman, in particular, was attempting
to establish his preeminence using an account of his past history that would
be difficult to check up on, a riddle whose structure was designed to
reinforce his account, and his knowledge of his listener’s state of
knowledge concerning both the shaman’s past and Kpelle social structure.
This was one item; it was constructed by the subject, not the “tester.” It
is very difficult for me to imagine how to insure that a test includes one
or more items “of this type.” Furthermore, because the example’s structure
and content depend upon the special circumstances surrounding it, how could
I insure that I would be able to present the test to the subject since it
was the “subject” who did a lot of the presenting in the example I have
described?
Here the contrast with Binet’s situation is very strong. Like Binet, we
have proceeded by figuring out what sorts of activities differentiate people
according to some notion of what it means to behave intelligently. Unlike
Binet, the activities we need to sample n West Africa to accomplish this
goal lead us into domains that are systematically absent from Binet’s tests.
These domains involve interactions among people in which flexibly employed
social knowledge is of paramount importance. They are not domains of
hypothetical knowledge; rather, they always involve some real operations on
the world, operations that require a great deal of care simply to describe.
We have no good notion of how to make such activities happen in a manner
analogous to the way that teachers make vocabulary tests and multiplication
problems happen. Furthermore, even if we solved all these problems, we
would have no real theory of the psychological processes that our subject
engaged in. Such problems have not been studied by cognitive psychologists.
On both practical and theoretical grounds, then, it appears virtually
impossible to come up with a way of testing Kpelle intelligence in a manner
really equivalent to what we understand to be intelligence tests in our
society. So long as we restrict our attention to Kpelle culture, this
conclusion should not cause much consternation. After all, the idea of a
West African Binet is rather absurd; Kpelle people have managed to pass on
their culture for many years without IQ tests to help them select clever
children and give extra assistance to the dull.
This archive was generated by hypermail 2b29 : Tue Jan 11 2000 - 14:04:09 PST