>In Chapter 8, Mike considers the difficulties in examining the actual
>cognitive practices of people in realistic contexts, employing the tools
>available to them for the kinds of tasks they actually use. At the same
>time he reviews the methods and results of successful investigations. The
>main attention is on tasks of symbol manipulation in the use of language
>and mathematics.
>
>What to you seem difficulties and opportunities in obtaining ecologically
>valid results concerning cognitive practices? Please be as specific and
>concrete as you can, by referring to actual accomplished investigations or
>projected thought experiments directed towards specific situations.
This is a true story about "the actual cognitive practices of people in
realistic contexts, employing the tools available to them for the kinds of
tasks they actually use". However, it is not a story about an actual
investigation designed to discover these cognitive practices nor is it a
projected thought experiment. Instead, it's a story about how three
different activities purport to represent a single practice and what
happened to a real individual whose life path was affected by this
assumption. Where I come in is that I was the teacher to whom this person
turned to TEACH him how to do behavior x in context C, which he could
already do in context A and seemed to be able to do, as far as I could
tell, in context B. Being in this role as a teacher many many times is
what makes me come to a book like Mike's with great hope and gratitude...
. . . although I can't say it's solved all my problems.
The question here is, was it really always behavior x in all three cases?
I happen to think not. I'm about ready to say that if you change the
context, you change the behavior, period.
Here is the situation:
I had a student, a 35-year old African American man, who was employed by
Amtrak. He had worked there long enough and was respected by his
co-workers and was being urged to take a certain test that would result in
a promotion to carman in the mechanical department. His co-workers and
boss believed, based on his actual work, that he could pass the test.
However, every time he took the test, he failed it. Finally they -- his
boss -- decided that this man had a reading problem and told him to go to a
community college and take a basic reading class. This is where I met him.
This man was a good student. He seemed to be able to read fine (at least
better than the typical student in that class) although since he worked the
night shift he often dozed off in class while sitting over a book or
papers. Since many kinds of expectations converged around him passing this
test I tried to figure out in more detail what was going on.
The problem, as he presented it, was that although he could do the work in
the yard, he could not pass the test about the work. I asked him to bring
me a copy of the test. He did. It was a 107-item multiple choice test with
3 pages of diagrams. The copy he brought me happened to have both the
correct answers marked on it and the wrong answers he had given.
Here is a sample question:
When workers are on, under, or between rolling equipment on a main
track, which of the following is true?
a) The equipment must be protected by derails.
b) A Blue Signal must be displayed at both ends of the equipment to
be protected.
c) All switches providing access to the main track must be lined
away and locked with an effective locking device.
d) Engines must be uncoupled from cars.
The correct answer is b; my student answered c. Obviously, a person could
get killed by doing this sort of thing wrong. You could also kill someone
else. You would not want to be working on the same crew as someone who got
this kind of thing wrong.
I then sat with him and asked him some of the questions and listened to his
oral answers and compared them with both the correct answers circled on the
test and the incorrect answers he had given last time he took it. As I
listened, I realized that there were now, in the world, three kinds of
evidence about whether he could do this work or not:
1. His actual work in the train yards. Apparently, people who worked with
him there felt that he did the work well enough to deserve a promotion.
They were the people who had decided that he had a reading problem.
2. His oral responses to the written questions on the test.These tended to
be lengthy and discursive, describing the physical environment of the yard
in great detail. They seemed to flow toward the correct answer; he never
missed it entirely, but also never hit it perfectly.
3. His written answers on the test. He got 59 out of 107 wrong.
The assumption of the test makers was that the test accurately sampled the
work and that success at the test would match and predict sucess at work.
First question: is this true? It probably is -- I think we can assume that
this test has been used enough and its content is critical enough so that
its validity has been confirmed: people who do well on the test actually do
know the material. Second question: But can there be other people, for
whom the test is NOT a valid sampling of the work, could could also do the
work well and be successful? The recommendation that my student enroll in a
reading class indicates to me that at least his boss felt that maybe the
test was not valid for everyone. His boss apparently felt that the test
ALSO measured reading ability, in addition to knowledge about the work.
(The boss did not send the student off to study the AAR Field Manual, for
example, which is cited as one of the sources of information for the test.)
So maybe the test samples both the work of a carman and ALSO the reading
level of a carman -- two different cognitive activities, right? Maybe?
Then there is his ability to orally respond to the questions when you're
sitting face to face with the person asking them, which, I'm going to
argue, is yet ANOTHER cognitive activity.....
Here is where it seems to me that I am entering a hall of mirrors from
which there is no escape. How can I know that behavior x is the same in
all three cases? Why should ANY two different activites be the same,
anyway? And if they are different, how can teaching one lead to success at
another?
In real life, people shrug their shoulders and do their best, of course.
Sometimes a cup of coffee helps, or a walk in the evening when it's cool.
Maybe I can get out of this hall of mirrors by noting that there is
probably a difference between "analysis of cognitive practice" and
"cognitive analysis of practice." Is what I'm asking for help in
"cognitive analysis of a practice" not "analysis of a cognitive practice?"
Somehow the first seems more do-able. After all, Mike's Chapter 8 is
titled "The Cognitive Analysis of Behavior in Context" -- and this should
point us toward the method of analysis rather than analysis of the
cognitive activity within the practice.
Thanks to all for this discussion --
Helena Worthen