Apologies for the serious lag in discussing CP. Chapters 8 and 9 have left
me with a lot to consider in writing some work I did several years ago.
Ecological validity hits hard at the research being conducted about learning
in and out of schools, and the tools/technology that reflect these activity
systems.
Mikes crisis with Peg Griffin and Bud Mehan in the reading instruction
teaching experiment seemed similar to one in which Nancy Roberts and I were
exploring interventions and computer modeling tools in a seventh grade
classroom. We STARTED after school, yet the routines and expectations for
participation drove the system to teacher centered instruction replete with
textbook examples of IRF. It is not that the teacher centered participation
was precluding learning, it was that we were looking for other processes to
emerge. We tried to coach the teacher, we even tried modeling
student-centered interactions, but the system of activity suffered from ch
inertia. Students wanted to be led in IRF lock-step fashion.
In desperation we took students out of school completely, did not include the
teacher, and after some fits and starts, created with the students a
qualitatively different system of interaction. We eventually helped the
students to engage in model evaluation processes previously observed in
problem solving behaviour of expert scientists (Clement), but distributed
through students and adults in the group.
The study becomes an existence proof of students engaging in rich modeling
processes, while displaying a naive epistemology of models according to a
study by Grosslight or an 'engineering' style of experimentation according to
Schauble. (Interesting enough the computer models were not applied nearly as
much as causal diagrams the students were taught to create.) Like Mike's
work with Marilyn, we observed goals emerging and morphing, artifacts
rejected and then appropriated and motive as well as understanding being
negotiated to a more or less common form. Well, so what?
Looking at the moves the adults made in contrast to those typically occuring
in schools refreshes one's perspective like a full pail of cold water.
Similar to Helena's study, we find subjects' participation in tasks, and what
we might traditionally claim that they 'know', to be quite dependent upon
the system of activity in which they participate. For example, clinical
interviews aimed at revealing their knowledge without interference by the
researcher provides a narrow lens for how well students can perform in
clinical interviews. Similarly, the teaching experiment we conducted at bbn
has little to say about what is possible in schools, although it might, like
so many other studies, have something to say about what is impossible in
traditional schooling.
If viewed as trying to understand how modeling can occur in school science,
then our study suffers from being being grossly invalid. If viewed as what
is possible for technology to aid learning in a premium context, then there
are some significant projections for the design of the technology and the
system of activity. But this now leads to where I am struggling....
Bill
--------------------------------------
Date: 7/11/97 6:03 PM
To: Bill Barowy
From: xmca who-is-at weber.ucsd.edu
Chuck wrote:
>In Chapter 8, Mike considers the difficulties in examining the actual
>cognitive practices of people in realistic contexts, employing the tools
>available to them for the kinds of tasks they actually use. At the same
>time he reviews the methods and results of successful investigations. The
>main attention is on tasks of symbol manipulation in the use of language
>and mathematics.
>
>What to you seem difficulties and opportunities in obtaining ecologically
>valid results concerning cognitive practices? Please be as specific and
>concrete as you can, by referring to actual accomplished investigations or
>projected thought experiments directed towards specific situations.
This is a true story about "the actual cognitive practices of people in
realistic contexts, employing the tools available to them for the kinds of
tasks they actually use". However, it is not a story about an actual
investigation designed to discover these cognitive practices nor is it a
projected thought experiment. Instead, it's a story about how three
different activities purport to represent a single practice and what
happened to a real individual whose life path was affected by this
assumption. Where I come in is that I was the teacher to whom this person
turned to TEACH him how to do behavior x in context C, which he could
already do in context A and seemed to be able to do, as far as I could
tell, in context B. Being in this role as a teacher many many times is
what makes me come to a book like Mike's with great hope and gratitude...
. . . although I can't say it's solved all my problems.
The question here is, was it really always behavior x in all three cases?
I happen to think not. I'm about ready to say that if you change the
context, you change the behavior, period.
Here is the situation:
I had a student, a 35-year old African American man, who was employed by
Amtrak. He had worked there long enough and was respected by his
co-workers and was being urged to take a certain test that would result in
a promotion to carman in the mechanical department. His co-workers and
boss believed, based on his actual work, that he could pass the test.
However, every time he took the test, he failed it. Finally they -- his
boss -- decided that this man had a reading problem and told him to go to a
community college and take a basic reading class. This is where I met him.
This man was a good student. He seemed to be able to read fine (at least
better than the typical student in that class) although since he worked the
night shift he often dozed off in class while sitting over a book or
papers. Since many kinds of expectations converged around him passing this
test I tried to figure out in more detail what was going on.
The problem, as he presented it, was that although he could do the work in
the yard, he could not pass the test about the work. I asked him to bring
me a copy of the test. He did. It was a 107-item multiple choice test with
3 pages of diagrams. The copy he brought me happened to have both the
correct answers marked on it and the wrong answers he had given.
Here is a sample question:
When workers are on, under, or between rolling equipment on a main
track, which of the following is true?
a) The equipment must be protected by derails.
b) A Blue Signal must be displayed at both ends of the equipment to
be protected.
c) All switches providing access to the main track must be lined
away and locked with an effective locking device.
d) Engines must be uncoupled from cars.
The correct answer is b; my student answered c. Obviously, a person could
get killed by doing this sort of thing wrong. You could also kill someone
else. You would not want to be working on the same crew as someone who got
this kind of thing wrong.
I then sat with him and asked him some of the questions and listened to his
oral answers and compared them with both the correct answers circled on the
test and the incorrect answers he had given last time he took it. As I
listened, I realized that there were now, in the world, three kinds of
evidence about whether he could do this work or not:
1. His actual work in the train yards. Apparently, people who worked with
him there felt that he did the work well enough to deserve a promotion.
They were the people who had decided that he had a reading problem.
2. His oral responses to the written questions on the test.These tended to
be lengthy and discursive, describing the physical environment of the yard
in great detail. They seemed to flow toward the correct answer; he never
missed it entirely, but also never hit it perfectly.
3. His written answers on the test. He got 59 out of 107 wrong.
The assumption of the test makers was that the test accurately sampled the
work and that success at the test would match and predict sucess at work.
First question: is this true? It probably is -- I think we can assume that
this test has been used enough and its content is critical enough so that
its validity has been confirmed: people who do well on the test actually do
know the material. Second question: But can there be other people, for
whom the test is NOT a valid sampling of the work, could could also do the
work well and be successful? The recommendation that my student enroll in a
reading class indicates to me that at least his boss felt that maybe the
test was not valid for everyone. His boss apparently felt that the test
ALSO measured reading ability, in addition to knowledge about the work.
(The boss did not send the student off to study the AAR Field Manual, for
example, which is cited as one of the sources of information for the test.)
So maybe the test samples both the work of a carman and ALSO the reading
level of a carman -- two different cognitive activities, right? Maybe?
Then there is his ability to orally respond to the questions when you're
sitting face to face with the person asking them, which, I'm going to
argue, is yet ANOTHER cognitive activity.....
Here is where it seems to me that I am entering a hall of mirrors from
which there is no escape. How can I know that behavior x is the same in
all three cases? Why should ANY two different activites be the same,
anyway? And if they are different, how can teaching one lead to success at
another?
In real life, people shrug their shoulders and do their best, of course.
Sometimes a cup of coffee helps, or a walk in the evening when it's cool.
Maybe I can get out of this hall of mirrors by noting that there is
probably a difference between "analysis of cognitive practice" and
"cognitive analysis of practice." Is what I'm asking for help in
"cognitive analysis of a practice" not "analysis of a cognitive practice?"
Somehow the first seems more do-able. After all, Mike's Chapter 8 is
titled "The Cognitive Analysis of Behavior in Context" -- and this should
point us toward the method of analysis rather than analysis of the
cognitive activity within the practice.
Thanks to all for this discussion --
Helena Worthen
------------------ RFC822 Header Follows ------------------
Received: by qm.terc.edu with SMTP;11 Jul 1997 18:00:08 U
Received: from weber.ucsd.edu by is.TERC.EDU (5.x/SMI-SVR4)
id AA11935; Fri, 11 Jul 1997 17:56:05 -0400
Received: (from procmail who-is-at localhost) by weber.ucsd.edu (8.8.3/8.8.3) id
MAA05940; Fri, 11 Jul 1997 12:44:16 -0700 (PDT)
Resent-Date: Fri, 11 Jul 1997 12:44:16 -0700 (PDT)
Date: Fri, 11 Jul 1997 14:44:06 -0500
Message-Id: <v01520d02afebc37d35d6 who-is-at [199.120.107.204]>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
To: xmca who-is-at weber.ucsd.edu
From: worthen who-is-at soli.inav.net (Helena Worthen)
Subject: CP8
Resent-Message-Id: <"Ma58V.A.vcB.P0oxz" who-is-at weber>
Resent-From: xmca who-is-at weber.ucsd.edu
Reply-To: xmca who-is-at weber.ucsd.edu
X-Mailing-List: <xmca who-is-at weber.ucsd.edu> archive/latest/4155
X-Loop: xmca who-is-at weber.ucsd.edu
Precedence: list
Resent-Sender: xmca-request who-is-at weber.ucsd.edu