[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [xmca] functions of testing

Hey, all--
This is my first weigh-in on XMCA, and as a pre-emptive caveat: I'm an educational researcher inside of a media studies program who will begin doctoral study in Learning Sciences at Indiana University in the fall. I'm new to the field, is what I'm saying, so please be kind to me.

In Massachusetts, where I live and work, there's a revived push to integrate so-called "21st century skills" into the MCAS, the standardized test that all public school students have to take in order to graduate. Those who oppose this effort--including the main newspaper here, the Boston Globe--worry that 21st century skills are too subjective to measure via standardized tests. So far, so good, right? Except implicit in this argument is that standardized tests are somehow "objective" in their current design. Add to this the fear that testing 21st c. skills will lead to a drop in MCAS scores, and we have a major problem on our hands.

Here's a clip of a recent op-ed about this in the Globe (http:// www.boston.com/bostonglobe/editorial_opinion/editorials/articles/ 2009/05/04/21st_century_skeptics/): "Massachusetts stands apart in public education precisely because it created high academic standards, developed an objective measure of student performance and progress through the MCAS test, and required a passing grade in order to graduate. Students, as a result, rank at or near the top of standardized testing not just nationally but on tough international achievement tests in math and science. Any retreat from this strategy would be a profound mistake.... [I]t is unclear why MCAS test makers and graders would concern themselves with work that is more appropriately reflected in a student's report card and cannot be measured by a quick, diagnostic test. MCAS testers should concentrate on accurately measuring math ability and reading comprehension, which surely correlate with a student's success in the workplace."

So to summarize: Massachusetts students are among the top in the nation because their achievement on standardized tests prepares them to...score well on standardized tests. It's like the iconic example of circular reasoning: The MCAS is useful because it prepares them for future learning. How do you know? Because Massachusetts students do well on other standardized tests. What prepares them to do well on those tests? Doing well on standardized tests, of course.

Then the other thing is that it's such a weird argument to make, that literacy practices like reading, writing, and doing math can be somehow isolated from the 21st-century contexts that make them meaningful. It's like asking someone if she knows how to tie her shoe, then making her prove it by writing a five-paragraph essay explaining how to do it. It's like asking someone to prove he can build a fire: But is the fire for warmth, for signaling for help, or for burning the whole house down?

Ok ok ok, thanks for listening.

Jenna McWilliams
Curriculum Specialist
Project New Media Literacies
MIT Comparative Media Studies

On May 10, 2009, at 4:53 PM, Dale Cyphert wrote:



I've long been interested in the "g" factor in intelligence tests; the notion is that when they try to control for all the appropriate variables (age, education, economic status, language, etc.) there still seems to be an unexplained factor. My sense has always been that it really measures competence with the Western rhetorical norms that are necessarily built into the tests.

The direct tests of literacy have always shown a bias, as well, for the norms of middle class behavior. Even with attempts to make the tests culture neutral, there are underlying assumptions that shape the goals and thus the outcomes.

Having jumped through all the hoops to get a doctorate, there is no question at all that I've been socialized into an institution that houses (and thereby controls) the "intellectual" capital of Western culture. As faculty in a business college, I probably have a better view than many of the complicated relationships between academia, political interests, and the business community, and the degree to which my current "tests" --grant writing, promotion and tenure binders, and so on-- must serve the interests of those with the power and money in order to GET either power or money.

Whether it's an unrealized bias of rhetorical norms, an intentional, perhaps even well-meaning, attempt to locate the "best" of a culture, or even the residual tricks of a conspiratorial Illuminati, the result is that we are all tested TOWARD something.


Dale Cyphert, PhD
Associate Professor and Interim Head
Department of Management
University of Northern Iowa
1227 W. 27th Street
Cedar Falls, IA 50614-1025

Jay Lemke wrote:
As long ago I used to do quite regularly, I'm updating the subject line of this thread again. Maybe it will continue and maybe not. But I was fascinated by Valerie's reference to Bucky Fuller and the thesis that elaborate testing, and by extension (or inclusion) the emphasis on being able to write the "right sort" of essays and other genres in academia and so many specialized fields can also serve the function of managing and controlling, dividing and conquering, really bright people. There are after all two sorts of principal threats to the ruling class. One is the great mass of working people who can stop, strike, rebel, etc. And we know of course a lot about the mechanisms of control, from hegemony to mystification, ideology, policing, etc. used in this case. But the other are the specialist elites, who are often given enough to make us feel we're doing "ok" under their system, though nowhere near what the ruling class appropriate for themselves. We are co-opted, bought out rather cheaply (by their standards), and very occasionally even promoted to positions of real power. But there must also be much less visible strategies at work, and I think that the system of academic (and later, professional, career) rewards is one of them. An illusion of local-scale meritocracy under the much bigger system of social injustice and maintenance of status quo power. And in some ways, I think, testing, even the best testing we can imagine (like my Gold Standard proposal yesterday) is a key means of this system of control. Those of us who do well on tests are even more likely to believe that this reflects our merit, our talent, our hard work -- even when maybe we doubt that those who do poorly do so because of a lack of these qualities. If the children of the oppressed do poorly for reasons having little to do with their innate talents or potential efforts, then should we not also reason that we do well for reasons equally unobvious, equally not to be attributed solely to us as individuals? We do well insofar as we are pre-tuned, pre-adapted to the needs that determine what is tested for and what is valued. Not our needs generally, nor those of the mass of people. We are selected because we are potentially useful to people who pay us, who fund us, who fund our institutions, who pay our policymakers. In some cases we fit with new needs, in some cases the traditions that define our usefulness are very old and represent long-unchanged aspects of the larger political economy and social system. I think an interesting history of testing could be written from such a point of view. Has it been? Valerie also noted the ways in which testing implements the divide- and-conquer strategy with respect to useful specialists. As a relatively small group numerically, with much less social diversity overall than the whole mass of the population, we ought to be able to more easily organize and unite, but we don't. We do well on very different measures of our usefulness, most obviously, say, between humanist scholars and scientists, and while one could point to much larger patterns of activity and discourse that split us apart, our modes of testing or of judging the value of work and productivity are still quite good guides to the history of how we have been "managed". No?
Jay Lemke
Educational Studies
University of Michigan
Ann Arbor, MI 48109
On May 10, 2009, at 11:42 AM, Valerie Wilkinson wrote:
Referring to some of the threads:
"Why don't people talk about wisdom anymore?" is definitely a rhetorical question that makes the tacit assumption that they/we don't. But wisdom, like love, is abstract until informed by examples.

I could ask the question, "Why do we shun the Platonic ideal?" I fear it wouldn't kick off much of a conversation. But is the idea of "organic" learning any more informative? It is strictly environmental, but the environment may include religious education and symbolic organizational practices which support the dominant paradigm or the people who make the rules or the people who watch out for everyone's safety.

David Kellogg said: "Here are some countervailing facts to consider, before we leap to conclusions about the malign effects of Confucianism (which, like most truly ancient cultural traditions, has an irrepressibly creative and humanist core) on dysfunctional American education." YES! and well, uh - it works if you can play the game - and there is always a dialectic going with Taoism somewhere.

It is so hard to get outside of a system you are in. And if you are in international academia, you are committed to a system in some guise that employs you or publishes your papers or creates the forum where you may share your ideas. To get talking points in that system you have to be able to talk to the talk. To talk the talk, it is best, but not requisite, to have grown up in the system.

Much of what we are talking about has been talked over in various fora - from IQ and differentiated intelligence to language and manners and then the whole cultural marginalizing process that forces some to accept a role which "native intelligence" could easily overcome - since experiential learning toward mastery is ascendant - except for the weights and burdens of various kinds laden upon the underprivileged by various social mechanisms, some of which are designed to do just that, weigh them down, keep them oppressed.

If "we" locate and export the gifted (alpha) to another level and focus the lowered tiered learning towards acceptance, satisfaction with a guarantee of "enough" - many gifted people (of the other intelligences besides articulated declarative knowledge) will spend the rest of their lives struggling to make ends meet, to pay their mortgages, take care of their kids ---

Interestingly, Bucky Fuller described the purpose of the elaborate written testing system, the complex poetry and memorization of classical texts to "manage" the more gifted in Operating Manual for Spaceship Earth. Since the "pirate captains" vanished (but did they?) there isn't any proof of his wonderfully provocative claims, but I'm pretty sure that the demand for "specialization" is one of the great causes of failure to communicate from group to group. It's crippling to have jargon barriers.

This note may seem to have gotten off the track of learning in kindergarten and the whole thing - but I believe that radical return to experiential learning from breast to bicycle to doing stuff with your friends will ground much learning experience. Of course we have to keep up with the books and specialize - but we have to do the other as well or more, or more in the beginning and always some - because experiential learning is integrative and inclusive.
(was this a rant?)
Valerie Wilkinson

On 2009.May.7, at 12:41  AM, Jay Lemke wrote:

I think that we mostly agree, Eugene, given different emphasis because of our different backgrounds.

I did think it was interesting that you noted that in totalitarian discourses the leakage across a binary division can be made to undermine basic moral principles. I suppose that there are times when one needs a way to undermine other people's, and maybe also one's own, moral certainties. But clearly doing so can also be very destructive, depending on the circumstances and the consequences.

So we have to tack between stronger binaries and weaker ones, and that takes a measure of wisdom. Why don't people talk about wisdom any > more?

As to the defense of science, of course it depends on what we want to mean by science or scientific. If it is just systematically gathered empirical information, then I think we always have to take it into account, but not necessarily be ruled by it. Realities exist, but they can also change and be changed. If it means some particular way of doing research, then I am less favorable, and more Feyerabendian. If it means honestly trying to examine alternative interpretations and proposals, then count me in! If it is defending a particular current scientific theory, say neo-Darwinian evolutionary theory, then I have to look carefully at a wide range of circumstances to make my choice.

"Complex process of mutually informing" sounds just right to me!


Jay Lemke
Educational Studies
University of Michigan
Ann Arbor, MI 48109

xmca mailing list