jay sez
>Peer review sucks. The little research there is on the effectiveness of
>peer review shows that, even in the sciences, the review rating of a
>single
>manuscript is a function of who it was sent to and that the range of
>responses from reviewers is often all the way from "crap" to "publish as
>is". This is one of the dirty little secrets of both science and the
>academy generally. (Another is how few people ever read any given journal
>article after it's published.) Peer review is not a reliable basis for
>public policy or private action when anything of material importance is
>at
>stake. It takes a LONG TIME before the reliability of research
>conclusions
>is established. Much longer than the timescale of peer review.
peer review, currently, DOES suck, but it doesn't _have_ to -- i mean,
are we agreeing that we are all incapable
of acting responsibly with each other's writings? isn't this more about
the criteria of publications as a standard of 'qualification?' -
i mean, a lot of very stupid people can publish a LOT, but that doesn't
make them less stupid than others who publish less,
... the standards of the university hiring practice speak more here than
any public policy,
peer review COULD be intelligent, responsible, but because... what, too
many idiots are in the arena, we need to police ourselves?
>
>Replication is a healthy standard, especially if we add to it
>"robustness";
>that is, not just the replicability of the exact same study, but the
>attempt to replicate the conclusions by doing variants of the study: in
>other places, with other people (or rats), using more subjects, using
>more
>diverse data sources, looking at the persistence of effects over longer
>time periods, varying other background conditions, etc.
i am reminded of Walter Benjamin's essay on art in the age of mechanical
reproduction - the emphasis on reproduction,
replication, isn't this a bit too much in the hard science realm
and contradictory to social activity? isn't the very possibility of a
social "science" an oxymoron?
who is protecting this, besides those with careers invested in
substantiating the possibility?
isn't this precisely the crisis of psychology?
i might be misreading - lordess knows i leap to assumptions...
the assumption of elaborating an other's design invites the usual
quantification questions, how can this be reliable when the 'rules' are
changed and so on... you implicate science as needing pliable methods,
which is, at the same time, contradictory to precious validity.
heh. please elaborate?
>
>I think it is very healthy to direct a lot more attention to the
>interface
>between research and policy.
frankly, i'd say we need more interface between social activity,
intelligent questions, and policy, but
i'd reckon we're speaking towards the same place here. salut!
>The technocratic paradigm: research say X,
>therefore we must do Y -- dangerously elides questions of value conflict
>from policy discourse. It invites the commissioning and publication of
>research designed to favor pre-determined policies which reflect
>particular
>interests and values. It abets the unwise practice of jumping to
>premature
>policy conclusions on the basis of limited research.
the problem is that policy makers are not especially smart, and so wee
research is easier to understand than grand
projects of interdisciplinary methods and outcomes. i mean, really, we're
talking about bureaucracy, a structure that breeds ignorance in order to
function. so, how can something like research, which is supposed to be
intelligent, inform something that is supposed to be stupid, like
bureaucracy?
(this is not meant to be rhetorical, i mean - really. how do we provide a
meaningful "interface" here?
>And if we really want
>something to worry about, there is always the deeply uncomfortable
>question
>of just how viable an ideal of participatory democracy remains, if key
>policy decisions come down to making expert decisions about the
>reliability
>of facts and the robustness of conclusions.
i worry, personally, that social scientists are still trying so hard to be
legitimated with hard scientists.
i mean, oh lordess, really. when does a person MOVE ON and produce a realm
of value that is NOT
compared to a particular tradition that sanctifies rigor and replication?
the social is contradictory to rigor, replication,
method,
every one is running around improvising every day, and social scientism
persists in assuming this can be cataloged.
really. isn't there room for alternative approaches??
diane, BEGS for difference,
************************************************************************************
"Waves of hands, hesitations at street corners, someone dropping a
cigarette in a gutter - all are stories. But which is the true story? That
I do not know. Hence I keep my phrases hung like clothes in a cupboard,
waiting for someone to wear them. Thus waiting, thus speculating, making
this note and then another, I do not cling to life."
Virginia Woolf, The Waves, 1931.
(...life clings to me...)
*************************************************************************************
diane celia hodges
university of british columbia, centre for the study of curriculum and
instruction
vancouver, bc
mailing address: 46 broadview avenue, montreal, qc, H9R 3Z2
This archive was generated by hypermail 2b29 : Thu Nov 01 2001 - 01:01:24 PST