RE: [xmca] Wearing out truth

[Peter Smagorinsky <smago@uga.edu>]

http://www.facebook.com/group.php?gid=113179885388071 is the facebook page of the Journal of non-significant results:
The Journal will publish original papers regarding the behavior and cognition of human and non-human animals, reporting research that did not find statistically significant results.

Tentatively, the goals of the journal are:

1) to underscore that science progresses not just through its successes but also its failures

2) to provide a forum for lively discussion of theory, research methodology, and subsequent analysis and interpretation

3) to serve as a resource for students and teachers of psychology about the reality of conducting psychological research

One of my very first publications in the early 1980s was in the Journal of Irreproducible Results, which is a journal that parodies scientific research. http://www.jir.com/
A classic from their archives is at http://www.jir.com/geographic.html
enjoy!

From: xmca-bounces@weber.ucsd.edu [mailto:xmca-bounces@weber.ucsd.edu] On Behalf Of mike cole
Sent: Sunday, January 09, 2011 11:00 PM
To: eXtended Mind, Culture, Activity
Subject: Re: [xmca] Wearing out truth


I LOVE the idea of a journal of non-significant research, Carol!! Hell, 90+
% of what
is published in our academic journals is totally non-signifiant.

By this criterion, everyone reads the journal!!
:-)
mike

On Thu, Jan 6, 2011 at 2:46 AM, Carol Macdonald <carolmacdon@gmail.com>wrote:

> Carol's contribution:
>
> A belated read of the Lehrer article cites an unfortunate set of studies.
> Western forms of acupuncture work only on meridians.  Chines forms of
> acupuncture use far fewer needles and work on hormones and energetics as
> well.  It also takes 14 years to train in Chinese acupuncture, while our
> physiotherapists can take a six month course. (Other professions of course
> take longer.)
>
> I think this example is perhaps a read herring, but I do believe that the
> replicability effect does hold in general.  But if we look at height across
> two generations,for example, there is not a totally predictable inheritance
> because of *the regression towards the mean.* So you may not be as tall as
> your tall mother. Is it not possible that this regression effect operates
> across other statistically determined situations?
>
> If one wants to do mainline psychology studies, then controlling the
> variables is absolutely crucial. If the subjects in the second study vary
> ever so slightly than in the first, then maybe we will get a lesser effect.
> But it's this control thing that come round to bite your own tail.  In
> CHAT,
> if we were comparing similar situations, we would know that there would be
> idiosyncracies in both, and that wouldn't bother us in the least.
>
> And did you know there is a Journal of Non-Significant Research--this is
> there to tell people what they don't have to bother with, but few people
> even know of its existence.
>
> Viva CHAT.
> Carol
>
> On 5 January 2011 22:25, David H Kirshner <dkirsh@lsu.edu> wrote:
>
> > Thanks, Michiel.
> > Unsettling.
> > As we strip away the modernist vision of a stable and lawful universe,
> > the postmodern sensibility of ambiguous multiplicity will have to
> > compete with a pre-modern impulse toward alternative grand
> > narratives--religious, or otherwise.
> > David
> >
> >
> > -----Original Message-----
> > From: xmca-bounces@weber.ucsd.edu [mailto:xmca-bounces@weber.ucsd.edu]
> > On Behalf Of Eijck, M.W. van
> > Sent: Wednesday, December 29, 2010 4:22 PM
> > To: eXtended Mind, Culture, Activity
> > Subject: RE: [xmca] Wearing out truth
> >
> > Thanks, Mike, for bringing this intriguing article to our attention.
> > This is definitely a very interesting crack in the scientific worldview.
> >
> > To add some spice to your letter, David, you might want to read about
> > the current scientific controversy on changing physical constants:
> >
> > http://www.sciencenews.org/view/generic/id/62985/title/Changing_one_of_n
> > atures_constants
> >
> > Michiel
> >
> > ________________________________________
> > From: xmca-bounces@weber.ucsd.edu [xmca-bounces@weber.ucsd.edu] On
> > Behalf Of David H Kirshner [dkirsh@lsu.edu]
> > Sent: Wednesday, December 29, 2010 7:01 AM
> > To: eXtended Mind, Culture,Activity
> > Subject: RE: [xmca] Wearing out truth
> >
> > Mike,
> > I, too, found the article intriguing--enough so as to write a letter to
> > the editor about it.
> > No doubt, it's just a matter of time before mine is selected for
> > publication out of the hundreds they will undoubtedly receive on this
> > article, but in case not, here's the comment for anyone interested.
> > David
> >
> > Letter to the Editor: New Yorker
> >
> > Jonah Lehrer ("The Truth Wears Off," December 13, 2010) provides a
> > provocative array of evidence for the "decline effect," the tendency for
> > statistically sound scientific studies published in journals of
> > psychology, ecology, and medicine to fail replicability tests in a
> > gradual decline of effect sizes: "'It was as if nature gave me this
> > great result and then tried to take it back'" (p. 53). Though Lehrer
> > presents many snippets of possible explanation, he provides no synthesis
> > or evaluation of arguments. The most plausible factor is publication
> > bias. Because publishable results almost always have to show
> > statistically significant results requiring a 95% confidence level, this
> > means up to 5% of statistically significant findings could be
> > attributable to chance error. But in that case, we'd expect subsequent
> > replications to generally show no effect size, not a gradually declining
> > sequence of effect sizes. The only possible alternative explanations
> > seem to be metaphysical in nature: "'it was like the cosmos was
> > habituating to my ideas'" (p. 53).
> >
> > Pursuing the metaphysics, a more palatable explanation than a physical
> > universe actively interested in what we think would be the solipsistic
> > view that the universe we experience is a projection of our own psyche.
> > Thus the pattern of decline recorded in Lehrer's report actually would
> > index our own declining psychic investment in replicated results. If
> > this were the case, we'd expect it to hold, also, for hard science
> > results, not just results involving human response, though in a less
> > obvious and dramatic form. This could be tested empirically by doing
> > repeated micro-measurements on new physical constants as may come to be
> > discovered in physics or chemistry. A slight lessening in degree of
> > accuracy of measurement of physical constants wouldn't have aroused the
> > attention of scientists in the past. However, a pattern of such
> > declining resolution could serve to validate the solipsistic thesis that
> > the universe out there is what we make it-now wouldn't THAT be
> > troubling!
> >
> >
> > -----Original Message-----
> > From: xmca-bounces@weber.ucsd.edu [mailto:xmca-bounces@weber.ucsd.edu]
> > On Behalf Of mike cole
> > Sent: Tuesday, December 28, 2010 10:33 PM
> > To: eXtended Mind, Culture,Activity
> > Subject: [xmca] Wearing out truth
> >
> > If you have not seen these materials, gathered by a colleague, i suggest
> > you take a look. Very interesting.
> > mike
> >
> > ---------- Forwarded message ----------
> > From: Frank Kessel <kesfam@pdq.net>
> > Date: Tue, Dec 28, 2010 at 5:06 PM
> > Subject: Truth?
> > To: Frank Kessel <kesfam@pdq.net>
> >
> >
> >  FOLKS: As you may (or may not!) remember, in a message forwarding the
> > news about how Republicans plan to (try to) cut/reduce the 'soft'
> > behavioral and social sciences from the NSF budget (see below-below), I
> > provided the abstract for a recent *New Yorker* article on the
> > less-than-firm quality of at least some of our hard(er) science
> > findings.  Well, in the 2nd part of his annual "Sidney Awards" columns,
> > David Brooks talks about the same article (see below).  Better still, he
> > [rovides a link that enables all of us non-subscribers to access the
> > whole article. So here 'tis (attached). . .
> > Methinks that, along with the earlier-in-the-year article and discussion
> > re the limitations of much of WEIRD (psychological) science, this
> > presents us with, well, let's simply say a non-trivial challenge ot
> > three!  FRANK
> >
> > P.S.  In case you don't know, Brooks' "Sidney Awards go to some of the
> > best magazine essays of the year. The one-man jury is biased against
> > political essays, since politics already gets so much coverage. But the
> > jury is biased in favor of pieces that illuminate the ideas and
> > conditions undergirding political events" . . . and are named for Sidney
> > Hook.
> >
> > ------------------------------------------------------------------------
> > ------------------------------------------------------------------------
> > ------------------------------------------------------------------------
> > ---------------------
> >
> > In earlier ages, people consulted oracles. We consult studies. We rely
> > on scientific findings to guide health care decisions, policy making and
> > much else. But in an
> > essay<http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer
> > >called
> > "The Truth Wears Off" in The New Yorker, Jonah Lehrer reports on
> > something strange.
> >
> > He describes a class of antipsychotic drugs, whose effectiveness was
> > demonstrated by several large clinical trials. But in a subsequent batch
> > of studies, the therapeutic power of the drugs appeared to wane
> > precipitously.
> >
> > This is not an isolated case. "But now all sorts of well-established,
> > multiply confirmed findings have started to look increasingly
> > uncertain,"
> > Lehrer writes. "It's as if our facts were losing their truth: claims
> > that have been enshrined in textbooks are suddenly unprovable."
> >
> > The world is fluid. Bias and randomness can creep in from all
> > directions.
> > For example, between 1966 and 1995 there were 47 acupuncture studies
> > conducted in Japan, Taiwan and China, and they all found it to be an
> > effective therapy. There were 94 studies in the U.S., Sweden and
> > Britain, and only 56 percent showed benefits. The lesson is not to throw
> > out studies, but to never underestimate the complexity of the world
> > around.
> >
> > ------------------------------------------------------------------------
> > ------------------------------------------------------------------------
> > ------------------------------------------------------------------------
> > ---------------
> >
> >
> > *From:* Frank Kessel [mailto:kesfam@pdq.net]
> > *Sent:* Thursday, December 23, 2010 3:14 PM
> > *To:* 'Frank Kessel'
> > *Subject:* News for the New Year: Cutting out the social science funding
> > from NSF
> >
> >  So the battles begin  . . . again = the wacky Wisconsin Senator's
> > "Golden Fleece Award" redux . . . but, alas, probably with far greater
> > political force.
> >
> > As for the, um, certainty of the, uh, hard sciences -- See attached.
> > (Will zip over complete article as soon as one or another subscriber
> > gains
> > access!)
> >
> >
> > ------------------------------
> >
> > >Representatives Eric Cantor's and John Boehner's 2009
> > >proposal<http://republicanleader.house.gov/UploadedFiles/06-04-09_Savin
> > >gs_Proposals_For_President.pdf>to President Obama seeking to cut in
> > >half the NSF's $198 million allotted
> > for awards in behavioral and social science. "Unlike NSF's other hard
> > science programs (such as engineering and biological sciences)," Boehner
> > and Cantor wrote in 2009, "these soft science programs are often more
> > controversial and less directly related to NSF's core mission."
> >
> > *Picking on Social Science *
> >
> > December 21, 2010  Inside Higher Ed
> >
> > A bid to question the merits of federal funding for social and
> > behavioral science research may be failing to capture the public's
> > attention, even as it signals that larger and more polarizing battles
> > over science, federal policy and money could lie ahead.
> >
> >
> > On Aug. 13, Rep. Adrian Smith, (R-Nebraska) posted a clip on
> > YouTube<http://www.youtube.com/watch?v=LSYTS-nRt4o>announcing the launch
> > of YouCut <http://republicanwhip.house.gov/YouCut/Review.htm> -- an
> > attempt to get citizens to crowdsource ways to cut the federal budget.
> > In YouCut's first foray, Rep. Smith invites viewers to share their
> > impressions of grants for research that have been awarded by the
> > National Science Foundation.
> > After praising the NSF for supporting discoveries in the "hard" sciences
> > (typically math, engineering, and the physical, natural and
> > computational sciences), which have spurred economic growth, he cites
> > what he sees as two of the NSF's more suspect awards.
> >
> > "University academics received a $750,000 grant to develop computer
> > models to analyze the on-field contribution of soccer
> > players<http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0838564>
> > ,"
> > he says (it's actually a Northwestern University project led by
> > engineering and business professors to develop strategies to better
> > assemble effective teams in virtual communities). Rep. Smith also
> > mentions a $1.2 million award to model the sound of objects
> > breaking<http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0905506
> > &loc=interstitialskip>for
> > use by the video game and movie industries (the project involves Cornell
> > University computer scientists finding ways to change how sound is
> > manufactured in interactive virtual environments).
> >
> > Attacks of this nature -- which tend to target perceived intellectual
> > and cultural elites -- often gain traction during periods in which
> > Republicans control at least one house of Congress, as they are about to
> > do. This position of power enables them to hold hearings, call for votes
> > on specific projects and exert some control over the purse strings of
> > federal agencies.
> > Such attacks already have begun. Republicans led a successful effort
> > earlier this month to pressure the Smithsonian Institution to remove a
> > work of art from the National Portrait Gallery's exhibition, "Hide/Seek:
> > Difference and Desire in American
> > Portraiture,"<http://www.insidehighered.com/news/2010/12/06/katz>for
> > its perceived anti-Christian imagery.
> >
> > "We've been down this road before," said Howard Silver, executive
> > director of the Consortium of Social Science Associations, who recited a
> > litany of efforts by politicians of both parties to question science
> > research. Often cited as the originator of this type of Congressional
> > activity is Sen.
> > William Proxmire (D-Wisconsin), whose Golden Fleece awards drew
> > attention to wasteful spending -- with scientific research one of his
> > favorite targets.
> > More recently, Pat Toomey, then Republican representative and now
> > senator from Pennsylvania, tried in 2003 to block National Institutes of
> > Health-backed research on sexual
> > risk-taking<http://www.iub.edu/%7Ekinsey/about/risk-research.html<
> http://www.iub.edu/~kinsey/about/risk-research.html<http://www.iub.edu/%7Ekinsey/about/risk-research.html>
> >
> > >.
> >
> >
> > Many times, these salvos -- in which politicians pounce on
> > silly-sounding research projects, often without understanding their
> > underlying purpose -- have ended up backfiring. Proxmire once ridiculed
> > federal money spent to study the screwworm fly, but later
> > conceded<http://www.ucop.edu/pres/comments/gfleece.html>that this
> > research led to extremely effective efforts to eradicate the nasty pest
> > that devoured the flesh of
> > cattle<http://books.google.com/books?id=UGjtXW4eoNkC&pg=PA75&lpg=PA75&dq
> > =Proxmire+Screw+worm&source=bl&ots=l6rspcnZqT&sig=ZwcYSadsMzP-V5s0ACFp3h
> > DNlT0&hl=en&ei=sqcLTaXcB8OB8gbM-IWZDg&sa=X&oi=book_result&ct=result&resn
> > um=7&ved=0CDkQ6AEwBg#v=onepage&q&f=false>.
> > Mark Sanford, as a U.S. representative from South Carolina, sought to
> > freeze NSF funding. He staked much of his argument that the agency
> > mismanaged money on the fact that it supported research into
> > ATMs<http://www.newscientist.com/article/mg15921536.200-not-that-kind-of
> > -poker.html>,
> > a term he mistakenly thought referred to automatic teller machines.
> > Instead, it describes asynchronous transfer mode, a telecommunications
> > innovation that enables data, voice and video to be transmitted in one
> > data stream.
> >
> > YouCut's scrutiny of the NSF -- which Silver described as "interesting
> > political theater" -- is more explicit than past efforts in dividing the
> > physical and natural sciences on the one hand from the behavioral and
> > social sciences on the other. Silver said he was troubled by
> > Representatives Eric Cantor's and John Boehner's 2009
> > proposal<http://republicanleader.house.gov/UploadedFiles/06-04-09_Saving
> > s_Proposals_For_President.pdf>to
> > President Obama seeking to cut in half the NSF's $198 million allotted
> > for awards in behavioral and social science. "Unlike NSF's other hard
> > science programs (such as engineering and biological sciences)," Boehner
> > and Cantor wrote in 2009, "these soft science programs are often more
> > controversial and less directly related to NSF's core mission." That
> > statement was written before both men were slated to assume much more
> > power
> > -- Boehner as Speaker of the House and Cantor as Majority Leader. And
> > their interest in finding areas in the budget to cut has remained
> > undimmed.
> >
> > Rep. Smith's address on YouTube continues to distinguish between the
> > value of these branches of science. While the two examples he offers as
> > evidence of questionable research blend different scientific
> > disciplines, Smith frames NSF's most worthy work as being in engineering
> > or the physical, natural and computational sciences. He asks viewers to
> > "help us identify grants which do not support the hard sciences or which
> > you don't think are a good use of taxpayer dollars."
> >
> > Experts and advocates for science in general, and social science in
> > particular, have questioned this separation. "It's sort of an easy way
> > to make political hay," said Silver, noting that social and behavioral
> > sciences account for a small fraction of the NSF's annual awards (about
> > 3 percent of the agency's $6 billion total budget, according to the
> > agency). "Is there a business in this country that doesn't need to
> > understand human or societal behavior? This whole business of saying
> > these aren't as useful or as important as natural or physical science
> > doesn't make sense to me."
> >
> > Similarly, Barry Toiv, vice president for public affairs at the
> > Association of American Universities, said his organization and its
> > allies will need to make a forceful case that social sciences are as
> > important an investment as the the natural and physical sciences, while
> > acknowledging that no formal campaign was in the works. Toiv cited the
> > University of Michigan's surveys of consumers
> > <http://press.sca.isr.umich.edu/press/about_survey> as an example of how
> > social science research benefits public policy. Another
> > example: the research now being conducted at Massachusetts Institute of
> > Technology
> > <http://web.mit.edu/newsoffice/2010/profile-pentland-1101.html>on
> > understanding nonverbal cues. The results could help troops in
> > Afghanistan better read the body language of people with whom they
> > cannot communicate verbally, he said. "A particular project may sound
> > funny or irrelevant, but you never know where that's going to lead,"
> > said Toiv.
> >
> > To a great extent, this argument has been won within the NSF, based on
> > its commitment to fund cross-disciplinary
> > research<http://www.insidehighered.com/news/2010/08/16/nsf>,
> > which was articulated in August by the agency's assistant director for
> > the social, behavioral and economic sciences.
> >
> > Among policy-makers, it's a different story, albeit one with some new
> > technological, rhetorical, disciplinary and political wrinkles. YouCut's
> > use of crowdsourcing marks a "clever" change in approach, said Al Teich,
> > director of science and policy programs for the American Association for
> > the Advancement of Science. "This is a high-tech twist on an old story,"
> > he said. Teich and others hope that it could be fruitfully deployed by
> > scientists and their advocates to make the case for their research. "It
> > might be a useful technique to engage the public in this sort of
> > conversation," he said.
> >
> > April L. Burke, founder of the lobbying and consulting firm Lewis-Burke
> > Associates, also thought the effort to focus attention, through YouCut,
> > on the NSF could serve as a valuable opportunity for the sorts of people
> > and institutions she represents -- research universities and scientific
> > organizations. Scientists and their advocates would do well to see the
> > latest scrutiny not as a blanket condemnation of science and cause for
> > offense, but as an invitation to make the case as to why the NSF should
> > fund social science, she said. "I'd rather see us launch a positive
> > campaign and talk to members of Congress and get them comfortable with
> > social sciences and physical sciences," said Burke, "and save our angry
> > powder for when we're really under attack."
> >
> > It is not clear that the bid to draw close scrutiny to the NSF has had
> > the desired effect. While the clip was posted in August, the effort only
> > started to generate more widespread attention in the past two weeks,
> > from such outlets as
> > *Wired*<http://www.wired.com/wiredscience/2010/12/nsf-youcut-review/>and
> > *USA
> > Today*<http://www.usatoday.com/tech/science/columnist/vergano/2010-12-05
> > -politics-science_N.htm>.
> > YouTube tracking data indicate that Rep. Smith's address had generated,
> > as of press time, about 12,200 views over the past four months. Twelve
> > viewers indicated that they like the page, while 195 dislike it.
> > Responses on Twitter to Rep. Smith's address have not been particularly
> > kind, either.
> > Some have called it a "witch hunt," "catastrophically dumb," and
> > dangerous in a country with a high rate of scientific illiteracy.
> >
> > Rep. Smith's office referred to Rep. Cantor's office questions from
> > *Inside Higher Ed* on the number of awards that have been challenged.
> > Rep. Cantor's office did not respond to several calls and e-mails --
> > including a request to clarify whether *any* citizens had raised
> > objections. Maria Zacharias, a spokeswoman for the NSF, said she was not
> > aware of any other grants being called into question by members of the
> > public, though such a lack of result is not necessarily unexpected.
> >
> > Several observers thought the real significance of the YouCut episode
> > was that it offered a preview of upcoming efforts to trim federal
> > spending on NSF and other agencies back to 2008 funding levels, and of
> > looming investigations over climate science or energy policy. Teich's
> > group, the AAAS, has offered a sober but slightly hopeful
> > view<http://www.aaas.org/news/releases/2010/1201post_election_energy.sht
> > ml?sa_campaign=Internal_Ads/AAAS/AAAS_News/2010-12-01/jump_page>of
> > the shifting landscape since last month's election, and suggested that
> > consensus in science funding, climate change and energy policy could
> > still be built.
> >
> > Toiv, of the AAU, declined to see the prospect of future conflicts over
> > research in strictly partisan terms. He said political pressure over the
> > deficit was bound to place discretionary domestic spending, which is the
> > source of NSF's money, under the microscope. "There was going to be a
> > problem no matter which party won," he said. But, he added, it remains
> > important for advocates of scientific research to highlight the economic
> > argument to support future funding. "We cannot hope to have the kind of
> > economic growth we need to address the deficit issue without, over the
> > long-term, making these investments now in basic research," said Toiv.
> > "These are investments. That's what needs to be clear."
> >
> > - Dan Berrett <dan.berrett@insidehighered.com>
> > __________________________________________
> > _____
> > xmca mailing list
> > xmca@weber.ucsd.edu
> > http://dss.ucsd.edu/mailman/listinfo/xmca
> > __________________________________________
> > _____
> > xmca mailing list
> > xmca@weber.ucsd.edu
> > http://dss.ucsd.edu/mailman/listinfo/xmca
> > __________________________________________
> > _____
> > xmca mailing list
> > xmca@weber.ucsd.edu
> > http://dss.ucsd.edu/mailman/listinfo/xmca
> >
>
>
>
> --
> WORK as:
> Visiting Lecturer
> Wits School of Education
> HOME (please use these details)
> 6 Andover Road
> Westdene
> Johannesburg 2092
> +27 (0)11 673 9265   +27 (0)82 562 1050
> __________________________________________
> _____
> xmca mailing list
> xmca@weber.ucsd.edu
> http://dss.ucsd.edu/mailman/listinfo/xmca
>
__________________________________________
_____
xmca mailing list
xmca@weber.ucsd.edu
http://dss.ucsd.edu/mailman/listinfo/xmca

________________________________

No virus found in this message.
Checked by AVG - www.avg.com<http://www.avg.com>
Version: 10.0.1191 / Virus Database: 1435/3370 - Release Date: 01/09/11
__________________________________________
_____
xmca mailing list
xmca@weber.ucsd.edu
http://dss.ucsd.edu/mailman/listinfo/xmca