Re: [xmca] Wearing out truth

[mike cole <lchcmike@gmail.com>]

I LOVE the idea of a journal of non-significant research, Carol!! Hell, 90+
% of what
is published in our academic journals is totally non-signifiant.

By this criterion, everyone reads the journal!!
:-)
mike

On Thu, Jan 6, 2011 at 2:46 AM, Carol Macdonald <carolmacdon@gmail.com>wrote:

> Carol's contribution:
>
> A belated read of the Lehrer article cites an unfortunate set of studies.
> Western forms of acupuncture work only on meridians.  Chines forms of
> acupuncture use far fewer needles and work on hormones and energetics as
> well.  It also takes 14 years to train in Chinese acupuncture, while our
> physiotherapists can take a six month course. (Other professions of course
> take longer.)
>
> I think this example is perhaps a read herring, but I do believe that the
> replicability effect does hold in general.  But if we look at height across
> two generations,for example, there is not a totally predictable inheritance
> because of *the regression towards the mean.* So you may not be as tall as
> your tall mother. Is it not possible that this regression effect operates
> across other statistically determined situations?
>
> If one wants to do mainline psychology studies, then controlling the
> variables is absolutely crucial. If the subjects in the second study vary
> ever so slightly than in the first, then maybe we will get a lesser effect.
> But it's this control thing that come round to bite your own tail.  In
> CHAT,
> if we were comparing similar situations, we would know that there would be
> idiosyncracies in both, and that wouldn't bother us in the least.
>
> And did you know there is a Journal of Non-Significant Research--this is
> there to tell people what they don't have to bother with, but few people
> even know of its existence.
>
> Viva CHAT.
> Carol
>
> On 5 January 2011 22:25, David H Kirshner <dkirsh@lsu.edu> wrote:
>
> > Thanks, Michiel.
> > Unsettling.
> > As we strip away the modernist vision of a stable and lawful universe,
> > the postmodern sensibility of ambiguous multiplicity will have to
> > compete with a pre-modern impulse toward alternative grand
> > narratives--religious, or otherwise.
> > David
> >
> >
> > -----Original Message-----
> > From: xmca-bounces@weber.ucsd.edu [mailto:xmca-bounces@weber.ucsd.edu]
> > On Behalf Of Eijck, M.W. van
> > Sent: Wednesday, December 29, 2010 4:22 PM
> > To: eXtended Mind, Culture, Activity
> > Subject: RE: [xmca] Wearing out truth
> >
> > Thanks, Mike, for bringing this intriguing article to our attention.
> > This is definitely a very interesting crack in the scientific worldview.
> >
> > To add some spice to your letter, David, you might want to read about
> > the current scientific controversy on changing physical constants:
> >
> > http://www.sciencenews.org/view/generic/id/62985/title/Changing_one_of_n
> > atures_constants
> >
> > Michiel
> >
> > ________________________________________
> > From: xmca-bounces@weber.ucsd.edu [xmca-bounces@weber.ucsd.edu] On
> > Behalf Of David H Kirshner [dkirsh@lsu.edu]
> > Sent: Wednesday, December 29, 2010 7:01 AM
> > To: eXtended Mind, Culture,Activity
> > Subject: RE: [xmca] Wearing out truth
> >
> > Mike,
> > I, too, found the article intriguing--enough so as to write a letter to
> > the editor about it.
> > No doubt, it's just a matter of time before mine is selected for
> > publication out of the hundreds they will undoubtedly receive on this
> > article, but in case not, here's the comment for anyone interested.
> > David
> >
> > Letter to the Editor: New Yorker
> >
> > Jonah Lehrer ("The Truth Wears Off," December 13, 2010) provides a
> > provocative array of evidence for the "decline effect," the tendency for
> > statistically sound scientific studies published in journals of
> > psychology, ecology, and medicine to fail replicability tests in a
> > gradual decline of effect sizes: "'It was as if nature gave me this
> > great result and then tried to take it back'" (p. 53). Though Lehrer
> > presents many snippets of possible explanation, he provides no synthesis
> > or evaluation of arguments. The most plausible factor is publication
> > bias. Because publishable results almost always have to show
> > statistically significant results requiring a 95% confidence level, this
> > means up to 5% of statistically significant findings could be
> > attributable to chance error. But in that case, we'd expect subsequent
> > replications to generally show no effect size, not a gradually declining
> > sequence of effect sizes. The only possible alternative explanations
> > seem to be metaphysical in nature: "'it was like the cosmos was
> > habituating to my ideas'" (p. 53).
> >
> > Pursuing the metaphysics, a more palatable explanation than a physical
> > universe actively interested in what we think would be the solipsistic
> > view that the universe we experience is a projection of our own psyche.
> > Thus the pattern of decline recorded in Lehrer's report actually would
> > index our own declining psychic investment in replicated results. If
> > this were the case, we'd expect it to hold, also, for hard science
> > results, not just results involving human response, though in a less
> > obvious and dramatic form. This could be tested empirically by doing
> > repeated micro-measurements on new physical constants as may come to be
> > discovered in physics or chemistry. A slight lessening in degree of
> > accuracy of measurement of physical constants wouldn't have aroused the
> > attention of scientists in the past. However, a pattern of such
> > declining resolution could serve to validate the solipsistic thesis that
> > the universe out there is what we make it-now wouldn't THAT be
> > troubling!
> >
> >
> > -----Original Message-----
> > From: xmca-bounces@weber.ucsd.edu [mailto:xmca-bounces@weber.ucsd.edu]
> > On Behalf Of mike cole
> > Sent: Tuesday, December 28, 2010 10:33 PM
> > To: eXtended Mind, Culture,Activity
> > Subject: [xmca] Wearing out truth
> >
> > If you have not seen these materials, gathered by a colleague, i suggest
> > you take a look. Very interesting.
> > mike
> >
> > ---------- Forwarded message ----------
> > From: Frank Kessel <kesfam@pdq.net>
> > Date: Tue, Dec 28, 2010 at 5:06 PM
> > Subject: Truth?
> > To: Frank Kessel <kesfam@pdq.net>
> >
> >
> >  FOLKS: As you may (or may not!) remember, in a message forwarding the
> > news about how Republicans plan to (try to) cut/reduce the 'soft'
> > behavioral and social sciences from the NSF budget (see below-below), I
> > provided the abstract for a recent *New Yorker* article on the
> > less-than-firm quality of at least some of our hard(er) science
> > findings.  Well, in the 2nd part of his annual "Sidney Awards" columns,
> > David Brooks talks about the same article (see below).  Better still, he
> > [rovides a link that enables all of us non-subscribers to access the
> > whole article. So here 'tis (attached). . .
> > Methinks that, along with the earlier-in-the-year article and discussion
> > re the limitations of much of WEIRD (psychological) science, this
> > presents us with, well, let's simply say a non-trivial challenge ot
> > three!  FRANK
> >
> > P.S.  In case you don't know, Brooks' "Sidney Awards go to some of the
> > best magazine essays of the year. The one-man jury is biased against
> > political essays, since politics already gets so much coverage. But the
> > jury is biased in favor of pieces that illuminate the ideas and
> > conditions undergirding political events" . . . and are named for Sidney
> > Hook.
> >
> > ------------------------------------------------------------------------
> > ------------------------------------------------------------------------
> > ------------------------------------------------------------------------
> > ---------------------
> >
> > In earlier ages, people consulted oracles. We consult studies. We rely
> > on scientific findings to guide health care decisions, policy making and
> > much else. But in an
> > essay<http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer
> > >called
> > "The Truth Wears Off" in The New Yorker, Jonah Lehrer reports on
> > something strange.
> >
> > He describes a class of antipsychotic drugs, whose effectiveness was
> > demonstrated by several large clinical trials. But in a subsequent batch
> > of studies, the therapeutic power of the drugs appeared to wane
> > precipitously.
> >
> > This is not an isolated case. "But now all sorts of well-established,
> > multiply confirmed findings have started to look increasingly
> > uncertain,"
> > Lehrer writes. "It's as if our facts were losing their truth: claims
> > that have been enshrined in textbooks are suddenly unprovable."
> >
> > The world is fluid. Bias and randomness can creep in from all
> > directions.
> > For example, between 1966 and 1995 there were 47 acupuncture studies
> > conducted in Japan, Taiwan and China, and they all found it to be an
> > effective therapy. There were 94 studies in the U.S., Sweden and
> > Britain, and only 56 percent showed benefits. The lesson is not to throw
> > out studies, but to never underestimate the complexity of the world
> > around.
> >
> > ------------------------------------------------------------------------
> > ------------------------------------------------------------------------
> > ------------------------------------------------------------------------
> > ---------------
> >
> >
> > *From:* Frank Kessel [mailto:kesfam@pdq.net]
> > *Sent:* Thursday, December 23, 2010 3:14 PM
> > *To:* 'Frank Kessel'
> > *Subject:* News for the New Year: Cutting out the social science funding
> > from NSF
> >
> >  So the battles begin  . . . again = the wacky Wisconsin Senator's
> > "Golden Fleece Award" redux . . . but, alas, probably with far greater
> > political force.
> >
> > As for the, um, certainty of the, uh, hard sciences -- See attached.
> > (Will zip over complete article as soon as one or another subscriber
> > gains
> > access!)
> >
> >
> > ------------------------------
> >
> > >Representatives Eric Cantor's and John Boehner's 2009
> > >proposal<http://republicanleader.house.gov/UploadedFiles/06-04-09_Savin
> > >gs_Proposals_For_President.pdf>to President Obama seeking to cut in
> > >half the NSF's $198 million allotted
> > for awards in behavioral and social science. "Unlike NSF's other hard
> > science programs (such as engineering and biological sciences)," Boehner
> > and Cantor wrote in 2009, "these soft science programs are often more
> > controversial and less directly related to NSF's core mission."
> >
> > *Picking on Social Science *
> >
> > December 21, 2010  Inside Higher Ed
> >
> > A bid to question the merits of federal funding for social and
> > behavioral science research may be failing to capture the public's
> > attention, even as it signals that larger and more polarizing battles
> > over science, federal policy and money could lie ahead.
> >
> >
> > On Aug. 13, Rep. Adrian Smith, (R-Nebraska) posted a clip on
> > YouTube<http://www.youtube.com/watch?v=LSYTS-nRt4o>announcing the launch
> > of YouCut <http://republicanwhip.house.gov/YouCut/Review.htm> -- an
> > attempt to get citizens to crowdsource ways to cut the federal budget.
> > In YouCut's first foray, Rep. Smith invites viewers to share their
> > impressions of grants for research that have been awarded by the
> > National Science Foundation.
> > After praising the NSF for supporting discoveries in the "hard" sciences
> > (typically math, engineering, and the physical, natural and
> > computational sciences), which have spurred economic growth, he cites
> > what he sees as two of the NSF's more suspect awards.
> >
> > "University academics received a $750,000 grant to develop computer
> > models to analyze the on-field contribution of soccer
> > players<http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0838564>
> > ,"
> > he says (it's actually a Northwestern University project led by
> > engineering and business professors to develop strategies to better
> > assemble effective teams in virtual communities). Rep. Smith also
> > mentions a $1.2 million award to model the sound of objects
> > breaking<http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0905506
> > &loc=interstitialskip>for
> > use by the video game and movie industries (the project involves Cornell
> > University computer scientists finding ways to change how sound is
> > manufactured in interactive virtual environments).
> >
> > Attacks of this nature -- which tend to target perceived intellectual
> > and cultural elites -- often gain traction during periods in which
> > Republicans control at least one house of Congress, as they are about to
> > do. This position of power enables them to hold hearings, call for votes
> > on specific projects and exert some control over the purse strings of
> > federal agencies.
> > Such attacks already have begun. Republicans led a successful effort
> > earlier this month to pressure the Smithsonian Institution to remove a
> > work of art from the National Portrait Gallery's exhibition, "Hide/Seek:
> > Difference and Desire in American
> > Portraiture,"<http://www.insidehighered.com/news/2010/12/06/katz>for
> > its perceived anti-Christian imagery.
> >
> > "We've been down this road before," said Howard Silver, executive
> > director of the Consortium of Social Science Associations, who recited a
> > litany of efforts by politicians of both parties to question science
> > research. Often cited as the originator of this type of Congressional
> > activity is Sen.
> > William Proxmire (D-Wisconsin), whose Golden Fleece awards drew
> > attention to wasteful spending -- with scientific research one of his
> > favorite targets.
> > More recently, Pat Toomey, then Republican representative and now
> > senator from Pennsylvania, tried in 2003 to block National Institutes of
> > Health-backed research on sexual
> > risk-taking<http://www.iub.edu/%7Ekinsey/about/risk-research.html<
> http://www.iub.edu/~kinsey/about/risk-research.html<http://www.iub.edu/%7Ekinsey/about/risk-research.html>
> >
> > >.
> >
> >
> > Many times, these salvos -- in which politicians pounce on
> > silly-sounding research projects, often without understanding their
> > underlying purpose -- have ended up backfiring. Proxmire once ridiculed
> > federal money spent to study the screwworm fly, but later
> > conceded<http://www.ucop.edu/pres/comments/gfleece.html>that this
> > research led to extremely effective efforts to eradicate the nasty pest
> > that devoured the flesh of
> > cattle<http://books.google.com/books?id=UGjtXW4eoNkC&pg=PA75&lpg=PA75&dq
> > =Proxmire+Screw+worm&source=bl&ots=l6rspcnZqT&sig=ZwcYSadsMzP-V5s0ACFp3h
> > DNlT0&hl=en&ei=sqcLTaXcB8OB8gbM-IWZDg&sa=X&oi=book_result&ct=result&resn
> > um=7&ved=0CDkQ6AEwBg#v=onepage&q&f=false>.
> > Mark Sanford, as a U.S. representative from South Carolina, sought to
> > freeze NSF funding. He staked much of his argument that the agency
> > mismanaged money on the fact that it supported research into
> > ATMs<http://www.newscientist.com/article/mg15921536.200-not-that-kind-of
> > -poker.html>,
> > a term he mistakenly thought referred to automatic teller machines.
> > Instead, it describes asynchronous transfer mode, a telecommunications
> > innovation that enables data, voice and video to be transmitted in one
> > data stream.
> >
> > YouCut's scrutiny of the NSF -- which Silver described as "interesting
> > political theater" -- is more explicit than past efforts in dividing the
> > physical and natural sciences on the one hand from the behavioral and
> > social sciences on the other. Silver said he was troubled by
> > Representatives Eric Cantor's and John Boehner's 2009
> > proposal<http://republicanleader.house.gov/UploadedFiles/06-04-09_Saving
> > s_Proposals_For_President.pdf>to
> > President Obama seeking to cut in half the NSF's $198 million allotted
> > for awards in behavioral and social science. "Unlike NSF's other hard
> > science programs (such as engineering and biological sciences)," Boehner
> > and Cantor wrote in 2009, "these soft science programs are often more
> > controversial and less directly related to NSF's core mission." That
> > statement was written before both men were slated to assume much more
> > power
> > -- Boehner as Speaker of the House and Cantor as Majority Leader. And
> > their interest in finding areas in the budget to cut has remained
> > undimmed.
> >
> > Rep. Smith's address on YouTube continues to distinguish between the
> > value of these branches of science. While the two examples he offers as
> > evidence of questionable research blend different scientific
> > disciplines, Smith frames NSF's most worthy work as being in engineering
> > or the physical, natural and computational sciences. He asks viewers to
> > "help us identify grants which do not support the hard sciences or which
> > you don't think are a good use of taxpayer dollars."
> >
> > Experts and advocates for science in general, and social science in
> > particular, have questioned this separation. "It's sort of an easy way
> > to make political hay," said Silver, noting that social and behavioral
> > sciences account for a small fraction of the NSF's annual awards (about
> > 3 percent of the agency's $6 billion total budget, according to the
> > agency). "Is there a business in this country that doesn't need to
> > understand human or societal behavior? This whole business of saying
> > these aren't as useful or as important as natural or physical science
> > doesn't make sense to me."
> >
> > Similarly, Barry Toiv, vice president for public affairs at the
> > Association of American Universities, said his organization and its
> > allies will need to make a forceful case that social sciences are as
> > important an investment as the the natural and physical sciences, while
> > acknowledging that no formal campaign was in the works. Toiv cited the
> > University of Michigan's surveys of consumers
> > <http://press.sca.isr.umich.edu/press/about_survey> as an example of how
> > social science research benefits public policy. Another
> > example: the research now being conducted at Massachusetts Institute of
> > Technology
> > <http://web.mit.edu/newsoffice/2010/profile-pentland-1101.html>on
> > understanding nonverbal cues. The results could help troops in
> > Afghanistan better read the body language of people with whom they
> > cannot communicate verbally, he said. "A particular project may sound
> > funny or irrelevant, but you never know where that's going to lead,"
> > said Toiv.
> >
> > To a great extent, this argument has been won within the NSF, based on
> > its commitment to fund cross-disciplinary
> > research<http://www.insidehighered.com/news/2010/08/16/nsf>,
> > which was articulated in August by the agency's assistant director for
> > the social, behavioral and economic sciences.
> >
> > Among policy-makers, it's a different story, albeit one with some new
> > technological, rhetorical, disciplinary and political wrinkles. YouCut's
> > use of crowdsourcing marks a "clever" change in approach, said Al Teich,
> > director of science and policy programs for the American Association for
> > the Advancement of Science. "This is a high-tech twist on an old story,"
> > he said. Teich and others hope that it could be fruitfully deployed by
> > scientists and their advocates to make the case for their research. "It
> > might be a useful technique to engage the public in this sort of
> > conversation," he said.
> >
> > April L. Burke, founder of the lobbying and consulting firm Lewis-Burke
> > Associates, also thought the effort to focus attention, through YouCut,
> > on the NSF could serve as a valuable opportunity for the sorts of people
> > and institutions she represents -- research universities and scientific
> > organizations. Scientists and their advocates would do well to see the
> > latest scrutiny not as a blanket condemnation of science and cause for
> > offense, but as an invitation to make the case as to why the NSF should
> > fund social science, she said. "I'd rather see us launch a positive
> > campaign and talk to members of Congress and get them comfortable with
> > social sciences and physical sciences," said Burke, "and save our angry
> > powder for when we're really under attack."
> >
> > It is not clear that the bid to draw close scrutiny to the NSF has had
> > the desired effect. While the clip was posted in August, the effort only
> > started to generate more widespread attention in the past two weeks,
> > from such outlets as
> > *Wired*<http://www.wired.com/wiredscience/2010/12/nsf-youcut-review/>and
> > *USA
> > Today*<http://www.usatoday.com/tech/science/columnist/vergano/2010-12-05
> > -politics-science_N.htm>.
> > YouTube tracking data indicate that Rep. Smith's address had generated,
> > as of press time, about 12,200 views over the past four months. Twelve
> > viewers indicated that they like the page, while 195 dislike it.
> > Responses on Twitter to Rep. Smith's address have not been particularly
> > kind, either.
> > Some have called it a "witch hunt," "catastrophically dumb," and
> > dangerous in a country with a high rate of scientific illiteracy.
> >
> > Rep. Smith's office referred to Rep. Cantor's office questions from
> > *Inside Higher Ed* on the number of awards that have been challenged.
> > Rep. Cantor's office did not respond to several calls and e-mails --
> > including a request to clarify whether *any* citizens had raised
> > objections. Maria Zacharias, a spokeswoman for the NSF, said she was not
> > aware of any other grants being called into question by members of the
> > public, though such a lack of result is not necessarily unexpected.
> >
> > Several observers thought the real significance of the YouCut episode
> > was that it offered a preview of upcoming efforts to trim federal
> > spending on NSF and other agencies back to 2008 funding levels, and of
> > looming investigations over climate science or energy policy. Teich's
> > group, the AAAS, has offered a sober but slightly hopeful
> > view<http://www.aaas.org/news/releases/2010/1201post_election_energy.sht
> > ml?sa_campaign=Internal_Ads/AAAS/AAAS_News/2010-12-01/jump_page>of
> > the shifting landscape since last month's election, and suggested that
> > consensus in science funding, climate change and energy policy could
> > still be built.
> >
> > Toiv, of the AAU, declined to see the prospect of future conflicts over
> > research in strictly partisan terms. He said political pressure over the
> > deficit was bound to place discretionary domestic spending, which is the
> > source of NSF's money, under the microscope. "There was going to be a
> > problem no matter which party won," he said. But, he added, it remains
> > important for advocates of scientific research to highlight the economic
> > argument to support future funding. "We cannot hope to have the kind of
> > economic growth we need to address the deficit issue without, over the
> > long-term, making these investments now in basic research," said Toiv.
> > "These are investments. That's what needs to be clear."
> >
> > - Dan Berrett <dan.berrett@insidehighered.com>
> > __________________________________________
> > _____
> > xmca mailing list
> > xmca@weber.ucsd.edu
> > http://dss.ucsd.edu/mailman/listinfo/xmca
> > __________________________________________
> > _____
> > xmca mailing list
> > xmca@weber.ucsd.edu
> > http://dss.ucsd.edu/mailman/listinfo/xmca
> > __________________________________________
> > _____
> > xmca mailing list
> > xmca@weber.ucsd.edu
> > http://dss.ucsd.edu/mailman/listinfo/xmca
> >
>
>
>
> --
> WORK as:
> Visiting Lecturer
> Wits School of Education
> HOME (please use these details)
> 6 Andover Road
> Westdene
> Johannesburg 2092
> +27 (0)11 673 9265   +27 (0)82 562 1050
> __________________________________________
> _____
> xmca mailing list
> xmca@weber.ucsd.edu
> http://dss.ucsd.edu/mailman/listinfo/xmca
>
__________________________________________
_____
xmca mailing list
xmca@weber.ucsd.edu
http://dss.ucsd.edu/mailman/listinfo/xmca