Re: [xmca] Analyses of gesture & speech

Some of you might be interested in a chapter Lilian Pozzer-Ardenghi
and I wrote some time ago, which goes far beyond gesture and speech
and takes on many other semiotic resources produced and reproduced by
agents in setting. I can make available. Cheers, Michael

TRACKING SITUATED, DISTRIBUTED,
AND EMBODIED COMMUNICATION IN REAL TIME
Wolff-Michael Roth and Lilian Pozzer-Ardenghi
University of Victoria
ABSTRACT
Recent studies in cognitive anthropology and cognitive science have
argued for the
situated, distributed, and embodied nature of cognition. Whereas many
scholars have
jumped on this bandwagon, detailed concrete cases of how and why
cognition ought to be
understood as situated, distributed, and embodied are relatively
scarce. Yet our research
shows that verbal and gestural deixis in real everyday communication
constitutes a prime
example for how extralinguistic, bodily and situational elements are
included in
communication, where they are tracked by participants in real time
and much faster than
conscious interpretive processes would allow for. We articulate a
model for the situated,
distributed, and embodied production and use of semiotic resources in
the course of
situated communication as integral part of ongoing events without
privileging language,
and concretize the proposed model with episodes from a database of
extended video
recordings featuring students learning through design activities. We
bring together the
idea of situated communication as a dialectical unit and as layered,
distributed
representation into a broader framework of human activity. Any moment
in space and
time is analyzed in terms of one, dialectical unit of analysis of
activity, which integrates
both conscious (motive, goals) and unconscious aspects (conditions)
of human agency.

In: Focus on Cognitive Psychology Research ISBN 1-59454-634-7
Editors: Michael A. Vanshevsky, pp.237-261 © 2006 Nova
Science Publishers, Inc.

On 3-May-07, at 9:00 PM, minati panda wrote:

Hi Julian and Phil,
Thanks. These will be of great help to us.
Minati

On 5/3/07, Phil Chappell <philchappell@mac.com> wrote:
>
> Baldry and Thibault's work, which Jay Lemke alludes to, offers
> methodological ideas that I find really useful in classroom
> "discourse" analysis. They cite four criteria for attributing
> semiotic significance on a particular bodily act.
>
> 1. Semiotic significance of a particular part of the body involved in
> the act. Facial gestures usually involve semiosis of affect; spatial
> distance with power and hierarchy (social); and posture with personal
> defence.
>
> 2. Bodily action links cross-modally with other semiotic aspects of
> the discourse event/activity, such that meaning is created on the
> basis of these co-contextualising relations.
>
> 3. Such bodily actions are ascribed the status of dialogic acts
> rather than behvioural units. The authors use the gesture of smiling
> here to show a syntagmatic relation of Bodily Act^Response to Bodily
> Act - a dialogic exchange relation.
>
> 4. A bodily act can function as a semiotic feature having semantic
> commonalities (Jay's covariate semantic relationships) across
> modalities within the same discourse event/activity..."formally
> distinct events elements in a text are linked on the basis of their
> belonging to a common semantic class" (p. 179). This would equate to
> cohesion if doing a purely linguistic analysis of the event. I'm
> finding it an incredibly interesting lens through which to gaze at
> classroom video/audio-taped data.
>
> Of course, the book goes much farther than this, with sections on
> analysing web pages and film texts and genres.
>
> Baldry, & Thibault 2006, Multimodal transcription and text analysis:
> a multimedia toolkit and coursebook, Equinox, Oakville, CT.
>
>
> Cheers,
>
> Phil
>
>
>
>
> On 03/05/2007, at 9:51 PM, Michael A. Evans wrote:
>
> > Julian,
> >
> > All of these sound wonderful - if you can send citations, links, or
> > copies
> > of the articles and issues you mention, I'd be very
> > appreciative...I'm going
> > to ask my math ed colleague if he knows about the working group on
> > gesture
> > and embodied cognition in PME because this is one definite line of
> > analysis
> > in my work...
> >
> > Along the embodied cognition lines, a piece by Michael Anderson
> > made an
> > interesting (though far from complete) attempt to establish points
> > where
> > embodied cognition and situated cognition become
> > intertwined...hopefully,
> > the references you send will fill in more dots...here's the
> > reference (and
> > I'd be happy to forward a copy if needed):
> >
> > Anderson, M.L. (2003). Embodied cognition: A field guide. Artificial
> > Intelligence, 149, 91-130...
> >
> > Thanks!
> > Michael~
> > --
> > ____________________________________
> > Michael A. Evans
> > Assistant Professor
> > Instructional Design & Technology Program
> > School of Education
> > Virginia Tech
> >
> >
> >
> >> From: Julian Williams <julian.williams@manchester.ac.uk>
> >> Reply-To: "eXtended Mind, Culture, Activity" <xmca@weber.ucsd.edu>
> >> Date: Thu, 03 May 2007 09:41:18 +0100
> >> To: <xmca@weber.ucsd.edu>
> >> Cc: <andreas.koukkoufis@manchester.ac.uk>
> >> Subject: Re: [xmca] Analyses of gesture & speech
> >>
> >> Michael/Minati
> >>
> >> There is a journal issue of RELIME (special issue editor Luis
> >> Radford)
> >> just out
> >> on semiotics and culture: Andreas Koukkoufis and I have a paper
> >> about use of
> >> manipulatives (gesture and speech) in teaching about integers
> through
> >> games/RME
> >> method - plus there is a special issue of Ed Stud in Maths (ESM)
> >> under review
> >> on gesture and multimodality (eds Edwards Arzarello, Radford) and
> >> we have a
> >> paper in that too. There are some conference papers in procs from
> >> PME 26
> >> (Melbourne) in the Working group on gestures and embodied
> >> cognition: the ESM
> >> issue largely comes formthat workign group.
> >>
> >> You could also look for Radford in ESM
> >>
> >> I could send papers to people who want them.
> >>
> >> Julian
> >>
> >>
> >>
> >> Quoting minati panda <pandam66@gmail.com>:
> >>
> >>> Hi Michael,
> >>> I would like to know more about your work. I work on folk
> >>> mathematics.
> >>> Currently, I am working with Mike on folk games where we are
> >>> using both
> >>> discourse analysis and semiotic analysis. Will be happy to read
> >>> more about
> >>> your work/querries.
> >>> Minati
> >>>
> >>>
> >>> On 5/2/07, Michael A. Evans <mae@vt.edu> wrote:
> >>>>
> >>>> Wow, Jay - this is wonderful stuff! Once a draft paper gets its
> >>>> legs later
> >>>> in the summer, I'd be happy to share with the list (for those
> >>>> who might be
> >>>> interested in this type analysis as well)...
> >>>>
> >>>> Cheers,
> >>>> Michael~
> >>>> --
> >>>> ____________________________________
> >>>> Michael A. Evans
> >>>> Assistant Professor
> >>>> Instructional Design & Technology Program
> >>>> School of Education
> >>>> Virginia Tech
> >>>>
> >>>>
> >>>>
> >>>>> From: Jay Lemke <jaylemke@umich.edu>
> >>>>> Reply-To: "eXtended Mind, Culture, Activity"
> <xmca@weber.ucsd.edu>
> >>>>> Date: Wed, 02 May 2007 21:33:17 -0400
> >>>>> To: "eXtended Mind, Culture, Activity" <xmca@weber.ucsd.edu>
> >>>>> Subject: Re: [xmca] Analyses of gesture & speech
> >>>>>
> >>>>>
> >>>>> You have already been given some good leads in this area. In
> >>>>> general,
> >>>>> multimodal semiotic analysis, which incorporates discourse
> >>>>> analysis
> >>>>> as one component, is widely recognized as the next step in the
> >>>>> evolution of DA, which was never intended to ignore activity
> >>>>> and the
> >>>>> context of situation, but until fairly recently there have not
> >>>>> been
> >>>>> analyses of visual images, gesture and movment, music, etc.
> >>>>> that were
> >>>>> both "semantically aware", i.e. linked to meaning, and
> compatible
> >>>>> enough with linguistic analysis of meaning to be combined.
> >>>>>
> >>>>> Some of the early pioneers of linguistic DA, like Gunther Kress
> >>>>> and
> >>>>> Theo van Leeuwen, write mainly about multimodal or multimedia
> >>>>> analysis today. Paul Thibault and Anthony Baldry have developed
> >>>>> systems for multimedia corpus analysis and multimodal
> >>>>> transcription
> >>>>> based on the principles of the earlier linguistic discourse
> >>>>> analysis.
> >>>>> There is a conference on linguistic DA and multimodal
> analysis in
> >>>>> Helsinki in July (google along with the organizer, Eija
> >>>>> Ventola), was
> >>>>> a broader one on multimodality last year in Pavia, Italty
> (google
> >>>>> with Anthony Baldry, TICOM) which will issue a volume this
> >>>>> year. Adam
> >>>>> Kendon, another pioneer of gesture analysis, spoke at that
> one (so
> >>>>> did I), and his work for some time now has integrated gesture
> >>>>> analysis with the broader situational and activity context.
> >>>>>
> >>>>> On my website you can see the syllabus for a course I did last
> >>>>> year
> >>>>> on video and multimedia analysis, and some earlier versions
> of it
> >>>>> that showed the integration with discourse analysis (but it's
> too
> >>>>> much for one semester). Go to homepage, then Student's
> Entrance to
> >>>>> see course information.
> >>>>>
> >>>>> Good luck!
> >>>>>
> >>>>> JAY.
> >>>>> www.umich.edu/~jaylemke
> >>>>>
> >>>>>> Wanted to thank everyone who forwarded articles, links, and
> >>>>>> tips...this
> >>>> will
> >>>>>> be extremely useful as we analyze video data over the summer
> >>>>>> months...
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Michael~
> >>>>>> --
> >>>>>> ____________________________________
> >>>>>> Michael A. Evans
> >>>>>> Assistant Professor
> >>>>>> Instructional Design & Technology Program
> >>>>>> School of Education
> >>>>>> Virginia Tech
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> From: Carol Macdonald <carolmacdon@gmail.com>
> >>>>>>> Reply-To: "eXtended Mind, Culture, Activity"
> >>>>>>> <xmca@weber.ucsd.edu>
> >>>>>>> Date: Mon, 30 Apr 2007 08:18:26 +0200
> >>>>>>> To: "eXtended Mind, Culture, Activity" <xmca@weber.ucsd.edu>
> >>>>>>> Subject: Re: [xmca] Analyses of gesture & speech
> >>>>>>>
> >>>>>>> I think it should not be too difficult (cf all the helpful
> >>>>>>> comments
> >>>> below).
> >>>>>>> Capture all your data on a godd audio video, watching the
> >>>>>>> number
> >>>> count.
> >>>>>>> Then using perhaps Sinclair and Coultard analyse the
> >>>>>>> language ( which
> >>>> in
> >>>>>>> any
> >>>>>>> case overlaps beween speakers), then go through it again,
> >>>>>>> this time
> >>>>>>> matching
> >>>>>>> the gestures to thre language. Once you have paired
> >>>>>>> language and
> >>>> gesture,
> >>>>>>> then you have to work out why this should be working
> >>>>>>> together: this
> >>>> is
> >>>>>>> monstrously time-consuming, but it will get you where you
> >>>>>>> want to go.
> >>>>>>> Carol
> >>>>>>>
> >>>>>>>
> >>>>>>> On 29/04/07, Wolff-Michael Roth <mroth@uvic.ca> wrote:
> >>>>>>>>
> >>>>>>>> You may find my review useful for seeing what has been
> >>>>>>>> published in
> >>>>>>>> the different disciplines referred to on the list:
> >>>>>>>> Roth, W.-M. (2002). Gestures: Their role in teaching and
> >>>>>>>> learning.
> >>>>>>>> Review of Educational Research, 71, 365-392.
> >>>>>>>>
> >>>>>>>> also, in
> >>>>>>>> Roth, W.-M., & Lee, Y. J. (2004). Interpreting unfamiliar
> >>>>>>>> graphs: A
> >>>>>>>> generative, activity-theoretic model. Educational Studies in
> >>>>>>>> Mathematics, 57, 265-290.
> >>>>>>>>
> >>>>>>>> we cover mathematical cognition, gestures, and CHAT.
> >>>>>>>> Hope this helps,
> >>>>>>>> Michael
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 28-Apr-07, at 8:40 AM, Michael A. Evans wrote:
> >>>>>>>>
> >>>>>>>> Dear All,
> >>>>>>>>
> >>>>>>>> I was hoping you could point me to resources that critique
> >>>>>>>> discourse
> >>>>>>>> analysis as an overly linguistic approach to interaction
> >>>>>>>> and meaning
> >>>>>>>> making...my request is based on a desire to ground analysis
> >>>>>>>> of video
> >>>>>>>> data of
> >>>>>>>> children using manipulatives (both physical and graphically-
> >>>>>>>> based)
> >>>> in
> >>>>>>>> collaborative efforts...what I want to capture is not
> only the
> >>>> speech
> >>>>>>>> but
> >>>>>>>> also gesture of primary students as they try to make sense
> >>>>>>>> of basic
> >>>>>>>> geometric concepts and principles using manipulatives
> >>>>>>>> (tangrams,
> >>>>>>>> pentominoes, geoboards, etc)...I'm searching for both
> >>>>>>>> philosophical
> >>>>>>>> (Vygotsky, Dewey, Pierce, Mead) and methodological
> >>>>>>>> references that
> >>>>>>>> emphasize
> >>>>>>>> the need to examine gesture and speech simultaneously...
> >>>>>>>>
> >>>>>>>> As for the latter, I've recently been working with David
> >>>>>>>> McNeill and
> >>>> his
> >>>>>>>> group at the U. of Chicago (http://
> >>>>>>>> mcneilllab.uchicago.edu/), but
> >>>> was
> >>>>>>>> hoping
> >>>>>>>> I could get more leads from the group...
> >>>>>>>>
> >>>>>>>> Thanks!
> >>>>>>>> Michael~
> >>>>>>>> --
> >>>>>>>> ____________________________________
> >>>>>>>> Michael A. Evans
> >>>>>>>> Assistant Professor
> >>>>>>>> Instructional Design & Technology Program
> >>>>>>>> School of Education
> >>>>>>>> Virginia Tech
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> xmca mailing list
> >>>>>>>> xmca@weber.ucsd.edu
> >>>>>>>> http://dss.ucsd.edu/mailman/listinfo/xmca
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> xmca mailing list
> >>>>>>>> xmca@weber.ucsd.edu
> >>>>>>>> http://dss.ucsd.edu/mailman/listinfo/xmca
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> 6 Andover Road
> >>>>>>> Westdene
> >>>>>>> 2092 Johannesburg
> >>>>>>> 011 673 9265 082 562 1050
> >>>>>>> _______________________________________________
> >>>>>>> xmca mailing list
> >>>>>>> xmca@weber.ucsd.edu
> >>>>>>> http://dss.ucsd.edu/mailman/listinfo/xmca
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> xmca mailing list
> >>>>>> xmca@weber.ucsd.edu
> >>>>>> http://dss.ucsd.edu/mailman/listinfo/xmca
> >>>>>
> >>>>>
> >>>>> --
> >>>>>
> >>>>> JAY L. LEMKE
> >>>>> Educational Studies
> >>>>> University of Michigan
> >>>>> 610 East University
> >>>>> Ann Arbor, MI 48109
> >>>>>
> >>>>> Ph: 734-763-9276
> >>>>> Fax: 734-936-1606
> >>>>> www.umich.edu/~jaylemke/
> >>>>> _______________________________________________
> >>>>> xmca mailing list
> >>>>> xmca@weber.ucsd.edu
> >>>>> http://dss.ucsd.edu/mailman/listinfo/xmca
> >>>>
> >>>> _______________________________________________
> >>>> xmca mailing list
> >>>> xmca@weber.ucsd.edu
> >>>> http://dss.ucsd.edu/mailman/listinfo/xmca
> >>>>
> >>> _______________________________________________
> >>> xmca mailing list
> >>> xmca@weber.ucsd.edu
> >>> http://dss.ucsd.edu/mailman/listinfo/xmca
> >>>
> >>
> >>
> >>
> >> _______________________________________________
> >> xmca mailing list
> >> xmca@weber.ucsd.edu
> >> http://dss.ucsd.edu/mailman/listinfo/xmca
> >
> > _______________________________________________
> > xmca mailing list
> > xmca@weber.ucsd.edu
> > http://dss.ucsd.edu/mailman/listinfo/xmca
>
> _______________________________________________
> xmca mailing list
> xmca@weber.ucsd.edu
> http://dss.ucsd.edu/mailman/listinfo/xmca
>
_______________________________________________
xmca mailing list
xmca@weber.ucsd.edu
http://dss.ucsd.edu/mailman/listinfo/xmca

_______________________________________________
xmca mailing list
xmca@weber.ucsd.edu
http://dss.ucsd.edu/mailman/listinfo/xmca
Received on Thu May 3 22:24 PDT 2007