Re: [xmca] Analyses of gesture & speech

From: Michael A. Evans <mae who-is-at vt.edu>
Date: Fri May 04 2007 - 04:54:23 PDT

Please forward a copy my way as well, Michael...

In regard to Paul's questions, "... what of gestures between students during
collaborative work?" this is precisely what we're investigating in our
current work - specifically, we are analyzing changes in gesture, gaze, and
speech as PreK-3 children arranged in triads attempt to complete tasks in 2D
geometry using a range of manipulatives (tangrams, geoboards, pentominoes,
pattern blocks, etc)...moreover, we are interested in how changes in the
artifact (from physical shapes on the tabletop to virtual shapes on the
computer screen) differently influence problem-solving and reasoning...here
is an excerpt from our manuscript that might begin to address Paul's
question:

" The virtual manipulative group produced a large number of pointing
gestures, some simulated manipulation gestures aimed at or contacting the
screen, and an occasional interactive gesture. The physical object group,
again in sharp contrast, performed few gestures of any kind; their concern
was with actions directed at objects. The two modalities ­ speech and
gestures of both the deictic and interactive kinds ­ can be correlated as
well. Thus, interactive gestures are linked with para level statements
[cohesion through references that include individual participants] and
deictic gestures with object level statements [cohesion through references
to object world]. But there are also cross-correlations between gestural
deixis and linguistic para. The methods outlined here seem directly
applicable to the situations to be studied in the proposed research."

Our goal in all of this is to develop and pilot test a tabletop interface (a
"math table" if you will) whereby children maintain the social exchange
facilitated by orientation around a table, while focusing on geometric
concepts and transformations of virtual manipulative...

Thanks, again, to everyone for providing such wonderful references and
discussion - this will be tremendously useful for write up this summer...

Cheers,
Michael~

-- 
____________________________________
Michael A. Evans
Assistant Professor
Instructional Design & Technology Program
School of Education
Virginia Tech  
> From: Wolff-Michael Roth <mroth@uvic.ca>
> Reply-To: "eXtended Mind, Culture, Activity" <xmca@weber.ucsd.edu>
> Date: Thu, 03 May 2007 21:22:44 -0700
> To: "eXtended Mind, Culture, Activity" <xmca@weber.ucsd.edu>
> Subject: Re: [xmca] Analyses of gesture & speech
> 
> Some of you might be interested in a chapter Lilian Pozzer-Ardenghi
> and I wrote some time ago, which goes far beyond gesture and speech
> and takes on many other semiotic resources produced and reproduced by
> agents in setting. I can make available. Cheers, Michael
> 
> TRACKING SITUATED, DISTRIBUTED,
> AND EMBODIED COMMUNICATION IN REAL TIME
> Wolff-Michael Roth and Lilian Pozzer-Ardenghi
> University of Victoria
> ABSTRACT
> Recent studies in cognitive anthropology and cognitive science have
> argued for the
> situated, distributed, and embodied nature of cognition. Whereas many
> scholars have
> jumped on this bandwagon, detailed concrete cases of how and why
> cognition ought to be
> understood as situated, distributed, and embodied are relatively
> scarce. Yet our research
> shows that verbal and gestural deixis in real everyday communication
> constitutes a prime
> example for how extralinguistic, bodily and situational elements are
> included in
> communication, where they are tracked by participants in real time
> and much faster than
> conscious interpretive processes would allow for. We articulate a
> model for the situated,
> distributed, and embodied production and use of semiotic resources in
> the course of
> situated communication as integral part of ongoing events without
> privileging language,
> and concretize the proposed model with episodes from a database of
> extended video
> recordings featuring students learning through design activities. We
> bring together the
> idea of situated communication as a dialectical unit and as layered,
> distributed
> representation into a broader framework of human activity. Any moment
> in space and
> time is analyzed in terms of one, dialectical unit of analysis of
> activity, which integrates
> both conscious (motive, goals) and unconscious aspects (conditions)
> of human agency.
> 
> 
> In: Focus on Cognitive Psychology Research            ISBN 1-59454-634-7
> Editors: Michael A. Vanshevsky, pp.237-261            © 2006 Nova
> Science Publishers, Inc.
> 
> 
> 
> 
> 
> 
> On 3-May-07, at 9:00 PM, minati panda wrote:
> 
> Hi Julian and Phil,
> Thanks. These will be of great help to us.
> Minati
> 
> 
> On 5/3/07, Phil Chappell <philchappell@mac.com> wrote:
>> 
>> Baldry and Thibault's work, which Jay Lemke alludes to, offers
>> methodological ideas that I find really useful in classroom
>> "discourse" analysis. They cite four criteria for attributing
>> semiotic significance on a particular bodily act.
>> 
>> 1. Semiotic significance of a particular part of the body involved in
>> the act. Facial gestures usually involve semiosis of affect; spatial
>> distance with power and hierarchy (social); and posture with personal
>> defence.
>> 
>> 2. Bodily action links cross-modally with other semiotic aspects of
>> the discourse event/activity, such that meaning is created on the
>> basis of these co-contextualising relations.
>> 
>> 3. Such bodily actions are ascribed the status of dialogic acts
>> rather than behvioural units. The authors use the gesture of smiling
>> here to show a syntagmatic relation of Bodily Act^Response to Bodily
>> Act - a dialogic exchange relation.
>> 
>> 4. A bodily act can function as a semiotic feature having semantic
>> commonalities (Jay's covariate semantic relationships) across
>> modalities within the same discourse event/activity..."formally
>> distinct events elements in a text are linked on the basis of their
>> belonging to a common semantic class" (p. 179). This would equate to
>> cohesion if doing a purely linguistic analysis of the event. I'm
>> finding it an incredibly interesting lens through which to gaze at
>> classroom video/audio-taped data.
>> 
>> Of course, the book goes much farther than this, with sections on
>> analysing web pages and film texts and genres.
>> 
>> Baldry, & Thibault 2006, Multimodal transcription and text analysis:
>> a multimedia toolkit and coursebook, Equinox, Oakville, CT.
>> 
>> 
>> Cheers,
>> 
>> Phil
>> 
>> 
>> 
>> 
>> On 03/05/2007, at 9:51 PM, Michael A. Evans wrote:
>> 
>>> Julian,
>>> 
>>> All of these sound wonderful - if you can send citations, links, or
>>> copies
>>> of the articles and issues you mention, I'd be very
>>> appreciative...I'm going
>>> to ask my math ed colleague if he knows about the working group on
>>> gesture
>>> and embodied cognition in PME because this is one definite line of
>>> analysis
>>> in my work...
>>> 
>>> Along the embodied cognition lines, a piece by Michael Anderson
>>> made an
>>> interesting (though far from complete) attempt to establish points
>>> where
>>> embodied cognition and situated cognition become
>>> intertwined...hopefully,
>>> the references you send will fill in more dots...here's the
>>> reference (and
>>> I'd be happy to forward a copy if needed):
>>> 
>>> Anderson, M.L. (2003). Embodied cognition: A field guide. Artificial
>>> Intelligence, 149, 91-130...
>>> 
>>> Thanks!
>>> Michael~
>>> --
>>> ____________________________________
>>> Michael A. Evans
>>> Assistant Professor
>>> Instructional Design & Technology Program
>>> School of Education
>>> Virginia Tech
>>> 
>>> 
>>> 
>>>> From: Julian Williams <julian.williams@manchester.ac.uk>
>>>> Reply-To: "eXtended Mind, Culture, Activity" <xmca@weber.ucsd.edu>
>>>> Date: Thu, 03 May 2007 09:41:18 +0100
>>>> To: <xmca@weber.ucsd.edu>
>>>> Cc: <andreas.koukkoufis@manchester.ac.uk>
>>>> Subject: Re: [xmca] Analyses of gesture & speech
>>>> 
>>>> Michael/Minati
>>>> 
>>>> There is a journal issue of RELIME (special issue editor Luis
>>>> Radford)
>>>> just out
>>>> on semiotics and culture: Andreas Koukkoufis and I have a paper
>>>> about use of
>>>> manipulatives (gesture and speech) in teaching about integers
>> through
>>>> games/RME
>>>> method - plus there is a special issue of Ed Stud in Maths (ESM)
>>>> under review
>>>> on gesture and multimodality (eds Edwards Arzarello, Radford) and
>>>> we have a
>>>> paper in that too. There are some conference papers in procs from
>>>> PME 26
>>>> (Melbourne) in the Working group on gestures and embodied
>>>> cognition: the ESM
>>>> issue largely comes formthat workign group.
>>>> 
>>>> You could also look for Radford in ESM
>>>> 
>>>> I could send papers to people who want them.
>>>> 
>>>> Julian
>>>> 
>>>> 
>>>> 
>>>> Quoting minati panda <pandam66@gmail.com>:
>>>> 
>>>>> Hi Michael,
>>>>> I would like to know more about your work. I work on folk
>>>>> mathematics.
>>>>> Currently, I am working with Mike on folk games where we are
>>>>> using both
>>>>> discourse analysis and semiotic  analysis. Will be happy to read
>>>>> more about
>>>>> your work/querries.
>>>>> Minati
>>>>> 
>>>>> 
>>>>> On 5/2/07, Michael A. Evans <mae@vt.edu> wrote:
>>>>>> 
>>>>>> Wow, Jay - this is wonderful stuff! Once a draft paper gets its
>>>>>> legs later
>>>>>> in the summer, I'd be happy to share with the list (for those
>>>>>> who might be
>>>>>> interested in this type analysis as well)...
>>>>>> 
>>>>>> Cheers,
>>>>>> Michael~
>>>>>> --
>>>>>> ____________________________________
>>>>>> Michael A. Evans
>>>>>> Assistant Professor
>>>>>> Instructional Design & Technology Program
>>>>>> School of Education
>>>>>> Virginia Tech
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> From: Jay Lemke <jaylemke@umich.edu>
>>>>>>> Reply-To: "eXtended Mind, Culture, Activity"
>> <xmca@weber.ucsd.edu>
>>>>>>> Date: Wed, 02 May 2007 21:33:17 -0400
>>>>>>> To: "eXtended Mind, Culture, Activity" <xmca@weber.ucsd.edu>
>>>>>>> Subject: Re: [xmca] Analyses of gesture & speech
>>>>>>> 
>>>>>>> 
>>>>>>> You have already been given some good leads in this area. In
>>>>>>> general,
>>>>>>> multimodal semiotic analysis, which incorporates discourse
>>>>>>> analysis
>>>>>>> as one component, is widely recognized as the next step in the
>>>>>>> evolution of DA, which was never intended to ignore activity
>>>>>>> and the
>>>>>>> context of situation, but until fairly recently there have not
>>>>>>> been
>>>>>>> analyses of visual images, gesture and movment, music, etc.
>>>>>>> that were
>>>>>>> both "semantically aware", i.e. linked to meaning, and
>> compatible
>>>>>>> enough with linguistic analysis of meaning to be combined.
>>>>>>> 
>>>>>>> Some of the early pioneers of linguistic DA, like Gunther Kress
>>>>>>> and
>>>>>>> Theo van Leeuwen, write mainly about multimodal or multimedia
>>>>>>> analysis today. Paul Thibault and Anthony Baldry have developed
>>>>>>> systems for multimedia corpus analysis and multimodal
>>>>>>> transcription
>>>>>>> based on the principles of the earlier linguistic discourse
>>>>>>> analysis.
>>>>>>> There is a conference on linguistic DA and multimodal
>> analysis in
>>>>>>> Helsinki in July (google along with the organizer, Eija
>>>>>>> Ventola), was
>>>>>>> a broader one on multimodality last year in Pavia, Italty
>> (google
>>>>>>> with Anthony Baldry, TICOM) which will issue a volume this
>>>>>>> year. Adam
>>>>>>> Kendon, another pioneer of gesture analysis, spoke at that
>> one (so
>>>>>>> did I), and his work for some time now has integrated gesture
>>>>>>> analysis with the broader situational and activity context.
>>>>>>> 
>>>>>>> On my website you can see the syllabus for a course I did last
>>>>>>> year
>>>>>>> on video and multimedia analysis, and some earlier versions
>> of it
>>>>>>> that showed the integration with discourse analysis (but it's
>> too
>>>>>>> much for one semester). Go to homepage, then Student's
>> Entrance to
>>>>>>> see course information.
>>>>>>> 
>>>>>>> Good luck!
>>>>>>> 
>>>>>>> JAY.
>>>>>>> www.umich.edu/~jaylemke
>>>>>>> 
>>>>>>>> Wanted to thank everyone who forwarded articles, links, and
>>>>>>>> tips...this
>>>>>> will
>>>>>>>> be extremely useful as we analyze video data over the summer
>>>>>>>> months...
>>>>>>>> 
>>>>>>>> Cheers,
>>>>>>>> Michael~
>>>>>>>> --
>>>>>>>> ____________________________________
>>>>>>>> Michael A. Evans
>>>>>>>> Assistant Professor
>>>>>>>> Instructional Design & Technology Program
>>>>>>>> School of Education
>>>>>>>> Virginia Tech
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>>>  From: Carol Macdonald <carolmacdon@gmail.com>
>>>>>>>>>  Reply-To: "eXtended Mind, Culture, Activity"
>>>>>>>>> <xmca@weber.ucsd.edu>
>>>>>>>>>  Date: Mon, 30 Apr 2007 08:18:26 +0200
>>>>>>>>>  To: "eXtended Mind, Culture, Activity" <xmca@weber.ucsd.edu>
>>>>>>>>>  Subject: Re: [xmca] Analyses of gesture & speech
>>>>>>>>> 
>>>>>>>>>  I think it should not be too difficult (cf all the helpful
>>>>>>>>> comments
>>>>>> below).
>>>>>>>>>  Capture all your data on  a godd audio video, watching the
>>>>>>>>> number
>>>>>> count.
>>>>>>>>>  Then using perhaps Sinclair and Coultard analyse the
>>>>>>>>> language ( which
>>>>>> in
>>>>>>>>> any
>>>>>>>>>  case overlaps beween speakers), then go through it again,
>>>>>>>>> this time
>>>>>>>>> matching
>>>>>>>>>  the gestures to thre language.  Once you have paired
>>>>>>>>> language and
>>>>>> gesture,
>>>>>>>>>  then you have to work out why this should be working
>>>>>>>>> together: this
>>>>>> is
>>>>>>>>>  monstrously time-consuming, but it will get you where you
>>>>>>>>> want to go.
>>>>>>>>>  Carol
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>  On 29/04/07, Wolff-Michael Roth <mroth@uvic.ca> wrote:
>>>>>>>>>> 
>>>>>>>>>>  You may find my review useful for seeing what has been
>>>>>>>>>> published in
>>>>>>>>>>  the different disciplines referred to on the list:
>>>>>>>>>>  Roth, W.-M. (2002). Gestures: Their role in teaching and
>>>>>>>>>> learning.
>>>>>>>>>>  Review of Educational Research, 71, 365-392.
>>>>>>>>>> 
>>>>>>>>>>  also, in
>>>>>>>>>>  Roth, W.-M., & Lee, Y. J. (2004). Interpreting unfamiliar
>>>>>>>>>> graphs: A
>>>>>>>>>>  generative, activity-theoretic model. Educational Studies in
>>>>>>>>>>  Mathematics, 57, 265-290.
>>>>>>>>>> 
>>>>>>>>>>  we cover mathematical cognition, gestures, and CHAT.
>>>>>>>>>>  Hope this helps,
>>>>>>>>>>  Michael
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>  On 28-Apr-07, at 8:40 AM, Michael A. Evans wrote:
>>>>>>>>>> 
>>>>>>>>>>  Dear All,
>>>>>>>>>> 
>>>>>>>>>>  I was hoping you could point me to resources that critique
>>>>>>>>>> discourse
>>>>>>>>>>  analysis as an overly linguistic approach to interaction
>>>>>>>>>> and meaning
>>>>>>>>>>  making...my request is based on a desire to ground analysis
>>>>>>>>>> of video
>>>>>>>>>>  data of
>>>>>>>>>>  children using manipulatives (both physical and graphically-
>>>>>>>>>> based)
>>>>>> in
>>>>>>>>>>  collaborative efforts...what I want to capture is not
>> only the
>>>>>> speech
>>>>>>>>>>  but
>>>>>>>>>>  also gesture of primary students as they try to make sense
>>>>>>>>>> of basic
>>>>>>>>>>  geometric concepts and principles using manipulatives
>>>>>>>>>> (tangrams,
>>>>>>>>>> pentominoes, geoboards, etc)...I'm searching for both
>>>>>>>>>> philosophical
>>>>>>>>>>  (Vygotsky, Dewey, Pierce, Mead) and methodological
>>>>>>>>>> references that
>>>>>>>>>>  emphasize
>>>>>>>>>>  the need to examine gesture and speech simultaneously...
>>>>>>>>>> 
>>>>>>>>>>  As for the latter, I've recently been working with David
>>>>>>>>>> McNeill and
>>>>>> his
>>>>>>>>>>  group at the U. of Chicago (http://
>>>>>>>>>> mcneilllab.uchicago.edu/), but
>>>>>> was
>>>>>>>>>>  hoping
>>>>>>>>>>  I could get more leads from the group...
>>>>>>>>>> 
>>>>>>>>>>  Thanks!
>>>>>>>>>>  Michael~
>>>>>>>>>>  --
>>>>>>>>>>  ____________________________________
>>>>>>>>>>  Michael A. Evans
>>>>>>>>>>  Assistant Professor
>>>>>>>>>>  Instructional Design & Technology Program
>>>>>>>>>>  School of Education
>>>>>>>>>>  Virginia Tech
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>  _______________________________________________
>>>>>>>>>>  xmca mailing list
>>>>>>>>>>  xmca@weber.ucsd.edu
>>>>>>>>>>  http://dss.ucsd.edu/mailman/listinfo/xmca
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>  _______________________________________________
>>>>>>>>>>  xmca mailing list
>>>>>>>>>>  xmca@weber.ucsd.edu
>>>>>>>>>>  http://dss.ucsd.edu/mailman/listinfo/xmca
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>  --
>>>>>>>>>  6 Andover Road
>>>>>>>>>  Westdene
>>>>>>>>>  2092 Johannesburg
>>>>>>>>>  011 673 9265  082 562 1050
>>>>>>>>>  _______________________________________________
>>>>>>>>>  xmca mailing list
>>>>>>>>>  xmca@weber.ucsd.edu
>>>>>>>>>  http://dss.ucsd.edu/mailman/listinfo/xmca
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> xmca mailing list
>>>>>>>> xmca@weber.ucsd.edu
>>>>>>>> http://dss.ucsd.edu/mailman/listinfo/xmca
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> 
>>>>>>> JAY L. LEMKE
>>>>>>> Educational Studies
>>>>>>> University of Michigan
>>>>>>> 610 East University
>>>>>>> Ann Arbor, MI 48109
>>>>>>> 
>>>>>>> Ph: 734-763-9276
>>>>>>> Fax: 734-936-1606
>>>>>>> www.umich.edu/~jaylemke/
>>>>>>> _______________________________________________
>>>>>>> xmca mailing list
>>>>>>> xmca@weber.ucsd.edu
>>>>>>> http://dss.ucsd.edu/mailman/listinfo/xmca
>>>>>> 
>>>>>> _______________________________________________
>>>>>> xmca mailing list
>>>>>> xmca@weber.ucsd.edu
>>>>>> http://dss.ucsd.edu/mailman/listinfo/xmca
>>>>>> 
>>>>> _______________________________________________
>>>>> xmca mailing list
>>>>> xmca@weber.ucsd.edu
>>>>> http://dss.ucsd.edu/mailman/listinfo/xmca
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> xmca mailing list
>>>> xmca@weber.ucsd.edu
>>>> http://dss.ucsd.edu/mailman/listinfo/xmca
>>> 
>>> _______________________________________________
>>> xmca mailing list
>>> xmca@weber.ucsd.edu
>>> http://dss.ucsd.edu/mailman/listinfo/xmca
>> 
>> _______________________________________________
>> xmca mailing list
>> xmca@weber.ucsd.edu
>> http://dss.ucsd.edu/mailman/listinfo/xmca
>> 
> _______________________________________________
> xmca mailing list
> xmca@weber.ucsd.edu
> http://dss.ucsd.edu/mailman/listinfo/xmca
> 
> _______________________________________________
> xmca mailing list
> xmca@weber.ucsd.edu
> http://dss.ucsd.edu/mailman/listinfo/xmca
_______________________________________________
xmca mailing list
xmca@weber.ucsd.edu
http://dss.ucsd.edu/mailman/listinfo/xmca
Received on Fri May 4 05:55 PDT 2007

This archive was generated by hypermail 2.1.8 : Fri Mar 21 2008 - 16:43:26 PDT