[Xmca-l] Re: Intonation and Gesture

Tue Apr 8 16:11:21 PDT 2014

Alex: Yes, you're absolutely right; we can't have melody without
rhythm too The conductor (Evilino Pido) and the singer (Natalie
Dessay) are very well coordinated. In SOME cases (e.g. when Natalie
looks up at Evelino through those great green eyes at the end of a
phrase), the coordination is quite obviously inter-personal, that is,
a matter of what Treverthenan calls primary intersubjectivity. But in
other cases (e.g. when Evilino left hand rises within a tenth of a
second of the moment that Natalie's right hand rises) they are both
orienting to the beat, and we have to call this secondary
intersubjectivity.

Adam raises the issue which I think explains this distinction
perfectly when he conducts the thought experiment of seeing how the
gestures change from rehearsal to performance. This isn't just a
thought experiment; it's exactly what my student Kim Donggyun did, and
I'll tell you about what he found in a moment. But for interpreting
this clip, I think that the distinction we want isn't rehearsal to
performance. What the clip shows is a RECORDING, not an operatic
performance on stage, and as Natalie explains, witih a recording you
concentrate entirely on getting the musical phrasing perfectly and you
don't bother to try to act. That means that Natalie is really just
conducting herself as part of the orchestra and not part of the cast,
and that means that there is more interpersonal coordination, more
primary intersubjectivity, than you would find in a stage performance,
where the singers must interact with each other and above all with the
script rather than with the orchestra and the conductor.(You can see
something of this if you compare the earliest filmed operatic
performances in the sixties and seventies with operatic performances
today, or if you compare contemporary Chinese opera with western
opera. In the sixties and seventies, and in Chinese opera, the singer
is a musician first and a character second; in current Western
performances it's the other way around.)

I find this exquisite--because it is so very Brechtian. In our day of
versimilitude and "method" acting, and in our cinema of "realistic"
three dimension animation and computer generated graphics, it is hard
to remember that one of the main purposes of art is to hold experience
at arms length and contemplate it as artifice. So I actually find
Natalie recording even more expressive--or perhaps I should say
contemplative--than the mimetic Natalie on stage.

Adam wonders if the gesture would disappear into the intonation as the
children in our experiment get more fluent. What I can tell you is
that the children WANTED that to happen, and that is why they locked
their hands behind their backs. The other thing they did was to look
at the floor during role plays rather than at each other or into air
(as they tried to remember the words they were supposed to say). This
kind of dispassionate recital was a kind of model of adult expertise,
whereas flailing with their hands or search the air (or the
whiteboard) for vocabulary was considered undignified. (We also found
some evidence that the children were RIGHT in this judgment--when we
translated the role plays into Korean and had them performed by
absolute beginners in the Korean language in Australia, we found that
there was MORE gesturing, even though the children were slightly
older. So gesturing--or at least non-volitional gesturing--really was
a symptom of helplessness.

The gesturing did not, however, disappear. As soon as we told the
children they couldn't lock their hands behind their back and we
docked them points for looking at the floor, we found them gesturing
again. Interestingly, though, we did find that as the children got
older and more confident (we compared third graders and fifth graders
in elementary school), the gestures changed qualitatively rather than
quantitatively--they became much better coordinated with the stresses
of what they said and they also better reflected the intonation. We
also used Praat software to see if taught gestures would improve
timing and intonation, and we found that, within certain limits, they
did.

Larry brings up the issue of "co-speech" and "embodied" communication
that is raised in McNeiil's boooks ("Hand and Mind", "Gesture and
Thought") and also the work of Goldin-Meadow. I have a number of
objections to this work which I still haven't quite sorted out in my
mind:

a) All of this work really rejects the idea of internalization, and
with it, mediation.

b) Gesture has to co-evolve with speech; it never disappears into it.
So we cannot have a theory of WRITTEN speech, and we cannot explain
the close link between intonation and gesture or between intonation
and information.

c) The "Growth Point" that McNeill proposes as a substitute for the
ZPD doesn't seem to me to distinguish between learning and
development.

I think, to tie this to previous threads, that "co-speech" and
"embodied" communication fits too easily into a theory of language
that is based on the INDIVIDUAL (specifically, the indivdual body)
rather than the social. I think, actually, the Gilbert/Magnus/Gibson
theory of direct perception of phonemes is a perfect illustration of
why this doesn't work. If the key to the meaning of segments were in
the body itself, the Tower of Babel would be nothing but a myth; we
would all speak the same language, and it would be English.

(Vygotsky, as usual, has a better idea--see HDHMF, p. 128! He says
that the ultimate basis for word meanings really is "literal", but it
is some kind of graphic-visual image which is then generalized and
abstracted. So, for example, "I stand on firm ground" would once mean
that your boots were dirty, and it still gives us a tiny glance at
dirty boots today, before opening out into the wider and even more
cultural vista of metaphor. But while the image we have in our heads
feels visual and co-extensive with the body, it is culturally
generated, as the very word "literal" implies!)

David Kellogg
Hankuk University of Foreign Studies

On 9 April 2014 07:05, Larry Purss <lpscholar2@gmail.com> wrote:
> Martin last year referenced Elena's work on language AS gesture.
> Here is here abstract describing her doctoral thesis.
>
> She has now moved to Spain and continuing her studies on gesture WITH A
> TEAM OF SCHOLARS exploring this theme
>
>
> A DISSERTATION
>
> Presented to the Department of Philosophy and the Graduate School of the
> University of Oregon in partial fulfillment of the requirements for the
> degree of
>
> Doctor of Philosophy
>
> December 2011
>
> ii
>
> DISSERTATION APPROVAL PAGE
>
> Student: Elena Clare Cuffari
>
> Title: Co-Speech Gesture in Communication and Cognition
>
> This dissertation has been accepted and approved in partial fulfillment of
> the requirements for the Doctor of Philosophy degree in the Department of
> Philosophy by:
>
> Mark Johnson Chairperson Ted Toadvine Member Naomi Zack Member
>
> Eric Pederson Outside Member
>
> and
>
> Kimberly Andrews Espy Vice President for Research & Innovation/Dean of the
>
> Graduate School
>
> Original approval signatures are on file with the University of Oregon
> Graduate School.
>
> Degree awarded December 2011
>
> iii
>
> (c) 2011 Elena Clare Cuffari
>
> iv
>
> DISSERTATION ABSTRACT
>
> Elena Clare Cuffari
>
> Doctor of Philosophy
>
> Department of Philosophy
>
> December 2011
>
> Title: Co-Speech Gesture in Communication and Cognition
>
> This dissertation stages a reciprocal critique between traditional and
> marginal
>
> philosophical approaches to language on the one hand and interdisciplinary
> studies of speech-accompanying hand gestures on the other. Gesturing with
> the hands while speaking is a ubiquitous, cross-cultural human practice.
> Yet this practice is complex, varied, conventional, nonconventional, and
> above all under-theorized. In light of the theoretical and empirical
> treatments of language and gesture that I engage in, I argue that the hand
> gestures that spontaneously accompany speech are a part of language; more
> specifically, they are enactments of linguistic meaning. They are
> simultaneously (acts of) cognition and communication. Human communication
> and cognition are what they are in part because of this practice of
> gesturing. This argument has profound implications for philosophy, for
> gesture studies, and for interdisciplinary work to come.
>
> As further, strong proof of the pervasively embodied way that humans make
> meaning in language, reflection on gestural phenomena calls for a complete
> re-orientation in traditional analytic philosophy of language. Yet
> philosophical awareness of intersubjectivity and normativity as conditions
> of meaning achievement is well-deployed in elaborating and refining the
> minimal theoretical apparatus of present-day gesture studies. Triangulating
> between the most social, communicative philosophies of meaning
>
> v
>
> and the most nuanced, reflective treatments of co-speech hand gesture, I
> articulate a new construal of language as embodied, world-embedded,
> intersubjectively normative, dynamic, multi-modal enacting of appropriative
> disclosure. Spontaneous co-speech gestures, while being indeed spontaneous,
> are nonetheless informed in various ways by conventions that they
> appropriate and deploy. Through this appropriation and deployment speakers
> enact, rather than represent, meaning, and they do so in various linguistic
> modalities. Seen thusly, gestures provide philosophers with a unique new
> perspective on the paradoxical determined-yet-free nature of all human
> meaning.
>
> vi
>
> CURRICULUM VITAE
>
> NAME OF AUTHOR: Elena Clare Cuffari
>
> GRADUATE AND UNDERGRADUATE SCHOOLS ATTENDED:
>
> University of Oregon, Eugene, OR
>
> Swarthmore College, Swarthmore, PA
>
> Oxford University, Oxford, England, UK
>
> DEGREES AWARDED:
>
> Doctor of Philosophy, Philosophy, 2011, University of Oregon
>
>
> On Tue, Apr 8, 2014 at 9:14 AM, Ed Wall <ewall@umich.edu> wrote:
>
>> There is a video with Elena Bodorva (I think titled Tools of the Mind)
>> that shows a child refining (with instruction his pointing as a means to
>> competently perform ordinal arithmetic. There is (or perhaps should be) a
>> lot of gesturing in early mathematics.
>>
>> Ed
>>
>> On Apr 8, 2014, at  6:24 AM, Rémi A. van Compernolle wrote:
>>
>> > David, Alex:
>> >
>> > Nice clip and commentaries. I wonder, though, what the singer's gestures
>> would have looked like as she was first learning the piece, and how their
>> qualities might have changed as she practiced. This is obviously of
>> interest to Vygotskians (genetic method, right?), and it has been the
>> object of some inquiry in education (e.g., Goldin-Meadow's work) as well as
>> in second language learning (including Alex's work).
>> >
>> > I think this might be something for your students to explore in their
>> investigations of elementary children learning to perform dialogues, David,
>> i.e. not just focusing on how much or how little gestures are used and
>> their relation with prosody but how they might develop, from initial
>> learning through to some kind of "competent" performance (whatever that
>> might mean). Also, could the kids be taught to gesture as a means of
>> mediating dialogue performance?
>> >
>> > Adam
>> >
>> > Rémi A. van Compernolle
>> > Assistant Professor of Second Language Acquisition & French and
>> Francophone Studies
>> > Department of Modern Languages
>> > Carnegie Mellon University
>> > Baker Hall A60M
>> > 412-268-1122
>> >
>> >
>> >
>> > On Apr 7, 2014, at 6:23 PM, Alex Rosborough <alex_rosborough@byu.edu>
>> wrote:
>> >
>> >> David,
>> >>
>> >> Thank you for sharing this and past posts. I'd like to preface this with
>> >> a, "long time listener, first time caller" type of statement... as I was
>> >> only recently (re)added to the list after a multiyear recess.
>> >>
>> >> I didn't take a deep look at the video but with slow down motion, I
>> think
>> >> your graduate student will see that there are a variety of gestures that
>> >> occur "at the same time" in sync with the conductor (as well as with her
>> >> rhythmic needs) - analogue style, right? So in many cases she IS hitting
>> >> rhythmic beats for HER notes... Look at the final or most important
>> >> "strokes" as David McNeill has termed them. She goes high for intonation
>> >> but then fingers or hand will often hit the most important point
>> (rhythmic
>> >> point) of the phrase (while arm is extended). This is often coupled
>> with a
>> >> "beat" stroke too. Sometimes, she does coordinate with the conductor but
>> >> we can't see the final lower stroke because of the limitations of the
>> >> camera... Unfortunately, the camera person does not seem too intent in
>> >> capturing the entire corporeal manifestation of the singing. What I'm
>> >> quite certain about is that during those down hand beat motions, she and
>> >> the conductor would be coordinated at their stop within fractions of a
>> >> second... We wouldn't be able to tell where they differed unless we
>> break
>> >> it down past 1/10 - 1/40 of a second but other data analysis has shown
>> >> that they would have looked very coordinated, especially in those
>> downward
>> >> strokes.
>> >>
>> >> So interestingly, it possible that she may be presenting purpose and
>> >> communication for multiple reasons using multiple modalities in an
>> >> embodied and prosodic way (intertwined); including melodic and rhythmic
>> >> expressions as well as the public/private message ensemble - nothing new
>> >> for Vygotsky people :) So her "conflicts" are purposeful as well in
>> >> meeting her needs and are interesting to study - as you noted. I agree
>> >> with you and your grad. Student's observations of the el. ed. children.
>> >>
>> >> Just some brainstorming - thanks for sharing.
>> >>
>> >> Alex Rosborough
>> >> Brigham Young University
>> >>
>> >>
>> >>
>> >> On 4/7/14 3:29 PM, "David Kellogg" <dkellogg60@gmail.com> wrote:
>> >>
>> >>> One of my graduate students has been studying the way in which
>> >>> children use their hands when they speak English.
>> >>>
>> >>> We started with the observation that fourth and fifth graders will
>> >>> gesture copiously when they speak, but fifth and sixth graders, when
>> >>> you ask them to perform a dialogue in front of the class, will come
>> >>> out and lock their hands behind their backs, apparently to prevent
>> >>> themselves from gesticulating. Sure enough, their delivery is far more
>> >>> flat in intonation. When we ask them to unlock their hands, there is a
>> >>> notable improvement in intonation.
>> >>>
>> >>> This morning I was looking at Natalie Dessaye rehearsing the mad scene
>> >>> from "Lucia de Lammermoor". She gesticulates a LOT. But you can see
>> >>> that her gesticulations are not at all mad--when her voice has to go
>> >>> high, she puts her hand way over her head. When she has to go low, she
>> >>> places her hand low.
>> >>>
>> >>> Her "conducting" actually conflicts with that of the conductor,
>> >>> because of course it's melodic and not rhythmic. But it's effective;
>> >>> it produces that exquisite sense of "bloom" in her high notes. It's
>> >>> not exactly what my students are doing, but it's close!
>> >>>
>> >>> https://www.youtube.com/watch?v=JlVKw3_VXv4
>> >>>
>> >>> David Kellogg
>> >>> Hankuk University of Foreign Studies
>> >>>
>> >>>
>> >>>
>> >>> .
>> >>
>> >>
>> >
>> >
>>
>>
>>