Popular in Social Sciences

Books

City Time: On Being Sentenced to Rikers Island

Independent Scholar David A. Campbell

Culture and Society

The City as a Growth Machine: Toward a Political Economy of Place

New York University Harvey Moltoch

Economics

Gender and Economics

University of California Santa Barbara Shelly Lundberg

American Politics

Keeping the Republic

Boston College Marc Landy

Banks and Policy Organisations

Greening the financial sector

European Investment Bank Matteo Ferrazzi

British Politics

Entrepreneurial thinking: the politics and practice of policy impact

University of Sheffield Matthew Flinders

Culture and Society

Carceral Entrapments: Views from the Prison/Street Interface in India

Indian Institute of Technology Delhi Mahuya Bandyopadhya

Gesture

View more insights in Social Sciences - Anthropology

View more figures from Indiana University

View the page of Erica Cartmill

Read the Study

Gesture is intimately entwined with human language and thought. It is a tool for communication as well as cognition: conveying information to interlocutors, orchestrating interaction, and supporting problem-solving and learning. Over the past 25 years, the community of scholars interested in gesture has grown from a specialized group to a multidisciplinary community incorporating gesture into a wide range of topics. Erica Cartmill discusses the most intriguing findings and questions in gesture research.

Image courtesy of interviewee. April 24, 2024

Generating Workspace

Export as PDF

This video video is a transcript of a discussion about gesture studies, emphasizing the speaker's background, ongoing research, and the relationship between gesture and language. The speaker clarifies that they have not yet written a book, although they have extensively researched gestures, particularly in adults, children, and great apes.

Key points include:

1. **Definitions of Gesture**: Gesture is defined as movements of the hands, arms, and head that occur in the context of conversation. The definition varies depending on the context, especially when considering non-verbal animals like great apes and very young children who may use gestures in lieu of speech.

2. **Intentionality in Gestures**: The speaker discusses how to determine intentionality behind gestures, which involves observing whether gestures are directed towards others and if there is an expectation of response.

3. **Types of Gestures**: Different types of gestures are described:
- **Representational Gestures**: Where the movement resembles the object it references (e.g., mimicking holding a cup).
- **Conventional Gestures**: Symbolic gestures agreed upon by a community (e.g., a thumbs-up).

4. **Interaction Between Gesture and Speech**: The relationship is described as intertwined, with gestures enhancing, disambiguating, or complementing speech. Evidence shows when one system fails, the other often does too.

5. **Gestures in Learning**: Gestures are not only for communication but also aid in cognitive processes and problem-solving for learners. They help in memory retention and can enhance understanding.

6. **Evolution of Gesture and Language**: The speaker suggests that earlier forms of language may have evolved through gestures before the development of spoken language and posits that gestures may have been the primary mode of communication in early human ancestors.

Overall, the discussion outlines the significance of gestures in communication, cognition, and language evolution, while highlighting the differences in gestural use across species and ages.

Language:

Export as PDF

X-TIMESTAMP-MAP:LOCAL=00:00:00.000,MPEGTS=900000

00:00.145 --> 00:01.245
So it's not a book.

00:01.445 --> 00:02.685
I haven't written a book yet.

00:03.145 --> 00:07.845
Um, so I think that that maybe the, I mean,

00:07.845 --> 00:09.685
I've written several pieces on gesture.

00:10.185 --> 00:14.085
Um, I mean, it's something I've studied in my, you know,

00:14.085 --> 00:15.325
in my lab for a while,

00:15.475 --> 00:18.125
something I worked on in my postdoc as well as my PhD.

00:18.905 --> 00:22.045
Um, but I haven't written a book like manuscript yet.

00:22.145 --> 00:23.805
So I don't know if you wanna like,

00:23.805 --> 00:25.285
change the introduction a little bit.

00:25.305 --> 00:29.825
You could just say, my work jester is broadly.

00:30.685 --> 00:32.905
Uh, so the field

00:32.905 --> 00:35.905
of gesture studies has actually grown quite a bit over the

00:35.905 --> 00:37.905
last maybe 20 years.

00:38.685 --> 00:40.385
And it really depends.

00:40.445 --> 00:43.465
The precise definition of gesture really depends on

00:43.525 --> 00:44.665
who you're talking to.

00:45.125 --> 00:46.425
Um, there's sort of different schools

00:46.445 --> 00:49.025
of thought when it comes to thinking about gesture.

00:49.985 --> 00:53.065
I like to have to take a definition of gesture that

00:53.685 --> 00:55.665
is movements of the hand and arms

00:56.525 --> 00:58.585
and head, that occurred

00:58.725 --> 01:00.505
during the course of conversation.

01:01.165 --> 01:04.905
So it's, that is a, I think a pretty good definition,

01:05.065 --> 01:08.425
a pretty encompassing definition of what we refer to

01:08.445 --> 01:09.705
as Coe gesture.

01:10.205 --> 01:13.145
So these are our gestures that occur within the context

01:13.285 --> 01:15.545
of speech or surrounding speech,

01:15.755 --> 01:18.465
which works really well if you're studying the,

01:18.565 --> 01:19.705
the hand movements

01:19.925 --> 01:23.625
and bodily communication of adults and of humans.

01:24.125 --> 01:25.905
Um, but I also work with children

01:26.245 --> 01:27.545
and I work with great apes,

01:27.765 --> 01:29.865
and I think there that definition doesn't work

01:29.865 --> 01:31.345
as well with great apes.

01:32.095 --> 01:34.465
They're not speaking, but they still use gesture.

01:34.965 --> 01:38.265
And with very young children, gestures might take the place

01:38.365 --> 01:40.545
of speech rather than accompany speech.

01:40.725 --> 01:43.905
So it's, I think when you're working on with gesture,

01:44.095 --> 01:45.105
when you're studying gesture,

01:45.245 --> 01:49.905
but you're studying it in creatures outside of adult humans,

01:50.455 --> 01:54.025
then you need to, to take a slightly different definition.

01:54.285 --> 01:57.105
So, for example, with children,

01:57.845 --> 02:01.785
it might not be that gestures are cos speech, right?

02:01.785 --> 02:03.825
Which imply that that gestures

02:03.845 --> 02:04.945
and speech are coming together.

02:05.335 --> 02:10.305
Instead, it might be that children are communicating to,

02:10.305 --> 02:12.385
to say an adult or to another child.

02:13.045 --> 02:16.545
And it might be that they're using vocalizations as well

02:16.545 --> 02:19.505
as gestures, but they're not using speech per se.

02:19.645 --> 02:21.065
It depends on the age of the child.

02:21.855 --> 02:24.105
Well, with great apes, which a lot

02:24.105 --> 02:26.705
of my work has really focused on the great apes,

02:26.935 --> 02:30.065
chimpanzees, bonobos, gorillas and orangutans,

02:30.485 --> 02:32.145
and humans we're also a great ape,

02:32.645 --> 02:34.305
but the non-human great apes.

02:34.725 --> 02:38.705
Um, when it comes to looking at gesture there, we have

02:38.705 --> 02:40.945
to again, sort of go back to the drawing board

02:41.005 --> 02:43.665
and think, okay, what does it mean to use gesture

02:44.215 --> 02:46.945
when you don't have a spoken language?

02:47.565 --> 02:50.065
So apes are, use a lot of gesture

02:50.125 --> 02:51.425
to communicate with one another.

02:51.925 --> 02:55.185
Um, but they're doing it not within the context

02:55.365 --> 02:56.945
of a conversation or

02:56.945 --> 02:58.825
with a spoken conversation. They're doing

02:58.825 --> 02:59.965
It,

03:00.095 --> 03:03.605
where gesture is often the primary medium of communication.

03:04.345 --> 03:08.565
And so there we use a set of, of criteria

03:09.195 --> 03:11.685
looking for evidence of intentionality.

03:12.025 --> 03:13.845
So where they intending to communicate.

03:14.385 --> 03:17.725
And there we're actually u sort of borrowing

03:17.725 --> 03:21.885
and modifying, criteria from

03:22.415 --> 03:24.325
early child communication, right?

03:24.325 --> 03:26.445
You have the same problem if you're studying a 1-year-old

03:26.665 --> 03:28.285
or you're studying an orangutan,

03:28.545 --> 03:32.125
and we don't know, sort of what movements they're making

03:32.125 --> 03:34.005
with their body are communicative

03:34.345 --> 03:35.645
and which ones are not, right?

03:35.645 --> 03:36.685
So it's, you have

03:36.685 --> 03:41.445
to really use evidence from the interaction to argue

03:41.715 --> 03:43.405
that the movement that they're making is

03:43.405 --> 03:44.565
not accidental, right?

03:44.565 --> 03:46.325
They don't just go shoo,

03:47.345 --> 03:50.045
or, you know, they're not just shooing away a bug.

03:50.235 --> 03:52.165
They're actually waving to get attention.

03:52.785 --> 03:56.325
And so the way that you do that is to look for evidence

03:56.355 --> 03:58.805
that they are intending to communicate.

03:59.265 --> 04:02.445
And so this is sort of brings in all of this philosophy, all

04:02.445 --> 04:04.325
of these, you know, it's a, it's a big question, right?

04:04.345 --> 04:05.445
How do you measure intention?

04:06.145 --> 04:08.725
And, you know, that's probably longer than we have today.

04:08.865 --> 04:12.925
But just in brief, what, what we use as,

04:13.345 --> 04:17.125
as evidence for intentionality is that,

04:17.285 --> 04:19.445
a gesture is directed towards another individual.

04:20.105 --> 04:21.325
So if I'm waving my hand,

04:21.385 --> 04:23.565
I'm gonna be focused towards you when I do it.

04:24.495 --> 04:28.105
That, there's expectation of a response

04:28.525 --> 04:33.025
and evidence for that can be several different directions.

04:33.205 --> 04:37.265
So it could be that I wait for a response, so I face you,

04:37.585 --> 04:39.345
I wave my hand and that I wait.

04:40.385 --> 04:43.165
Or it could be I face you, I wave my hand.

04:43.465 --> 04:46.525
If you don't respond, then I persist.

04:46.845 --> 04:50.005
I keep communicating and I change the way I communicate.

04:50.065 --> 04:53.165
So I might slow down, you know, I go from hey to,

04:55.095 --> 04:58.315
or I might change to a different, to a different signal.

04:58.455 --> 05:01.115
So I might start with, you know, waving my hand

05:01.135 --> 05:04.355
and then clap to get your attention, wave bigger,

05:04.745 --> 05:06.515
wave both hands, something like that.

05:06.575 --> 05:10.715
So you're seeing this, this persistence and this elaboration

05:10.735 --> 05:14.635
or escalation of the signal and waiting for a response.

05:15.055 --> 05:17.915
So there's, there's evidence that I'm communicating

05:17.915 --> 05:19.075
to a particular individual.

05:19.995 --> 05:22.475
I am doing it with an expectation

05:22.535 --> 05:24.235
of getting a particular kind of response.

05:24.295 --> 05:26.875
And so that's what we use as evidence and intentionality.

05:28.095 --> 05:31.805
So, so representational gestures really

05:32.495 --> 05:37.485
refer more to the, the kinds of gestures where,

05:38.435 --> 05:42.445
they, they have meaning through the movements that they're,

05:42.785 --> 05:46.685
that you're making with your hands, rather than,

05:47.355 --> 05:49.565
than, through the thing that you're gesturing to.

05:50.105 --> 05:53.205
So for example, you know, if I hold up a cup

05:53.545 --> 05:55.245
and I point to the cup, we describe

05:55.245 --> 05:56.325
that as a dy dick gesture.

05:56.745 --> 05:58.245
It can also have a diic gesture

05:58.265 --> 05:59.885
of just holding something up, right?

05:59.915 --> 06:01.605
It's directing attention to something.

06:02.145 --> 06:03.805
And we typically don't describe those

06:03.805 --> 06:05.285
as representational gestures

06:05.285 --> 06:07.845
because the meaning is

06:08.445 --> 06:11.205
anchored in the object itself rather than in the gesture.

06:11.305 --> 06:13.445
The gesture is just directing attention towards that.

06:14.605 --> 06:18.065
Um, representational gestures would encompass things like

06:18.845 --> 06:23.425
iconic gestures where the movement of the jester resembles

06:24.405 --> 06:25.505
the, the reference.

06:25.725 --> 06:28.945
So for example, with cup, I could use my hands

06:29.125 --> 06:30.345
to show the shape of the cup,

06:31.005 --> 06:33.825
or I could hold an imaginary cup, right?

06:33.845 --> 06:36.665
So it could be I'm holding like more of a glass of water,

06:36.685 --> 06:37.785
or I'm holding a mug

06:38.245 --> 06:40.545
and I could, you know, pretend to drink out of it,

06:40.925 --> 06:43.025
or I could trace, you know, the circle

06:43.445 --> 06:45.145
and the, the handle of the mug

06:45.375 --> 06:49.305
that would be representing the object through,

06:49.395 --> 06:50.785
similarity in form.

06:51.245 --> 06:54.225
And sometimes in function, if I'm illustrating the function,

06:54.925 --> 06:58.315
now, ity can be quite complicated.

06:58.335 --> 06:59.395
So it seems really simple.

06:59.455 --> 07:01.235
You look at it and you're like, the movements

07:01.235 --> 07:04.595
that you're making map onto the thing that, that you mean.

07:05.835 --> 07:09.855
But there's actually, evidence that, that ity can is,

07:10.115 --> 07:13.175
is more complicated for children and for apes.

07:13.515 --> 07:17.095
So for adults, we see something just like playing charades.

07:17.095 --> 07:18.855
It's like, you're acting something out.

07:19.175 --> 07:21.055
I get that, that your movement is,

07:21.195 --> 07:22.935
is representing something else, right?

07:22.935 --> 07:24.415
It's a representational gesture.

07:25.035 --> 07:28.575
Um, but for, for young chil very young children,

07:28.875 --> 07:30.775
and for apes, it can be more complicated

07:30.775 --> 07:34.615
because you have to, you have to understand what features

07:34.715 --> 07:36.575
of the movement are important,

07:37.075 --> 07:38.615
and you have to understand the analogy

07:38.645 --> 07:40.695
between those features of the movement

07:40.915 --> 07:42.335
and the features of the real world.

07:42.875 --> 07:46.455
For example, if I was going to to, to, you know,

07:46.455 --> 07:50.375
gesture about a tree, I could have, you know, the trunk,

07:50.615 --> 07:52.015
I could gesture the branches,

07:52.415 --> 07:54.655
I could gesture the tree waving in the wind.

07:55.135 --> 07:56.695
I could gesture about all the leaves.

07:57.015 --> 08:00.815
I could do a, you know, a palm tree versus a big oak.

08:01.435 --> 08:03.575
Um, there, you know, I could talk about climbing the tree.

08:04.115 --> 08:08.775
And within, within a conversational context, it's,

08:08.955 --> 08:10.615
it can really make a lot of sense, right?

08:10.615 --> 08:14.255
It can be very helpful. Um, representational gestures can,

08:14.355 --> 08:16.855
can kind of provide a lot of quality

08:16.955 --> 08:19.895
and depth to, you know, if I just say tree, you don't know

08:19.895 --> 08:21.815
what kind of tree I'm talking about, but if I gesture about

08:21.815 --> 08:23.455
the tree, you learn something about it.

08:23.715 --> 08:27.295
Its size, its shape, its placement, how it moves

08:28.115 --> 08:30.655
the relationship I have to it, you know, is it over there?

08:30.755 --> 08:32.975
Is it over here? Is it big? Is it small?

08:33.155 --> 08:35.815
Am I in the tree itself? Is the, am i under the tree?

08:36.355 --> 08:40.655
Um, and so, so representational gestures can provide all of

08:40.655 --> 08:44.335
that richness, but if you just have one in isolation,

08:44.835 --> 08:48.415
it can be pretty, it's a pretty difficult task actually, to,

08:48.475 --> 08:52.975
to figure out what parts of the gesture are, are mapping on

08:53.195 --> 08:54.295
to the thing in the real world.

08:55.355 --> 08:57.815
So the, the example of the tree i

08:57.845 --> 09:00.095
that I gave was really about iconic gesture.

09:00.285 --> 09:02.375
It's where the, the movement of the hands

09:03.365 --> 09:04.775
maps onto the features

09:04.775 --> 09:06.575
of something in the, in the real world.

09:07.165 --> 09:10.655
There's another set of, of gestures which are, are referred

09:10.655 --> 09:12.615
to either as conventional gestures

09:12.755 --> 09:16.135
or sometimes as emblems, where they're really symbolic.

09:16.555 --> 09:18.415
And so there, the, the form

09:18.415 --> 09:20.095
of the gesture doesn't resemble

09:20.095 --> 09:21.295
the thing that it stands for.

09:22.155 --> 09:24.095
It it only means that thing

09:24.095 --> 09:26.895
because we as a community agree that it does.

09:27.235 --> 09:30.175
So the community can be the entire world,

09:30.595 --> 09:33.935
or the community can be, you know, a city or a family

09:34.435 --> 09:36.375
or even just one person, right?

09:36.375 --> 09:39.095
So you can have a secret handshake, say with a friend

09:39.155 --> 09:40.815
or a secret, you know, hand symbol

09:41.005 --> 09:42.535
that you agree means something

09:42.795 --> 09:44.775
and outside, you know, people who looking from the

09:44.775 --> 09:45.935
outside wouldn't understand that.

09:46.195 --> 09:49.415
And so we do see that those kinds of gestures can vary.

09:49.435 --> 09:51.255
You know, they vary cross-culturally,

09:51.325 --> 09:55.015
they're actually more similar across cultures than,

09:55.045 --> 09:59.015
than you would expect given that, that they have to sort of,

09:59.015 --> 10:00.015
they have this meaning

10:00.015 --> 10:02.895
because people agree that they mean something, you know,

10:02.895 --> 10:04.735
but think about, you know, the thumbs up gesture.

10:05.285 --> 10:07.895
Okay? Um, things like that.

10:08.155 --> 10:11.455
Uh, one of my, my favorite gestures is,

10:11.475 --> 10:13.095
so I lived in Poland for a few years,

10:13.155 --> 10:16.455
and one of my favorite gestures there is this, you know,

10:16.455 --> 10:17.775
like tap, tap of the neck.

10:18.155 --> 10:22.655
And I, like in the US I it, when I ask, say,

10:22.775 --> 10:24.135
students in the classroom, what do you think this means?

10:24.155 --> 10:25.495
And they'll say like, cut it

10:25.495 --> 10:26.655
out, or something like that.

10:27.195 --> 10:29.175
In Poland, at least in the part of Poland

10:29.175 --> 10:31.135
where I lived, it means drunk.

10:31.405 --> 10:33.175
Like you're filled up to here.

10:33.285 --> 10:34.855
Like, they were really drunk.

10:35.435 --> 10:37.615
And you also think about the kinds of rude gestures.

10:37.915 --> 10:40.575
I'm not gonna do them here, but those I think are really

10:40.575 --> 10:45.295
nice examples of these symbolic conventional gestures

10:45.365 --> 10:46.815
that vary from place to place.

10:48.485 --> 10:52.705
So, to come, to come back to your question about,

10:54.705 --> 10:56.685
about how jester works with speech.

10:57.105 --> 11:01.025
So there are, there are different ways

11:01.025 --> 11:03.745
of thinking about the relationship between JE and speech.

11:04.405 --> 11:08.055
Um, you know, I think that maybe the,

11:08.055 --> 11:11.015
the most comprehensive one is to simply say that

11:11.045 --> 11:14.455
that gesture and speech together form a unified system.

11:14.915 --> 11:17.495
So there's really good evidence that there's a tight,

11:18.285 --> 11:20.735
deep rooted relationship between gesture and speech.

11:20.765 --> 11:23.495
It's not just that we speak and we gesture

11:23.795 --> 11:26.815
and we, we happen to do both at the same time, right?

11:26.815 --> 11:29.095
They really, they, they are,

11:29.845 --> 11:32.535
sometimes I say intimately entwined, right?

11:32.535 --> 11:34.135
They're really tightly integrated.

11:34.715 --> 11:37.495
Um, if you prevent people from gesturing,

11:37.635 --> 11:39.855
say if you have them sit on their hands or,

11:40.395 --> 11:43.295
or, you put their hands in gloves

11:43.315 --> 11:45.735
and you velcro them to a table, you'll see

11:45.735 --> 11:48.295
that their speech will show more disfluencies.

11:48.635 --> 11:50.135
So they'll say, and,

11:50.135 --> 11:52.375
they'll have a harder time recalling words

11:53.105 --> 11:54.925
And, and they won't speak as fluently

11:54.925 --> 11:56.005
and they won't speak as long.

11:56.585 --> 11:59.085
And, you see the same thing happen when you

11:59.285 --> 12:00.365
restrict, speech.

12:00.505 --> 12:02.765
So in the case of stuttering,

12:02.765 --> 12:04.485
there's really nice evidence from

12:05.125 --> 12:06.845
researchers working on stuttering that

12:06.845 --> 12:09.845
during stuttering gesture is also interrupted.

12:10.145 --> 12:14.065
So if they were separate systems, then you would expect

12:14.065 --> 12:16.785
that if one system fails, the other system would take over.

12:17.055 --> 12:19.825
There's actually evidence that when one system is impaired,

12:20.045 --> 12:21.865
it, the other system is also impaired.

12:22.445 --> 12:24.385
And so, you know, there, there's evidence from that.

12:24.385 --> 12:27.865
There's evidence from neuroimaging, and,

12:28.365 --> 12:32.625
and EEG, that, that there is actually, sort

12:32.625 --> 12:36.545
of simultaneous processing of both speech and of jester.

12:36.725 --> 12:38.585
And so I think one of the things that's really interesting

12:38.585 --> 12:41.625
about jester is that it's part of language,

12:42.205 --> 12:44.025
but it's different than speech.

12:44.085 --> 12:47.945
So speech has really specific rules, you know,

12:49.265 --> 12:51.345
different rules depending on the language that you speak,

12:51.645 --> 12:55.105
but every, every language has rules, has grammatical rules,

12:55.685 --> 12:59.585
that, you know, that really tightly constrain,

12:59.925 --> 13:04.185
speech and in a way that, that some things are grammatical,

13:04.185 --> 13:06.385
some things are not grammatical, you know, and,

13:06.405 --> 13:09.825
and it's, it's much easier to sort of predict,

13:10.215 --> 13:12.145
what is allowable in speech than

13:12.145 --> 13:13.345
what is allowable in gesture,

13:13.345 --> 13:15.305
because gesture is much more wide open.

13:15.965 --> 13:19.585
And so gesture almost has sort of an analog.

13:19.605 --> 13:21.305
If you think about speech as being more digital,

13:21.685 --> 13:24.785
one thing at a time in a row, following rules,

13:25.825 --> 13:26.905
gesture allows us

13:27.005 --> 13:30.545
to communicate in a more analog holistic manner.

13:31.005 --> 13:33.985
Um, it it providing different information

13:34.495 --> 13:35.865
from what's happening in speech.

13:36.125 --> 13:39.505
And so gesture can reinforce what's happening in speech.

13:39.845 --> 13:42.265
It can disambiguate what's happening in speech, you know,

13:42.265 --> 13:44.145
so if I say, I want that one.

13:45.155 --> 13:46.575
If I just say, I want that one,

13:47.535 --> 13:48.795
who the heck knows what that is?

13:48.965 --> 13:52.355
Right? Almost always when we use something like this

13:52.455 --> 13:55.275
or that here or there in speech, we're gesturing.

13:56.325 --> 13:58.065
And so you can disambiguate

13:58.065 --> 13:59.145
what's what you're saying in speech.

13:59.525 --> 14:01.905
You can also add to what you're saying in speech.

14:02.165 --> 14:03.425
So sometimes what gesture,

14:03.765 --> 14:07.585
the information gesture is providing is complimentary,

14:07.645 --> 14:09.985
but different from the information in speech.

14:10.525 --> 14:13.345
And so thinking about those relationships,

14:13.565 --> 14:15.105
and what information is being

14:15.545 --> 14:17.265
provided in speech, what information is being

14:17.505 --> 14:18.865
provided in gesture and,

14:19.085 --> 14:23.825
and, how the, the sum of both of those modalities

14:24.355 --> 14:27.025
gives a fuller pitch picture of communication.

14:27.525 --> 14:29.945
Um, you know, I think is a very interesting question.

14:30.835 --> 14:31.955
I think I'm maybe gonna talk more

14:31.955 --> 14:33.075
about the evolution of language.

14:33.235 --> 14:34.475
'cause that's what I work on.

14:34.615 --> 14:36.235
Um, I don't really work on gesture and learning,

14:36.475 --> 14:37.755
although there's beautiful research on that.

14:38.535 --> 14:43.165
Um, so I think there's some really nice evidence

14:43.595 --> 14:48.285
that that learners who are approaching new problems,

14:48.745 --> 14:50.885
can use their gestures in ways

14:51.115 --> 14:53.725
that help them understand a problem better.

14:54.265 --> 14:56.725
So, it was off, there's a great body

14:56.725 --> 15:00.205
of literature on gesture for thinking as well

15:00.205 --> 15:01.445
as gesture for communication.

15:01.745 --> 15:03.485
And so a lot of the things that we do

15:03.485 --> 15:05.845
with our hands while we're talking are not just

15:05.865 --> 15:08.965
for communicating to other people, they're also for us.

15:09.665 --> 15:12.845
And so by putting, you're almost putting ideas out there

15:12.945 --> 15:15.485
and then returning to them, you, you know,

15:15.485 --> 15:17.725
you can draw things just as you were,

15:17.725 --> 15:19.645
as if you were sketching something on paper.

15:19.745 --> 15:21.125
You can draw things with your hands,

15:21.585 --> 15:23.045
you can think you can rotate them,

15:23.065 --> 15:24.565
you can think about them in different ways.

15:24.745 --> 15:28.085
It provides an opportunity to externalize some

15:28.205 --> 15:29.805
of our internal thought processes.

15:30.425 --> 15:33.685
And, and there's, you know, there's evidence from,

15:33.955 --> 15:37.685
from experimental work that shows that that actually helps

15:38.195 --> 15:40.765
with learning certain kinds of problems with working

15:40.765 --> 15:44.165
through more difficult problems, even if you understand,

15:44.465 --> 15:46.685
you know, sort of a basic level of a problem.

15:47.385 --> 15:51.965
And with, with helping in memory tasks so you're, can,

15:52.145 --> 15:54.925
can be a resource to support what we call working memory

15:55.705 --> 15:57.725
is the kind of memory that you need when you're say,

15:58.125 --> 16:00.325
remembering how to answer a question someone just asks.

16:00.825 --> 16:04.845
So, so, we know that

16:05.035 --> 16:06.045
that gestures

16:06.075 --> 16:08.765
that learners themselves make can help them learn.

16:09.305 --> 16:12.165
We also know that learners are sensitive to the gestures

16:12.165 --> 16:13.205
that instructors make.

16:13.665 --> 16:18.445
And so there are people working on, on trying to,

16:18.625 --> 16:21.645
to design better, better lessons

16:22.235 --> 16:26.365
that involve gesture as one, as one possible visual aid.

16:28.265 --> 16:31.445
And then coming to the question of gesture in the evolution

16:31.445 --> 16:34.325
of language, this is, possibly my favorite question.

16:34.385 --> 16:36.205
So I work on the evolution of language

16:36.345 --> 16:38.445
and on je, so I love the intersection of these.

16:38.945 --> 16:43.165
Um, there's, there are theories

16:43.165 --> 16:47.005
that jester actually played a much bigger role early in,

16:47.005 --> 16:48.365
earlier in human evolution.

16:48.585 --> 16:52.525
So when we look at the, at the existing great apes,

16:52.825 --> 16:56.005
humans, orangutans, chimpanzees, bonobos,

16:56.005 --> 16:57.845
and gorillas, all of them jester

16:58.355 --> 17:00.445
only humans have spoken language,

17:01.545 --> 17:04.645
but, the other apes have vocalizations,

17:04.945 --> 17:08.005
but their gestures seem to, to be more similar

17:08.065 --> 17:10.285
to human gesture than their vocalizations are

17:10.285 --> 17:11.405
similar to human speech.

17:12.345 --> 17:16.605
And so one possibility is that the, the gestural

17:17.445 --> 17:20.805
communication of our last common ancestor we're talking 12,

17:21.125 --> 17:24.325
13 million years ago, that for that species

17:25.565 --> 17:29.125
jester was really, sort of the most important modality.

17:29.665 --> 17:33.605
And that, as you know, as the great apes evolved to,

17:33.825 --> 17:37.405
you know, to the present era, je stayed really important.

17:37.945 --> 17:42.725
And that when, when humans and chimpanzees and bonobos,

17:43.465 --> 17:46.605
and the lineages that led to chimpanzees

17:46.605 --> 17:47.765
and bonobos on the one hand,

17:47.825 --> 17:48.965
and to humans on the other hand,

17:49.235 --> 17:52.805
when those lineages diverged, about, you know, a,

17:52.885 --> 17:57.285
a few million years ago, that jester

17:57.825 --> 18:00.405
was, was the most important modality

18:00.905 --> 18:02.885
for our last common ancestor.

18:03.185 --> 18:06.525
So if jester was fact the most important modality,

18:07.885 --> 18:10.535
then it could be the case that

18:11.855 --> 18:13.975
language actually evolved first within

18:14.035 --> 18:15.215
the gestural modality.

18:15.635 --> 18:17.335
Gester shows more flexibility.

18:17.875 --> 18:21.495
Um, it great, you know, in great apes in human children.

18:21.955 --> 18:25.695
Um, it's, it's really, it's the most important modality.

18:26.515 --> 18:28.735
And, and there are a bunch of,

18:28.795 --> 18:31.655
of theories pulling evidence from different areas, some

18:31.655 --> 18:35.655
of which I've discussed today, that suggest that early

18:36.315 --> 18:38.695
in human evolutionary history, you know,

18:38.695 --> 18:42.695
over the last million years, let's say 500,000 years, that,

18:42.995 --> 18:46.895
you know, that gesture may have, you know,

18:46.895 --> 18:48.015
really been there first.

18:48.475 --> 18:51.015
And that speech is, was a later development.

18:51.155 --> 18:55.095
And so I, I'm not saying that language evolved fully

18:55.155 --> 18:56.255
as sign language, right?

18:56.255 --> 18:58.935
So the sign languages that we see in humans today are,

18:59.115 --> 19:02.975
are full, structured languages, just

19:02.975 --> 19:04.375
as any spoken language is.

19:05.075 --> 19:06.455
Not saying that, you know,

19:06.755 --> 19:09.775
our evolutionary ancestors had sign language,

19:09.995 --> 19:12.895
but they may have had more complicated gestural systems

19:13.445 --> 19:18.375
that in fact, laid the foundation for the kinds of,

19:19.195 --> 19:21.975
of highly structured grammatical systems

19:22.245 --> 19:25.175
that you see in both spoken and in sign languages today.

00:00 - So it's not a book. I haven't written a book yet. so I think that that maybe the, I mean, I've written several pieces on gesture. I mean, it's something I've studied in my, you know, in my lab for a while,

00:15 - something I worked on in my postdoc as well as my PhD. but I haven't written a book like manuscript yet. So I don't know if you wanna like, change the introduction a little bit. You could just say, my work jester is broadly. so the field of gesture studies has actually grown quite a bit over the last maybe 20 years. And it really depends. The precise definition of gesture really depends on

00:43 - who you're talking to. there's sort of different schools of thought when it comes to thinking about gesture. I like to have to take a definition of gesture that is movements of the hand and arms and head, that occurred during the course of conversation. So it's, that is a, I think a pretty good definition, a pretty encompassing definition of what we refer to as Coe gesture.

01:10 - So these are our gestures that occur within the context of speech or surrounding speech, which works really well if you're studying the, the hand movements and bodily communication of adults and of humans. but I also work with children and I work with great apes, and I think there that definition doesn't work as well with great apes. They're not speaking, but they still use gesture.

01:34 - And with very young children, gestures might take the place of speech rather than accompany speech. So it's, I think when you're working on with gesture, when you're studying gesture, but you're studying it in creatures outside of adult humans, then you need to, to take a slightly different definition. So, for example, with children, it might not be that gestures are cos speech, right? Which imply that that gestures and speech are coming together.

02:05 - Instead, it might be that children are communicating to, to say an adult or to another child. And it might be that they're using vocalizations as well as gestures, but they're not using speech per se. It depends on the age of the child. Well, with great apes, which a lot of my work has really focused on the great apes, chimpanzees, bonobos, gorillas and orangutans, and humans we're also a great ape, but the non-human great apes.

02:34 - when it comes to looking at gesture there, we have to again, sort of go back to the drawing board and think, okay, what does it mean to use gesture when you don't have a spoken language? So apes are, use a lot of gesture to communicate with one another. but they're doing it not within the context of a conversation or with a spoken conversation. They're doing It,

03:00 - where gesture is often the primary medium of communication. And so there we use a set of, of criteria looking for evidence of intentionality. So where they intending to communicate. And there we're actually u sort of borrowing and modifying, criteria from early child communication, right? You have the same problem if you're studying a 1-year-old or you're studying an orangutan, and we don't know, sort of what movements they're making

03:32 - with their body are communicative and which ones are not, right? So it's, you have to really use evidence from the interaction to argue that the movement that they're making is not accidental, right? They don't just go shoo, or, you know, they're not just shooing away a bug. They're actually waving to get attention. And so the way that you do that is to look for evidence

03:56 - that they are intending to communicate. And so this is sort of brings in all of this philosophy, all of these, you know, it's a, it's a big question, right? How do you measure intention? And, you know, that's probably longer than we have today. But just in brief, what, what we use as, as evidence for intentionality is that, a gesture is directed towards another individual. So if I'm waving my hand, I'm gonna be focused towards you when I do it.

04:24 - That, there's expectation of a response and evidence for that can be several different directions. So it could be that I wait for a response, so I face you, I wave my hand and that I wait. Or it could be I face you, I wave my hand. If you don't respond, then I persist. I keep communicating and I change the way I communicate. So I might slow down, you know, I go from hey to, or I might change to a different, to a different signal. So I might start with, you know, waving my hand

05:01 - and then clap to get your attention, wave bigger, wave both hands, something like that. So you're seeing this, this persistence and this elaboration or escalation of the signal and waiting for a response. So there's, there's evidence that I'm communicating to a particular individual. I am doing it with an expectation of getting a particular kind of response. And so that's what we use as evidence and intentionality. So, so representational gestures really

05:32 - refer more to the, the kinds of gestures where, they, they have meaning through the movements that they're, that you're making with your hands, rather than, than, through the thing that you're gesturing to. So for example, you know, if I hold up a cup and I point to the cup, we describe that as a dy dick gesture. It can also have a diic gesture of just holding something up, right? It's directing attention to something.

06:02 - And we typically don't describe those as representational gestures because the meaning is anchored in the object itself rather than in the gesture. The gesture is just directing attention towards that. representational gestures would encompass things like iconic gestures where the movement of the jester resembles the, the reference. So for example, with cup, I could use my hands to show the shape of the cup,

06:31 - or I could hold an imaginary cup, right? So it could be I'm holding like more of a glass of water, or I'm holding a mug and I could, you know, pretend to drink out of it, or I could trace, you know, the circle and the, the handle of the mug that would be representing the object through, similarity in form. And sometimes in function, if I'm illustrating the function, now, ity can be quite complicated.

06:58 - So it seems really simple. You look at it and you're like, the movements that you're making map onto the thing that, that you mean. But there's actually, evidence that, that ity can is, is more complicated for children and for apes. So for adults, we see something just like playing charades. It's like, you're acting something out. I get that, that your movement is, is representing something else, right? It's a representational gesture.

07:25 - but for, for young chil very young children, and for apes, it can be more complicated because you have to, you have to understand what features of the movement are important, and you have to understand the analogy between those features of the movement and the features of the real world. For example, if I was going to to, to, you know, gesture about a tree, I could have, you know, the trunk, I could gesture the branches,

07:52 - I could gesture the tree waving in the wind. I could gesture about all the leaves. I could do a, you know, a palm tree versus a big oak. there, you know, I could talk about climbing the tree. And within, within a conversational context, it's, it can really make a lot of sense, right? It can be very helpful. representational gestures can, can kind of provide a lot of quality and depth to, you know, if I just say tree, you don't know what kind of tree I'm talking about, but if I gesture about

08:21 - the tree, you learn something about it. Its size, its shape, its placement, how it moves the relationship I have to it, you know, is it over there? Is it over here? Is it big? Is it small? Am I in the tree itself? Is the, am i under the tree? and so, so representational gestures can provide all of that richness, but if you just have one in isolation, it can be pretty, it's a pretty difficult task actually, to, to figure out what parts of the gesture are, are mapping on to the thing in the real world.

08:55 - So the, the example of the tree i that I gave was really about iconic gesture. It's where the, the movement of the hands maps onto the features of something in the, in the real world. There's another set of, of gestures which are, are referred to either as conventional gestures or sometimes as emblems, where they're really symbolic. And so there, the, the form of the gesture doesn't resemble

09:20 - the thing that it stands for. It it only means that thing because we as a community agree that it does. So the community can be the entire world, or the community can be, you know, a city or a family or even just one person, right? So you can have a secret handshake, say with a friend or a secret, you know, hand symbol that you agree means something and outside, you know, people who looking from the

09:44 - outside wouldn't understand that. And so we do see that those kinds of gestures can vary. You know, they vary cross-culturally, they're actually more similar across cultures than, than you would expect given that, that they have to sort of, they have this meaning because people agree that they mean something, you know, but think about, you know, the thumbs up gesture. Okay? things like that. one of my, my favorite gestures is,

10:11 - so I lived in Poland for a few years, and one of my favorite gestures there is this, you know, like tap, tap of the neck. And I, like in the US I it, when I ask, say, students in the classroom, what do you think this means? And they'll say like, cut it out, or something like that. In Poland, at least in the part of Poland where I lived, it means drunk. Like you're filled up to here.

10:33 - Like, they were really drunk. And you also think about the kinds of rude gestures. I'm not gonna do them here, but those I think are really nice examples of these symbolic conventional gestures that vary from place to place. So, to come, to come back to your question about, about how jester works with speech. So there are, there are different ways of thinking about the relationship between JE and speech. you know, I think that maybe the,

11:08 - the most comprehensive one is to simply say that that gesture and speech together form a unified system. So there's really good evidence that there's a tight, deep rooted relationship between gesture and speech. It's not just that we speak and we gesture and we, we happen to do both at the same time, right? They really, they, they are, sometimes I say intimately entwined, right? They're really tightly integrated. if you prevent people from gesturing,

11:37 - say if you have them sit on their hands or, or, you put their hands in gloves and you velcro them to a table, you'll see that their speech will show more disfluencies. So they'll say, and, they'll have a harder time recalling words And, and they won't speak as fluently and they won't speak as long. And, you see the same thing happen when you restrict, speech.

12:00 - So in the case of stuttering, there's really nice evidence from researchers working on stuttering that during stuttering gesture is also interrupted. So if they were separate systems, then you would expect that if one system fails, the other system would take over. There's actually evidence that when one system is impaired, it, the other system is also impaired. And so, you know, there, there's evidence from that. There's evidence from neuroimaging, and,

12:28 - and EEG, that, that there is actually, sort of simultaneous processing of both speech and of jester. And so I think one of the things that's really interesting about jester is that it's part of language, but it's different than speech. So speech has really specific rules, you know, different rules depending on the language that you speak, but every, every language has rules, has grammatical rules, that, you know, that really tightly constrain, speech and in a way that, that some things are grammatical,

13:04 - some things are not grammatical, you know, and, and it's, it's much easier to sort of predict, what is allowable in speech than what is allowable in gesture, because gesture is much more wide open. And so gesture almost has sort of an analog. If you think about speech as being more digital, one thing at a time in a row, following rules, gesture allows us to communicate in a more analog holistic manner.

13:31 - it it providing different information from what's happening in speech. And so gesture can reinforce what's happening in speech. It can disambiguate what's happening in speech, you know, so if I say, I want that one. If I just say, I want that one, who the heck knows what that is? Right? Almost always when we use something like this or that here or there in speech, we're gesturing. And so you can disambiguate

13:58 - what's what you're saying in speech. You can also add to what you're saying in speech. So sometimes what gesture, the information gesture is providing is complimentary, but different from the information in speech. And so thinking about those relationships, and what information is being provided in speech, what information is being provided in gesture and, and, how the, the sum of both of those modalities

14:24 - gives a fuller pitch picture of communication. you know, I think is a very interesting question. I think I'm maybe gonna talk more about the evolution of language. 'cause that's what I work on. I don't really work on gesture and learning, although there's beautiful research on that. so I think there's some really nice evidence that that learners who are approaching new problems, can use their gestures in ways

14:51 - that help them understand a problem better. So, it was off, there's a great body of literature on gesture for thinking as well as gesture for communication. And so a lot of the things that we do with our hands while we're talking are not just for communicating to other people, they're also for us. And so by putting, you're almost putting ideas out there and then returning to them, you, you know, you can draw things just as you were,

15:17 - as if you were sketching something on paper. You can draw things with your hands, you can think you can rotate them, you can think about them in different ways. It provides an opportunity to externalize some of our internal thought processes. And, and there's, you know, there's evidence from, from experimental work that shows that that actually helps with learning certain kinds of problems with working through more difficult problems, even if you understand,

15:44 - you know, sort of a basic level of a problem. And with, with helping in memory tasks so you're, can, can be a resource to support what we call working memory is the kind of memory that you need when you're say, remembering how to answer a question someone just asks. So, so, we know that that gestures that learners themselves make can help them learn. We also know that learners are sensitive to the gestures that instructors make.

16:13 - And so there are people working on, on trying to, to design better, better lessons that involve gesture as one, as one possible visual aid. And then coming to the question of gesture in the evolution of language, this is, possibly my favorite question. So I work on the evolution of language and on je, so I love the intersection of these. there's, there are theories that jester actually played a much bigger role early in, earlier in human evolution.

16:48 - So when we look at the, at the existing great apes, humans, orangutans, chimpanzees, bonobos, and gorillas, all of them jester only humans have spoken language, but, the other apes have vocalizations, but their gestures seem to, to be more similar to human gesture than their vocalizations are similar to human speech. And so one possibility is that the, the gestural communication of our last common ancestor we're talking 12,

17:21 - 13 million years ago, that for that species jester was really, sort of the most important modality. And that, as you know, as the great apes evolved to, you know, to the present era, je stayed really important. And that when, when humans and chimpanzees and bonobos, and the lineages that led to chimpanzees and bonobos on the one hand, and to humans on the other hand, when those lineages diverged, about, you know, a, a few million years ago, that jester

17:57 - was, was the most important modality for our last common ancestor. So if jester was fact the most important modality, then it could be the case that language actually evolved first within the gestural modality. Gester shows more flexibility. it great, you know, in great apes in human children. it's, it's really, it's the most important modality. And, and there are a bunch of,

18:28 - of theories pulling evidence from different areas, some of which I've discussed today, that suggest that early in human evolutionary history, you know, over the last million years, let's say 500,000 years, that, you know, that gesture may have, you know, really been there first. And that speech is, was a later development. And so I, I'm not saying that language evolved fully as sign language, right? So the sign languages that we see in humans today are,

18:59 - are full, structured languages, just as any spoken language is. Not saying that, you know, our evolutionary ancestors had sign language, but they may have had more complicated gestural systems that in fact, laid the foundation for the kinds of, of highly structured grammatical systems that you see in both spoken and in sign languages today.

Faculti AI can make mistakes