Popular in Economics

Economics

Gender and Economics

University of California Santa Barbara Shelly Lundberg

Banks and Policy Organisations

Greening the financial sector

European Investment Bank Matteo Ferrazzi

Economic History

Gravity with Gravitas: A Solution to the Border Puzzle

Boston College James E. Anderson

Economic Inequalities

Visions of Inequality

City University of New York Branko Milanovic

Economics

School Finance Equalization Increases Intergenerational Mobility

Yale University Barbara Biasi

Economics

Olivier Blanchard on U.S. Pandemic-Era Inflation

Peterson Institute for International Economics Olivier Blanchard

Assessing Nudge Scalability

View more insights in Social Sciences - Economics - Behavioral economics

View more figures from University of California Los Angeles

View the page of Hengchen Dai

Read the Study

Hengchen Dai discusses factors that can explain variation in estimated intervention efficacy across evaluations and help policymakers better predict behavioral responses to interventions in their settings.

Image courtesy of interviewee. May 20, 2024

Generating Workspace

Export as PDF

This video video discusses the concept of "nudges," which are interventions designed using behavioral insights to change human behavior in a beneficial way. The speaker emphasizes the importance of understanding human psychology and how various messaging and choice environments can influence behavior positively, such as increasing COVID-19 vaccine uptake.

Collaborating with UCLA Health, the speaker conducted large-scale randomized control trials (RCTs) to evaluate the effectiveness of different text messages in encouraging vaccine appointments. Key findings include the importance of making scheduling easy by including direct links in text messages and sending reminders to keep vaccination top-of-mind. Concepts like psychological ownership—making individuals feel a dose of the vaccine has been reserved for them—were also effective in boosting vaccine uptake.

The speaker elaborates on implications for behavioral interventions, stating that ease of access, reminder messages, and a sense of ownership can enhance participation. Further, they highlight that the effectiveness of nudges may vary by context, and what works in one setting may not translate to another. They suggest that researchers should consider the outcome measures used—localized versus broader outcomes—as well as the characteristics of the target population when assessing the effectiveness of interventions. Nudges are thought to work best on individuals who are undecided rather than those who are firmly resistant or already eager to vaccinate.

In conclusion, the speaker underscores the need for careful consideration of how and where behavioral insights are applied and the importance of understanding the specific populations targeted by such interventions.

Language:

Export as PDF

X-TIMESTAMP-MAP:LOCAL=00:00:00.000,MPEGTS=900000

00:00.175 --> 00:02.085
Let's take it at a broad level.

00:02.345 --> 00:05.245
In general, I think about nudges as intervention

00:05.315 --> 00:08.805
that leverage behavioral insight about humans, right?

00:08.865 --> 00:12.565
So we know human psychology, we know how humans respond

00:12.625 --> 00:14.565
to different types of language, different types

00:14.565 --> 00:17.725
of information, presentation, different types

00:18.065 --> 00:19.725
of, choice sets.

00:20.225 --> 00:23.645
And so the nudge unit, as well as scientists

00:23.705 --> 00:25.685
who care about behavioral interventions, tend

00:25.685 --> 00:28.005
to leverage those insights about human behavior

00:28.265 --> 00:29.765
to design the choice environment,

00:29.985 --> 00:33.645
design the information sets, and design the message

00:33.905 --> 00:35.325
or communications in order

00:35.325 --> 00:37.885
to change human's behavior in a desirable way.

00:38.305 --> 00:41.925
By desirable way, I mean things like that would be socially

00:41.985 --> 00:44.805
and organizationally desirable,

00:44.985 --> 00:48.125
and also can be beneficial for individuals in the long term.

00:48.385 --> 00:50.165
For example, covid vaccine, right?

00:50.195 --> 00:53.885
That would be an activity that governments would encourage

00:54.525 --> 00:57.045
citizens to undertake that's beneficial

00:57.065 --> 00:58.565
for individual and for the society.

00:58.865 --> 01:00.485
And there are a lot of intervention

01:00.485 --> 01:02.005
that draw from behavioral insights

01:02.025 --> 01:03.845
to encourage covid uptake.

01:04.075 --> 01:07.325
Just as one timely example in my research that tries

01:07.325 --> 01:08.845
to leverage behavioral insights

01:09.025 --> 01:13.725
or use nudges, to increase COD vaccine uptake,

01:14.045 --> 01:16.645
I collaborated with UCLA Health.

01:17.325 --> 01:19.565
U-C-A-L-A is my home institution,

01:19.785 --> 01:22.605
and UCLA health is, one

01:22.605 --> 01:25.125
of the largest healthcare system in California.

01:25.865 --> 01:29.405
We had the opportunity to collaborate with, UCLA health

01:29.465 --> 01:32.125
to conduct a large scale randomized control trials

01:32.625 --> 01:36.965
to test different text messages, in terms of

01:37.105 --> 01:40.365
how effective they are in encouraging COVID-19 vaccine.

01:40.825 --> 01:45.165
So back in 2021, we sent out text messages to

01:45.925 --> 01:49.205
thousands of thousands of patients who are eligible,

01:49.265 --> 01:50.365
at different points in time.

01:50.665 --> 01:53.165
And we varied the content of the text message

01:53.585 --> 01:57.725
and then we use their immunization registry records

01:58.225 --> 01:59.245
to assess whether

01:59.245 --> 02:02.085
or not they end up getting the COVID-19 vaccine.

02:02.975 --> 02:04.315
And then a few years later,

02:04.505 --> 02:07.635
when the Bivalent booster become available in the fall

02:07.635 --> 02:12.195
of 2022, we again collaborated with UCA Health

02:13.055 --> 02:15.115
to, randomize the content

02:15.375 --> 02:17.795
of the text messages we send out to patients

02:17.795 --> 02:19.635
who are eligible for the five and booster.

02:19.735 --> 02:23.235
And again, look at how different types of,

02:23.265 --> 02:27.155
text messages affect the Bible booster take up rates.

02:27.595 --> 02:29.675
A couple of things we have learned from those trials.

02:30.535 --> 02:34.275
One is it's important to make it easy for people

02:34.495 --> 02:35.555
to schedule appointments.

02:35.895 --> 02:39.635
So in our text messages, we tend to include a link

02:39.825 --> 02:40.955
that people can click

02:41.055 --> 02:42.795
and then directly schedule appointments.

02:43.175 --> 02:44.755
So we call this to make things easy.

02:45.195 --> 02:49.435
I think the net unit in UK has highlighted this as one

02:49.435 --> 02:50.555
of the basic principle

02:50.575 --> 02:52.715
for influencing citizens behavior, right?

02:52.715 --> 02:54.795
You wanna make things easy. And then two,

02:55.135 --> 02:58.775
the text reminders really help make things, top

02:58.775 --> 03:00.185
Of mind for people.

03:00.655 --> 03:03.145
Some people may have the intention to get the covid vaccine

03:03.245 --> 03:05.105
or the Bible and the booster, but they're busy

03:05.325 --> 03:06.465
and there are a lot of things going on.

03:06.465 --> 03:08.625
They forget about it, sending a text reminder,

03:08.735 --> 03:10.865
make things salient on top of their mind.

03:10.925 --> 03:14.625
And actually, I think s saliency is another principle

03:14.625 --> 03:17.545
highlighted often by the nudge unit in uk.

03:18.485 --> 03:21.625
Uh, and further, some of our text messages

03:22.345 --> 03:25.025
leverage a concept called psychological ownership.

03:25.405 --> 03:28.025
We try to make people feel that a dose

03:28.085 --> 03:30.985
of the COVID vaccine has been made available to them

03:31.245 --> 03:34.065
and they should claim their dose by scheduling appointment.

03:34.565 --> 03:38.025
And this type of language helps to further

03:38.545 --> 03:41.145
increase vaccine uptake during the initial phase

03:41.245 --> 03:43.545
of the vaccine rollout back in 2021,

03:43.765 --> 03:46.545
and it'll work again in 2022.

03:47.005 --> 03:48.625
Uh, when we try to encourage people

03:48.805 --> 03:50.225
to get the violent booster,

03:50.455 --> 03:54.025
this concept means people have a sense of ownership about

03:54.545 --> 03:56.505
a property, about an object.

03:57.205 --> 04:00.785
Uh, so in this case, you could imagine,

04:01.705 --> 04:04.345
everyone technically if they're eligible for Covid vaccine,

04:04.435 --> 04:06.905
there should be a dose available for them.

04:07.325 --> 04:09.145
Uh, this could be an issue early on

04:09.145 --> 04:10.825
with supply was a problem.

04:11.365 --> 04:14.865
Uh, but when people actually become eligible to go

04:14.965 --> 04:18.185
to get the vaccine at UCA Health, actually we only reach out

04:18.185 --> 04:20.545
to people when we know the demand is sorry,

04:20.565 --> 04:21.785
the supply is there for them.

04:22.205 --> 04:26.185
So in our setting, technically supply should not be an issue

04:26.185 --> 04:28.345
because UCLA health already reserved the dose

04:28.365 --> 04:30.065
for people they reach, we reach out for.

04:30.495 --> 04:32.985
However, if we highlight that

04:33.585 --> 04:36.545
a dose has just been made available to you, come

04:36.545 --> 04:38.105
and get it, it is your dose.

04:38.565 --> 04:40.425
It instilled the sense of ownership.

04:40.425 --> 04:43.785
People feel as if a dose was reserved for them

04:43.845 --> 04:46.705
and they should really claim it, claim the dose

04:46.765 --> 04:48.985
before it take by other people.

04:49.685 --> 04:52.065
And similarly, when the booster

04:52.085 --> 04:54.665
by the booster was available, it was not,

04:54.665 --> 04:57.105
there was not really, a supply issue.

04:57.165 --> 05:00.945
And especially we direct people to CVS pharmacy,

05:00.945 --> 05:05.505
because we know CVS pharmacy, has abundance of in,

05:05.565 --> 05:08.265
has abundance in, vaccination appointments.

05:09.335 --> 05:11.025
However, there when we tell people,

05:11.115 --> 05:15.545
claim your dose today this word claim your dose still make

05:15.545 --> 05:17.945
people feel that there is a dose for them

05:17.975 --> 05:20.585
that they should try to grab it before it is gone.

05:20.845 --> 05:23.505
So it's a sense of a scarcity, it's a sense of ownership

05:23.655 --> 05:25.865
that really try to get people to act.

05:26.345 --> 05:29.225
I think our research broadly has a few implications.

05:29.685 --> 05:32.745
One, if we just think about what works in our trials.

05:32.745 --> 05:34.065
So then the implication is, one,

05:34.065 --> 05:35.505
you wanna make things easy for people.

05:36.305 --> 05:40.025
Reminders, can help make things, on top of mind.

05:40.205 --> 05:41.985
And ownership language helps.

05:42.085 --> 05:45.065
So to the extent possible, you wanna instill the sense

05:45.065 --> 05:47.985
of genuine authentic scarcity, right?

05:47.985 --> 05:50.625
Yeah. And you try to encourage people to act, assume

05:50.765 --> 05:54.225
by telling them to claim whatever belongs to them today.

05:54.695 --> 05:56.065
Well, that is one aspect

05:56.135 --> 05:58.085
that is just thinking about what works.

05:58.465 --> 06:02.525
But I think what is also very interesting in our work is we

06:02.595 --> 06:06.365
also shed some light on the transferability

06:06.465 --> 06:08.685
and the scalability of behavioral insights.

06:09.425 --> 06:12.605
The idea is that one intervention may seem

06:12.605 --> 06:14.125
to work in one setting,

06:14.425 --> 06:16.965
but it may not work again in another setting.

06:17.355 --> 06:19.965
It's very important for us to understand

06:20.395 --> 06:23.565
what can carries over as general behavioral insights

06:23.565 --> 06:26.685
that can be translated in another different setting.

06:26.995 --> 06:30.685
What are the factors that would influence, the ability

06:31.025 --> 06:34.205
of us translated results from one context to another?

06:34.465 --> 06:37.365
And that is something my recent research has been looking at

06:37.945 --> 06:39.805
and from the, trials,

06:40.065 --> 06:42.405
the randomized control trials I've told you about.

06:42.905 --> 06:45.205
And we, and we also combine it

06:45.315 --> 06:48.925
with a cover dataset about RCTs run by other scholars.

06:48.985 --> 06:53.165
And we combine our own RCTs with online surveys.

06:53.485 --> 06:56.845
Collectively, we have draw some insights,

06:56.895 --> 06:58.845
about the scalability and the transferability.

06:59.345 --> 07:01.445
I'm happy to share if this is the right time.

07:01.925 --> 07:05.325
A couple of insights. One is we find that,

07:06.065 --> 07:09.405
the effectiveness of behavioral interventions

07:09.425 --> 07:13.525
or nudges depends on how you even measure the out measure,

07:13.625 --> 07:15.605
the success of the intervention to begin with.

07:16.705 --> 07:19.925
If researchers measure the, success

07:19.985 --> 07:24.725
of intervention based on very localized dvs, such as whether

07:24.725 --> 07:28.245
or not people get vaccinated at UCA health in a trial

07:28.595 --> 07:30.845
that is in collaboration with U Health,

07:31.265 --> 07:33.285
you may end up getting larger results.

07:33.665 --> 07:36.885
Uh, the, the intervention may seem to have a bigger impact,

07:37.425 --> 07:39.525
on vaccination within U Health.

07:40.075 --> 07:42.245
Then if you look at some more global

07:42.425 --> 07:45.845
and broader outcomes, such as whether people get vaccinated

07:46.365 --> 07:50.045
anywhere in California, the idea is when you measure

07:50.045 --> 07:51.325
something that's more local,

07:51.795 --> 07:54.605
like whether people do something within the organization you

07:54.805 --> 07:58.685
collaborate with, you might simply shift the people from

07:58.685 --> 08:01.605
taking up the activities such as getting the vaccine from

08:01.605 --> 08:04.365
outside of UCA health to inside UCA health.

08:04.665 --> 08:08.405
So it could be, more or less a displacement effect.

08:08.465 --> 08:11.045
You just change the way people get vaccinated rather than

08:11.045 --> 08:12.565
whether they get vaccinated at all.

08:12.905 --> 08:16.605
So in our own study, we find that examine,

08:16.605 --> 08:20.645
vaccine uptake, within US warehouse leads

08:20.645 --> 08:22.885
to a larger effect size that if we look at,

08:22.885 --> 08:25.245
so whether our text message get people

08:25.345 --> 08:28.765
to receive the COVID-19 vaccine anywhere at all.

08:29.145 --> 08:31.725
So that is one evidence from our research.

08:32.105 --> 08:33.925
And then we also looked at,

08:33.925 --> 08:37.365
randomized control trials conducted by academics as well

08:37.505 --> 08:38.845
as well as government units.

08:39.225 --> 08:41.205
And you ask us,

08:41.255 --> 08:45.005
early on why we see some difference in effect size between

08:45.515 --> 08:48.205
RCTs run by governments versus by academics.

08:48.985 --> 08:51.805
One potential reason our research suggests is

08:51.805 --> 08:54.525
that academics tend to use outcome measures

08:54.525 --> 08:55.780
that are more local, right?

08:55.780 --> 08:58.325
Because we as academics tend to work with

08:59.385 --> 09:00.645
one specific organization

09:00.745 --> 09:04.205
and we look at whether people do some beneficial activities

09:04.305 --> 09:05.605
within that organization.

09:06.025 --> 09:07.125
But governments tend

09:07.125 --> 09:09.925
to have broad records based on administrative records.

09:10.395 --> 09:13.125
They are able to look at global

09:13.265 --> 09:17.205
and broad outcomes like whether the citizen pay certain

09:18.035 --> 09:21.125
dues in any other ways possible,

09:21.185 --> 09:24.325
or whether the citizen submits application in any

09:24.325 --> 09:25.685
channel possible, right?

09:26.065 --> 09:29.045
So the government units tend to assess much broader outcome.

09:29.505 --> 09:32.565
And because those more local outcomes tend

09:32.565 --> 09:34.285
to show bigger effect size,

09:34.985 --> 09:38.125
and we see, academics tend to use more local outcomes,

09:38.355 --> 09:40.405
that provides one potential explanation

09:40.465 --> 09:44.325
for why the effect size observed in academic papers tend

09:44.325 --> 09:47.325
to be larger than effect size observed in the government

09:47.885 --> 09:49.765
agencies that run those behavioral interventions.

09:50.345 --> 09:53.165
So that's really one lesson that we are very excited about

09:53.235 --> 09:56.285
that is trying to understand the scope

09:56.385 --> 09:58.405
of the outcome measures you are using.

09:58.705 --> 09:59.725
And when you find

09:59.795 --> 10:02.245
that other people do not replicate your effect

10:02.665 --> 10:04.445
or they observe much smaller effect,

10:05.135 --> 10:08.185
think about whether you are not even talking about the same

10:08.415 --> 10:09.625
measure to begin with.

10:10.935 --> 10:13.115
And then another interesting lesson is

10:13.695 --> 10:15.155
who we are trying to influence?

10:15.935 --> 10:19.515
Are we trying to influence people who are hesitant,

10:20.025 --> 10:22.715
very hesitant to get a vaccine, for example,

10:23.215 --> 10:26.755
or we are trying to influence a general population

10:26.755 --> 10:28.235
that can have a mixture of people

10:28.335 --> 10:29.995
who are very hesitant versus people

10:29.995 --> 10:32.835
who are very excited versus people on the margin, right?

10:32.855 --> 10:36.155
So like trying to understand the sample, it matters

10:36.185 --> 10:39.835
because, we have find that nudges tend

10:39.835 --> 10:41.995
to have a large effect on those people

10:41.995 --> 10:44.275
who are more on the margin, on the margin, right?

10:44.495 --> 10:46.595
Not those people who are resistant, not people

10:46.755 --> 10:49.075
who are very already very eager, they're gonna do it.

10:49.075 --> 10:49.955
So they're gonna get a vaccine

10:50.315 --> 10:51.515
regardless of what we do to them.

10:52.055 --> 10:54.635
Um, those people on the margin are those people

10:54.655 --> 10:57.155
who have some intention to get vaccine,

10:57.575 --> 10:59.075
but they have not taken action.

10:59.085 --> 11:00.515
Maybe because they forget, maybe

11:00.515 --> 11:02.235
because they procrastinate, maybe

11:02.235 --> 11:06.835
because they do not love the hassle cost of deciding where

11:06.855 --> 11:07.955
and when to get vaccine.

11:08.415 --> 11:09.835
So they have put off, right.

11:10.335 --> 11:12.035
And, nudges

11:12.135 --> 11:15.555
as more light touch behavioral intervention may work better

11:15.575 --> 11:18.395
on those people who already have some intention,

11:19.335 --> 11:22.075
are not fundamentally resistant to the idea of vaccination,

11:22.215 --> 11:24.835
but just need some push to get them

11:25.095 --> 11:26.795
to actually follow through on their intention.

11:27.175 --> 11:30.795
Or they may have some, small misperceptions

11:31.065 --> 11:34.075
that we can correct with simple information intervention,

11:34.655 --> 11:35.835
that are fine.

11:36.055 --> 11:37.755
But if they're fundamentally resistant,

11:37.755 --> 11:39.035
they do not trust the system,

11:39.055 --> 11:41.675
though I think we need a much heavy handed intervention

11:42.215 --> 11:44.075
to address that fundamental issue.

11:44.335 --> 11:48.315
So that's another insight we learn, is when you find

11:49.105 --> 11:51.675
that your results do not replicate in another setting,

11:51.975 --> 11:54.595
or you do not, see the scalability

11:54.655 --> 11:56.355
of your results in another setting.

11:56.565 --> 11:59.955
Think about whether you are rolling out the intervention in

12:00.255 --> 12:03.635
the same type of population as the population

12:03.635 --> 12:05.355
where the idea was initially tested.

00:00 - Let's take it at a broad level. In general, I think about nudges as intervention that leverage behavioral insight about humans, right? So we know human psychology, we know how humans respond to different types of language, different types of information, presentation, different types

00:18 - of, choice sets. And so the nudge unit, as well as scientists who care about behavioral interventions, tend to leverage those insights about human behavior to design the choice environment, design the information sets, and design the message or communications in order to change human's behavior in a desirable way. By desirable way, I mean things like that would be socially and organizationally desirable,

00:44 - and also can be beneficial for individuals in the long term. For example, covid vaccine, right? That would be an activity that governments would encourage citizens to undertake that's beneficial for individual and for the society. And there are a lot of intervention that draw from behavioral insights to encourage covid uptake. Just as one timely example in my research that tries to leverage behavioral insights

01:09 - or use nudges, to increase COD vaccine uptake, I collaborated with UCLA Health. U-C-A-L-A is my home institution, and UCLA health is, one of the largest healthcare system in California. We had the opportunity to collaborate with, UCLA health to conduct a large scale randomized control trials to test different text messages, in terms of how effective they are in encouraging COVID-19 vaccine. So back in 2021, we sent out text messages to

01:45 - thousands of thousands of patients who are eligible, at different points in time. And we varied the content of the text message and then we use their immunization registry records to assess whether or not they end up getting the COVID-19 vaccine. And then a few years later, when the Bivalent booster become available in the fall of 2022, we again collaborated with UCA Health to, randomize the content

02:15 - of the text messages we send out to patients who are eligible for the five and booster. And again, look at how different types of, text messages affect the Bible booster take up rates. A couple of things we have learned from those trials. One is it's important to make it easy for people to schedule appointments. So in our text messages, we tend to include a link that people can click and then directly schedule appointments.

02:43 - So we call this to make things easy. I think the net unit in UK has highlighted this as one of the basic principle for influencing citizens behavior, right? You wanna make things easy. And then two, the text reminders really help make things, top Of mind for people. Some people may have the intention to get the covid vaccine or the Bible and the booster, but they're busy and there are a lot of things going on.

03:06 - They forget about it, sending a text reminder, make things salient on top of their mind. And actually, I think s saliency is another principle highlighted often by the nudge unit in uk. and further, some of our text messages leverage a concept called psychological ownership. We try to make people feel that a dose of the COVID vaccine has been made available to them and they should claim their dose by scheduling appointment. And this type of language helps to further

03:38 - increase vaccine uptake during the initial phase of the vaccine rollout back in 2021, and it'll work again in 2022. when we try to encourage people to get the violent booster, this concept means people have a sense of ownership about a property, about an object. so in this case, you could imagine, everyone technically if they're eligible for Covid vaccine, there should be a dose available for them.

04:07 - this could be an issue early on with supply was a problem. but when people actually become eligible to go to get the vaccine at UCA Health, actually we only reach out to people when we know the demand is sorry, the supply is there for them. So in our setting, technically supply should not be an issue because UCLA health already reserved the dose for people they reach, we reach out for. However, if we highlight that

04:33 - a dose has just been made available to you, come and get it, it is your dose. It instilled the sense of ownership. People feel as if a dose was reserved for them and they should really claim it, claim the dose before it take by other people. And similarly, when the booster by the booster was available, it was not, there was not really, a supply issue. And especially we direct people to CVS pharmacy,

05:00 - because we know CVS pharmacy, has abundance of in, has abundance in, vaccination appointments. However, there when we tell people, claim your dose today this word claim your dose still make people feel that there is a dose for them that they should try to grab it before it is gone. So it's a sense of a scarcity, it's a sense of ownership that really try to get people to act. I think our research broadly has a few implications. One, if we just think about what works in our trials.

05:32 - So then the implication is, one, you wanna make things easy for people. Reminders, can help make things, on top of mind. And ownership language helps. So to the extent possible, you wanna instill the sense of genuine authentic scarcity, right? Yeah. And you try to encourage people to act, assume by telling them to claim whatever belongs to them today. Well, that is one aspect that is just thinking about what works.

05:58 - But I think what is also very interesting in our work is we also shed some light on the transferability and the scalability of behavioral insights. The idea is that one intervention may seem to work in one setting, but it may not work again in another setting. It's very important for us to understand what can carries over as general behavioral insights that can be translated in another different setting. What are the factors that would influence, the ability

06:31 - of us translated results from one context to another? And that is something my recent research has been looking at and from the, trials, the randomized control trials I've told you about. And we, and we also combine it with a cover dataset about RCTs run by other scholars. And we combine our own RCTs with online surveys. Collectively, we have draw some insights, about the scalability and the transferability. I'm happy to share if this is the right time.

07:01 - A couple of insights. One is we find that, the effectiveness of behavioral interventions or nudges depends on how you even measure the out measure, the success of the intervention to begin with. If researchers measure the, success of intervention based on very localized dvs, such as whether or not people get vaccinated at UCA health in a trial that is in collaboration with U Health, you may end up getting larger results. the, the intervention may seem to have a bigger impact,

07:37 - on vaccination within U Health. Then if you look at some more global and broader outcomes, such as whether people get vaccinated anywhere in California, the idea is when you measure something that's more local, like whether people do something within the organization you collaborate with, you might simply shift the people from taking up the activities such as getting the vaccine from outside of UCA health to inside UCA health. So it could be, more or less a displacement effect.

08:08 - You just change the way people get vaccinated rather than whether they get vaccinated at all. So in our own study, we find that examine, vaccine uptake, within US warehouse leads to a larger effect size that if we look at, so whether our text message get people to receive the COVID-19 vaccine anywhere at all. So that is one evidence from our research. And then we also looked at, randomized control trials conducted by academics as well

08:37 - as well as government units. And you ask us, early on why we see some difference in effect size between RCTs run by governments versus by academics. One potential reason our research suggests is that academics tend to use outcome measures that are more local, right? Because we as academics tend to work with one specific organization and we look at whether people do some beneficial activities

09:04 - within that organization. But governments tend to have broad records based on administrative records. They are able to look at global and broad outcomes like whether the citizen pay certain dues in any other ways possible, or whether the citizen submits application in any channel possible, right? So the government units tend to assess much broader outcome. And because those more local outcomes tend

09:32 - to show bigger effect size, and we see, academics tend to use more local outcomes, that provides one potential explanation for why the effect size observed in academic papers tend to be larger than effect size observed in the government agencies that run those behavioral interventions. So that's really one lesson that we are very excited about that is trying to understand the scope of the outcome measures you are using. And when you find

09:59 - that other people do not replicate your effect or they observe much smaller effect, think about whether you are not even talking about the same measure to begin with. And then another interesting lesson is who we are trying to influence? Are we trying to influence people who are hesitant, very hesitant to get a vaccine, for example, or we are trying to influence a general population that can have a mixture of people

10:28 - who are very hesitant versus people who are very excited versus people on the margin, right? So like trying to understand the sample, it matters because, we have find that nudges tend to have a large effect on those people who are more on the margin, on the margin, right? Not those people who are resistant, not people who are very already very eager, they're gonna do it. So they're gonna get a vaccine regardless of what we do to them.

10:52 - those people on the margin are those people who have some intention to get vaccine, but they have not taken action. Maybe because they forget, maybe because they procrastinate, maybe because they do not love the hassle cost of deciding where and when to get vaccine. So they have put off, right. And, nudges as more light touch behavioral intervention may work better

11:15 - on those people who already have some intention, are not fundamentally resistant to the idea of vaccination, but just need some push to get them to actually follow through on their intention. Or they may have some, small misperceptions that we can correct with simple information intervention, that are fine. But if they're fundamentally resistant, they do not trust the system, though I think we need a much heavy handed intervention

11:42 - to address that fundamental issue. So that's another insight we learn, is when you find that your results do not replicate in another setting, or you do not, see the scalability of your results in another setting. Think about whether you are rolling out the intervention in the same type of population as the population where the idea was initially tested.

Faculti AI can make mistakes