Popular in Social Sciences

Books

City Time: On Being Sentenced to Rikers Island

Independent Scholar David A. Campbell

Culture and Society

The City as a Growth Machine: Toward a Political Economy of Place

New York University Harvey Moltoch

Economics

Gender and Economics

University of California Santa Barbara Shelly Lundberg

American Politics

Keeping the Republic

Boston College Marc Landy

Banks and Policy Organisations

Greening the financial sector

European Investment Bank Matteo Ferrazzi

British Politics

Entrepreneurial thinking: the politics and practice of policy impact

University of Sheffield Matthew Flinders

Culture and Society

Carceral Entrapments: Views from the Prison/Street Interface in India

Indian Institute of Technology Delhi Mahuya Bandyopadhya

High-Stakes Objective and Subjective Teacher Evaluation Measures and Student Skill Development

View more insights in Social Sciences - Economics - Microeconomics

View more figures from Abt Associates

View the page of Ayman Shakeel

Read the Study

Ayman Shakeel discusses the relative contributions of the subjective versus objective evaluation measures in capturing teacher effectiveness at increasing cognitive and non-cognitive skills.

Image courtesy of interviewee. December 1, 2023

Generating Workspace

Export as PDF

This video video discusses the evolution of teacher evaluation methods in the U.S., driven by the proliferation of state longitudinal data aimed at accurately assessing teacher performance. Policymakers use two primary types of measures: objective measures, such as test score value-added (which quantifies teacher contributions to student growth), and subjective measures, which include classroom observations by supervisors and student evaluations.

The focus of the analysis is on a reform in a large urban public school district that combined these evaluation measures with implications for teacher salaries. The study aims to answer three primary questions regarding the relationship between these evaluation measures and students’ cognitive and non-cognitive skills, whether the measures provide additional information beyond each other, and their relationship with long-term student success.

Key findings indicate that both supervisor observations and student surveys significantly relate to student achievement and behaviors, providing additional insights beyond test score value-added measures. Particularly, teachers with high survey scores have a notable impact on reducing student absences, especially among low-performing students, which highlights the importance of supportive teaching for equity in education.

The study emphasizes the need for both objective and subjective measures to foster effective teaching practices and mitigate potential distortions in incentives. It concludes that while test score value-added measures are crucial, the subjective measures also play a significant role in providing meaningful information related to teacher effectiveness and student outcomes.

Language:

Export as PDF

X-TIMESTAMP-MAP:LOCAL=00:00:00.000,MPEGTS=900000

00:00.455 --> 00:03.255
Proliferation of state longitudinal data has resulted in

00:03.255 --> 00:05.495
efforts by policymakers to

00:06.015 --> 00:08.135
evaluate teacher performance more accurately.

00:08.475 --> 00:10.735
And they mainly rely on two types of measures.

00:11.235 --> 00:13.175
The first is called the objective measure,

00:13.265 --> 00:15.535
which is a quantitative measure of teacher quality.

00:15.955 --> 00:20.055
And the most common type of quantitative measure that,

00:20.235 --> 00:23.215
is used is called test score value added test score value

00:23.215 --> 00:25.935
added basically just captures teacher contributions

00:25.935 --> 00:27.255
to student test score growth.

00:27.635 --> 00:29.535
And then we also have subjective measures.

00:30.035 --> 00:32.975
Uh, the most common subjective measures are supervisors

00:32.975 --> 00:35.815
or principals observing classroom instruction as well

00:35.815 --> 00:39.015
as young students in some cases evaluating teachers.

00:39.865 --> 00:40.895
While many states

00:40.955 --> 00:43.535
and districts are using a combination of these measures,

00:43.735 --> 00:45.375
I guess the big picture question is,

00:46.005 --> 00:49.775
does the combination really helps identify teachers

00:49.775 --> 00:51.375
that are raising valuable skills?

00:51.955 --> 00:54.535
So in order to get at this big picture question,

00:54.835 --> 00:55.975
I'm using the context

00:56.155 --> 00:59.775
of a large urban public school district that had this reform

00:59.775 --> 01:02.735
where they're evaluated the teachers on the basis of task

01:02.755 --> 01:06.415
or value added supervisor observations and student surveys,

01:06.675 --> 01:09.375
and then with some qualifications they tie their subsequent

01:09.375 --> 01:10.375
to your salaries to it.

01:10.995 --> 01:13.575
So using this context, I'm going

01:13.575 --> 01:15.935
to answer three main questions in my study.

01:16.635 --> 01:20.295
The first is, are these evaluation measures significantly

01:20.325 --> 01:22.575
related to the student's cognitive as well

01:22.575 --> 01:23.735
as non-cognitive skills?

01:24.295 --> 01:26.815
I use test scores as proxies for cognitive skills

01:27.275 --> 01:29.255
and absences in receipt of suspensions.

01:29.255 --> 01:31.575
They serve as proxies for non-cognitive skills.

01:32.395 --> 01:35.855
The next question I ask is, are these measures,

01:35.855 --> 01:38.135
giving us any information on top of one another?

01:38.635 --> 01:40.535
And are they significantly related

01:40.555 --> 01:42.455
to the student's longer term success?

01:42.835 --> 01:44.615
And I measure the longer term success

01:44.875 --> 01:46.895
by their subsequent year outcomes.

01:47.835 --> 01:50.295
My paper contributes to two strands of literature

01:50.295 --> 01:52.735
regarding the measurement of worker productivity.

01:53.515 --> 01:55.575
The first is related to the effectiveness

01:55.575 --> 01:56.895
of the evaluation measures.

01:57.275 --> 01:59.935
So there's a rich theoretical literature regarding

02:00.125 --> 02:04.175
what is an optimal incentive contract where agents can,

02:04.435 --> 02:06.735
you know, agent output is multi-dimensional.

02:06.885 --> 02:09.935
There's work by TRO and Milgram and Baker.

02:10.435 --> 02:14.215
And, what I'm doing is, in the context of education,

02:14.785 --> 02:18.055
there is a famous work by Jackson.

02:18.315 --> 02:20.855
And now there are other papers that talk about the fact that

02:21.735 --> 02:22.775
teachers, they contribute

02:22.775 --> 02:24.295
to the development of a lot of skills.

02:24.485 --> 02:26.615
Teacher quality is multi-dimensional,

02:26.685 --> 02:29.735
however, test score value added does not fully capture

02:29.735 --> 02:31.335
everything that a teacher is doing.

02:31.795 --> 02:33.615
So that's the first strand of literature.

02:34.075 --> 02:35.615
My paper also contributes

02:35.715 --> 02:38.295
to the subjective performance assessment literature.

02:39.035 --> 02:40.935
Um, there's work by Jacob

02:41.075 --> 02:45.175
and Nev Grant Craft et all that talks about the fact,

02:45.225 --> 02:47.575
about the relationship between,

02:47.875 --> 02:51.775
the supervisor observations and student test test scores.

02:52.315 --> 02:55.215
And I am extending that strand of the literature

02:55.215 --> 02:57.415
by presenting new evidence on supervisor

02:57.415 --> 02:58.935
observations as well as student

02:59.075 --> 03:00.075
Surveys.

03:00.095 --> 03:01.485
Let's take a step back

03:01.585 --> 03:04.365
and talk about the teacher evaluation framework in the

03:04.365 --> 03:07.085
United States and how it has evolved over time.

03:07.625 --> 03:08.685
So historically,

03:08.825 --> 03:11.645
the evaluations were primarily based on supervisor

03:11.825 --> 03:14.005
or classroom observations by principals.

03:14.385 --> 03:16.925
Now, the advantage of supervisor observations is

03:16.925 --> 03:19.725
that they can be readily conducted for most teachers

03:20.065 --> 03:23.205
and can potentially give them, good feedback.

03:23.945 --> 03:26.365
But in the recent years, there has been a push

03:26.425 --> 03:28.405
for having some quantitative measure

03:28.715 --> 03:30.525
that is test score value added, be a part

03:30.525 --> 03:32.445
of the evaluation framework, mainly

03:32.445 --> 03:34.885
because of concerns that supervisor observations

03:34.945 --> 03:36.165
are often lenient.

03:36.745 --> 03:39.405
Now, the good thing about the quantitative measure is

03:39.405 --> 03:42.645
that it really gives us like a quantitative idea of

03:42.665 --> 03:44.405
how much teachers are contributing

03:44.405 --> 03:45.605
to the student test scores.

03:46.345 --> 03:47.645
But this measure

03:47.785 --> 03:49.965
as a standalone measure also has certain

03:50.075 --> 03:51.205
potential drawbacks.

03:51.545 --> 03:55.285
It can induce a perverse incentive for strategic behaviors,

03:55.665 --> 03:57.725
and there's a disproportionate focus on a

03:57.725 --> 03:58.925
single type of outcome.

03:59.505 --> 04:02.365
So one solution which a lot of states now use is

04:02.365 --> 04:04.605
that they use a combination of the two measures.

04:04.875 --> 04:07.845
However, in the presence of test-based accountability,

04:08.435 --> 04:10.885
even the combination of the two may not be able

04:10.885 --> 04:12.685
to blunt the adverse effects of each.

04:13.485 --> 04:15.525
For example, principals may be hesitant

04:15.525 --> 04:17.285
to give lower ratings to teachers

04:17.585 --> 04:20.205
who are concerned are engaging in strategic behavior.

04:21.065 --> 04:24.685
So one potential solution, and this has gained more light

04:24.695 --> 04:26.845
after the measure of effective teaching project

04:27.225 --> 04:30.325
and the Obama race to the top initiative, is the idea

04:30.345 --> 04:34.365
of also including student surveys in the evaluation process.

04:35.345 --> 04:38.885
So students, they spend the most time with the teachers.

04:39.105 --> 04:41.285
So we may expect their evaluations

04:41.665 --> 04:44.245
to tell us something meaningful about the teacher quality,

04:44.665 --> 04:46.365
but at the same time, there are concerns

04:46.365 --> 04:48.725
that students may discriminate against teachers

04:49.385 --> 04:51.165
or, you know, this measure, inclusion

04:51.165 --> 04:53.285
of this measure may use a popularity contest.

04:54.025 --> 04:56.885
So it's really important to see whether this measure

04:56.945 --> 04:58.525
or any of the measures that are part

04:58.525 --> 05:01.645
of the teacher evaluation framework are really telling us

05:01.645 --> 05:03.165
something that the teacher is doing.

05:03.785 --> 05:04.845
In 2014,

05:05.085 --> 05:07.885
a large urban public school district had this reform

05:07.885 --> 05:09.845
where they evaluate the teachers on the basis

05:09.945 --> 05:13.205
of student achievement, supervisor observations

05:13.305 --> 05:14.445
and student surveys,

05:14.945 --> 05:17.685
and then they assign teachers evaluation,

05:17.685 --> 05:20.975
scores based on these measures, which are then tied

05:20.995 --> 05:23.735
to categorical evaluation ratings, which

05:23.735 --> 05:24.815
with some qualifications

05:24.835 --> 05:28.535
and determine the teacher subsequent year salaries.

05:29.235 --> 05:30.935
So I have administrative data

05:31.335 --> 05:33.535
provided by the district for the student test scores,

05:33.895 --> 05:35.215
absences and suspensions.

05:35.295 --> 05:37.495
I have teacher level data as well.

05:38.035 --> 05:41.175
And I'm going to focus on grades three to seven

05:41.275 --> 05:45.415
for the years 2014 till 2019, there is an issue

05:45.615 --> 05:46.655
of correlated errors.

05:46.715 --> 05:48.055
So for example, students

05:48.125 --> 05:50.975
that are better performing may just be predisposed

05:50.975 --> 05:52.655
to giving teachers better ratings,

05:52.655 --> 05:54.615
and that may not necessarily reflect

05:54.765 --> 05:55.935
what the teacher is doing.

05:56.555 --> 05:59.085
So in order to get around with that issue,

05:59.225 --> 06:01.445
and this is something that is standard in the literature,

06:01.865 --> 06:05.085
instead of regressing the student outcomes on these

06:05.175 --> 06:07.405
evaluation measures, I first create

06:07.405 --> 06:10.325
what is called a lever out evaluation measures.

06:10.385 --> 06:13.565
So I predict a teacher's test score value added

06:13.565 --> 06:17.565
or the subjective scores in year T using all her years

06:17.905 --> 06:20.725
except for Year T to avoid biasing the estimate.

06:21.225 --> 06:22.365
And then what I do is

06:22.365 --> 06:24.565
that I regress the student outcomes on these

06:24.565 --> 06:25.765
leave year out measures.

06:26.225 --> 06:28.045
And I also include some controls

06:28.045 --> 06:29.525
that is student level controls.

06:29.555 --> 06:31.925
Classroom level controls school by year

06:31.985 --> 06:33.445
and grade by year fixed effect.

06:33.445 --> 06:37.885
In my model, I present new evidence regarding the,

06:38.005 --> 06:40.285
relationship between the objective

06:40.285 --> 06:42.405
and the subjective measures and the student skills.

06:43.225 --> 06:45.205
So I'll start off with this objective measure

06:45.235 --> 06:46.925
that is supervisor observations.

06:47.145 --> 06:50.525
So I find that the supervisor observations are pretty

06:50.525 --> 06:55.045
significantly related to the students, subsequent year,

06:55.065 --> 06:57.565
as well as contemporaneous, achievement.

06:57.745 --> 07:00.725
And they give us additional information on top of

07:00.725 --> 07:02.485
what we learned by test score value added

07:03.025 --> 07:04.805
for the contemporaneous as well

07:04.805 --> 07:06.405
as subsequent year reading achievement.

07:07.245 --> 07:10.645
Interestingly, the student surveys are also giving us

07:10.965 --> 07:13.325
information about the student's contemporaneous

07:13.425 --> 07:15.485
and subsequent year achievement.

07:16.145 --> 07:19.205
And of the three measures, teachers

07:19.205 --> 07:20.925
with high service scores, they seem

07:20.925 --> 07:24.205
to be the most effective in reducing the student's

07:24.205 --> 07:27.165
contemporaneous absences For the subject

07:27.265 --> 07:30.605
for the objective measure test score value added, I find

07:30.605 --> 07:33.845
that test score value added is pretty significantly related

07:33.985 --> 07:36.485
to not only contemporaneous achievement,

07:36.505 --> 07:39.285
but also contemporaneous absences as well

07:39.285 --> 07:41.485
as subsequent year achievement and absences.

07:41.945 --> 07:45.005
So in some, the subjective measures are giving us

07:45.005 --> 07:47.925
information about the students' current tier as well

07:48.205 --> 07:49.845
as subsequent tier achievement

07:50.385 --> 07:52.885
and contemporaneous absences,

07:52.905 --> 07:54.885
and they're giving us information on top of

07:54.885 --> 07:57.325
what we are learning by what test score value added.

07:57.555 --> 08:00.005
However, in this multiple measure system

08:00.515 --> 08:03.965
test score value added is the most significantly related to

08:04.555 --> 08:06.085
most of the outcomes,

08:06.425 --> 08:08.365
the valued outcomes that we care about.

08:09.515 --> 08:12.375
The also, a lot of times policy makers,

08:12.375 --> 08:13.695
they care about the performance

08:13.695 --> 08:15.655
of the low performing students

08:15.755 --> 08:17.415
who down the lane may be at the risk

08:17.415 --> 08:18.735
of dropping out of high school.

08:19.355 --> 08:21.615
So what I do is that I see whether any

08:21.615 --> 08:24.175
of these measures are giving us differential information

08:24.175 --> 08:25.255
about the performance

08:25.315 --> 08:27.655
of low versus high performing students.

08:28.195 --> 08:29.575
So I define students

08:29.675 --> 08:33.175
to be low performing if they have prior achievement

08:33.235 --> 08:36.055
or ap, prior achievement below the median

08:36.435 --> 08:38.575
and prior absences above the median

08:38.575 --> 08:40.535
and vice versa for the high performing students.

08:41.275 --> 08:44.775
And one of the major takeaways from that analysis is that

08:45.575 --> 08:49.295
teachers, with high service scores are more effective in

08:49.565 --> 08:51.175
improving the achievement

08:51.275 --> 08:53.175
and reducing the absences of students

08:53.175 --> 08:54.295
that are low performing.

08:54.635 --> 08:57.165
And this has important implications for equity.

08:57.735 --> 09:01.885
There is still a lot of debate about, the effectiveness

09:01.885 --> 09:03.205
of the subjective measures.

09:03.705 --> 09:06.205
Uh, but given what I can see in my study

09:06.225 --> 09:09.165
and what I find is that it doesn't seem to be the case

09:09.165 --> 09:11.885
that these subjective measures are completely noisy

09:11.965 --> 09:13.365
measures of teacher quality.

09:13.945 --> 09:15.565
The students especially, they seem

09:15.565 --> 09:18.885
to be doing a pretty decent job at evaluating the teachers.

09:19.545 --> 09:24.365
Um, also the fact that, the fact that teachers

09:24.365 --> 09:27.085
with better survey scores are more effective in improving

09:27.385 --> 09:29.365
the achievement and reducing the absences

09:29.625 --> 09:31.045
of low performing students,

09:31.145 --> 09:34.005
it has really important implications for equity.

09:34.345 --> 09:37.325
And this finding is consistent with the idea that a caring

09:37.465 --> 09:40.685
or a considerate teacher may particularly matter more

09:40.705 --> 09:43.485
for the students who, you know, who we are concerned

09:43.485 --> 09:45.565
that down the lane may drop out of high school.

09:45.865 --> 09:48.605
So it's, so this finding is very important from the equity

09:48.605 --> 09:49.725
perspective as well.

09:50.465 --> 09:52.685
Um, although I do find that yes,

09:52.745 --> 09:56.165
in this multiple measure system test score value added is

09:56.225 --> 09:58.885
the most significantly related to some

09:58.885 --> 10:01.445
of these valued outcomes that we really care about.

10:01.915 --> 10:04.365
It's very important to keep the context in mind.

10:04.915 --> 10:07.525
This is a context where teachers are not only being

10:07.555 --> 10:09.965
evaluated on the basis of the objective measure,

10:10.425 --> 10:12.205
but also the subjective measures.

10:12.345 --> 10:15.765
And a good measure is not only one that is related

10:15.865 --> 10:18.085
to the outcomes, but also

10:18.085 --> 10:20.125
that mitigates incentive distortions.

10:20.585 --> 10:25.205
So maybe if, the subjective measures were not present,

10:25.865 --> 10:29.045
we may not have observed these strong relationship between,

10:29.425 --> 10:31.445
the objective measure and the outcomes.

10:31.745 --> 10:35.485
And so we definitely need more work in order

10:35.505 --> 10:38.245
to understand the effectiveness of the whole,

10:38.465 --> 10:40.045
in these incentive structures.

10:40.205 --> 10:42.205
Nevertheless, the fact that many states

10:42.225 --> 10:45.045
and districts use these measures to evaluate teachers,

10:45.195 --> 10:48.045
it's reassuring to know that these measures are fruitful,

10:48.345 --> 10:50.485
and giving us some meaningful information.

00:00 - Proliferation of state longitudinal data has resulted in efforts by policymakers to evaluate teacher performance more accurately. And they mainly rely on two types of measures. The first is called the objective measure, which is a quantitative measure of teacher quality.

00:15 - And the most common type of quantitative measure that, is used is called test score value added test score value added basically just captures teacher contributions to student test score growth. And then we also have subjective measures. the most common subjective measures are supervisors or principals observing classroom instruction as well as young students in some cases evaluating teachers. While many states and districts are using a combination of these measures,

00:43 - I guess the big picture question is, does the combination really helps identify teachers that are raising valuable skills? So in order to get at this big picture question, I'm using the context of a large urban public school district that had this reform where they're evaluated the teachers on the basis of task or value added supervisor observations and student surveys, and then with some qualifications they tie their subsequent to your salaries to it.

01:10 - So using this context, I'm going to answer three main questions in my study. The first is, are these evaluation measures significantly related to the student's cognitive as well as non-cognitive skills? I use test scores as proxies for cognitive skills and absences in receipt of suspensions. They serve as proxies for non-cognitive skills. The next question I ask is, are these measures, giving us any information on top of one another?

01:38 - And are they significantly related to the student's longer term success? And I measure the longer term success by their subsequent year outcomes. My paper contributes to two strands of literature regarding the measurement of worker productivity. The first is related to the effectiveness of the evaluation measures. So there's a rich theoretical literature regarding what is an optimal incentive contract where agents can,

02:04 - you know, agent output is multi-dimensional. There's work by TRO and Milgram and Baker. And, what I'm doing is, in the context of education, there is a famous work by Jackson. And now there are other papers that talk about the fact that teachers, they contribute to the development of a lot of skills. Teacher quality is multi-dimensional, however, test score value added does not fully capture everything that a teacher is doing.

02:31 - So that's the first strand of literature. My paper also contributes to the subjective performance assessment literature. there's work by Jacob and Nev Grant Craft et all that talks about the fact, about the relationship between, the supervisor observations and student test test scores. And I am extending that strand of the literature by presenting new evidence on supervisor observations as well as student

02:59 - Surveys. Let's take a step back and talk about the teacher evaluation framework in the United States and how it has evolved over time. So historically, the evaluations were primarily based on supervisor or classroom observations by principals. Now, the advantage of supervisor observations is that they can be readily conducted for most teachers and can potentially give them, good feedback.

03:23 - But in the recent years, there has been a push for having some quantitative measure that is test score value added, be a part of the evaluation framework, mainly because of concerns that supervisor observations are often lenient. Now, the good thing about the quantitative measure is that it really gives us like a quantitative idea of how much teachers are contributing to the student test scores.

03:46 - But this measure as a standalone measure also has certain potential drawbacks. It can induce a perverse incentive for strategic behaviors, and there's a disproportionate focus on a single type of outcome. So one solution which a lot of states now use is that they use a combination of the two measures. However, in the presence of test-based accountability, even the combination of the two may not be able

04:10 - to blunt the adverse effects of each. For example, principals may be hesitant to give lower ratings to teachers who are concerned are engaging in strategic behavior. So one potential solution, and this has gained more light after the measure of effective teaching project and the Obama race to the top initiative, is the idea of also including student surveys in the evaluation process. So students, they spend the most time with the teachers. So we may expect their evaluations

04:41 - to tell us something meaningful about the teacher quality, but at the same time, there are concerns that students may discriminate against teachers or, you know, this measure, inclusion of this measure may use a popularity contest. So it's really important to see whether this measure or any of the measures that are part of the teacher evaluation framework are really telling us something that the teacher is doing. In 2014,

05:05 - a large urban public school district had this reform where they evaluate the teachers on the basis of student achievement, supervisor observations and student surveys, and then they assign teachers evaluation, scores based on these measures, which are then tied to categorical evaluation ratings, which with some qualifications and determine the teacher subsequent year salaries. So I have administrative data

05:31 - provided by the district for the student test scores, absences and suspensions. I have teacher level data as well. And I'm going to focus on grades three to seven for the years 2014 till 2019, there is an issue of correlated errors. So for example, students that are better performing may just be predisposed to giving teachers better ratings, and that may not necessarily reflect

05:54 - what the teacher is doing. So in order to get around with that issue, and this is something that is standard in the literature, instead of regressing the student outcomes on these evaluation measures, I first create what is called a lever out evaluation measures. So I predict a teacher's test score value added or the subjective scores in year T using all her years except for Year T to avoid biasing the estimate. And then what I do is

06:22 - that I regress the student outcomes on these leave year out measures. And I also include some controls that is student level controls. Classroom level controls school by year and grade by year fixed effect. In my model, I present new evidence regarding the, relationship between the objective and the subjective measures and the student skills. So I'll start off with this objective measure

06:45 - that is supervisor observations. So I find that the supervisor observations are pretty significantly related to the students, subsequent year, as well as contemporaneous, achievement. And they give us additional information on top of what we learned by test score value added for the contemporaneous as well as subsequent year reading achievement. Interestingly, the student surveys are also giving us information about the student's contemporaneous

07:13 - and subsequent year achievement. And of the three measures, teachers with high service scores, they seem to be the most effective in reducing the student's contemporaneous absences For the subject for the objective measure test score value added, I find that test score value added is pretty significantly related to not only contemporaneous achievement, but also contemporaneous absences as well as subsequent year achievement and absences.

07:41 - So in some, the subjective measures are giving us information about the students' current tier as well as subsequent tier achievement and contemporaneous absences, and they're giving us information on top of what we are learning by what test score value added. However, in this multiple measure system test score value added is the most significantly related to most of the outcomes, the valued outcomes that we care about.

08:09 - The also, a lot of times policy makers, they care about the performance of the low performing students who down the lane may be at the risk of dropping out of high school. So what I do is that I see whether any of these measures are giving us differential information about the performance of low versus high performing students. So I define students

08:29 - to be low performing if they have prior achievement or ap, prior achievement below the median and prior absences above the median and vice versa for the high performing students. And one of the major takeaways from that analysis is that teachers, with high service scores are more effective in improving the achievement and reducing the absences of students that are low performing. And this has important implications for equity.

08:57 - There is still a lot of debate about, the effectiveness of the subjective measures. but given what I can see in my study and what I find is that it doesn't seem to be the case that these subjective measures are completely noisy measures of teacher quality. The students especially, they seem to be doing a pretty decent job at evaluating the teachers. also the fact that, the fact that teachers with better survey scores are more effective in improving

09:27 - the achievement and reducing the absences of low performing students, it has really important implications for equity. And this finding is consistent with the idea that a caring or a considerate teacher may particularly matter more for the students who, you know, who we are concerned that down the lane may drop out of high school. So it's, so this finding is very important from the equity perspective as well. although I do find that yes,

09:52 - in this multiple measure system test score value added is the most significantly related to some of these valued outcomes that we really care about. It's very important to keep the context in mind. This is a context where teachers are not only being evaluated on the basis of the objective measure, but also the subjective measures. And a good measure is not only one that is related to the outcomes, but also that mitigates incentive distortions.

10:20 - So maybe if, the subjective measures were not present, we may not have observed these strong relationship between, the objective measure and the outcomes. And so we definitely need more work in order to understand the effectiveness of the whole, in these incentive structures. Nevertheless, the fact that many states and districts use these measures to evaluate teachers, it's reassuring to know that these measures are fruitful, and giving us some meaningful information.

Faculti AI can make mistakes