Popular in Professional

Culture and Society

The City as a Growth Machine: Toward a Political Economy of Place

New York University Harvey Moltoch

Education

The Role of Education in Reducing Racial Inequality

University of Southern California Pedro A. Noguera

Law

Diversity and Social Mobility in the UK legal profession

Law Society I. Stephanie Boyce

Internet Law

The Law of AI for Good

University of San Diego Orly Lobel

Pharmacy

Cytotoxic Activity and Anti-Migratory Effect of Berberine-Phytantriol Liquid Crystalline on Non-Small-Cell Lung Cancer

University of Technology Sydney Kamal Dua

Human Rights Law

Transforming Restorative Justice for Transitional Settings

University of Nottingham Kerry Clamp

Education

JD-Next: A valid and reliable tool to predict diverse students’ success in law school

View more insights in Professional - Law - Socio-legal Studies

View more figures from Boston University

View the page of Christopher Robertson

Read the Study

Christopher Robertson discusses how attacks against admissions exams have grown more frequent from those trying to increase access and lessen inequality in higher education.

Image courtesy of interviewee. January 16, 2024

Generating Workspace

Export as PDF

This video video discusses the challenges faced by law schools in the United States regarding enrollment and preparation disparities among students based on race, gender, ethnicity, and wealth. It highlights the development of a new program, JD Next, aimed at better preparing aspiring law students and predicting their success without reproducing the disparities seen in traditional standardized tests like the LSAT.

JD Next combines a preparatory course and an exam designed to assess students' understanding of legal principles and their ability to analyze common law cases. This video video approach focuses on the learning potential of students in a supportive environment, rather than merely measuring fixed intelligence. The program aims to provide equitable preparation for all students, regardless of their background.

The outcomes of JD Next have shown predictive validity akin to the LSAT while significantly reducing racial disparities in scores. It was found that the racial score gap observed in LSAT scores was largely eliminated with JD Next, making it a more equitable assessment tool. Furthermore, JD Next demonstrated strong incremental validity across various law schools, meaning it provides useful predictive information beyond what legacy exams offer.

The findings suggest that law schools should adopt JD Next as a standardized testing solution to mitigate historical inequalities in legal education admissions. This video video argues for broader implications of this approach in standardized testing across various fields, advocating for methods that assess students' abilities in context, rather than solely testing fixed intelligence levels.

Language:

Export as PDF

Export as PDF
Here are some articles that discuss similar themes around law school admissions, equitable assessment practices, and addressing disparities in educational settings:

1. **"Reimagining Law School Admissions: Moving Beyond the LSAT"**
This article explores the limitations of the LSAT and discusses alternative assessment methods that promote diversity and inclusion within legal education. It analyzes various programs aimed at equitable preparation for law students from underrepresented backgrounds.

2. **"Equity in Legal Education: A New Approach to Assessing Law Student Potential"**
This piece highlights initiatives similar to JD Next that aim to reform admissions processes in law schools. It features interviews with educators and policymakers advocating for assessments that better predict student success based on potential rather than traditional testing.

3. **"Addressing Racial Disparities in Law School Admissions: New Strategies and Solutions"**
An exploration of recent changes in law school admissions policies, this article discusses how schools are working to eliminate racial and socioeconomic disparities through innovative assessment tools and programs, echoing the goals of JD Next.

4. **"The Case for Holistic Admissions in Legal Education"**
This article presents arguments for a holistic approach to admissions, emphasizing the importance of assessing a broader range of student qualities and capabilities. It provides examples of law schools implementing such approaches to foster diversity and inclusion.

5. **"Evaluating Alternatives to Standardized Testing in Higher Education"**
A comprehensive review of various alternative assessments being considered across higher education institutions, this article convideoualizes the JD Next program within a larger movement to rethink standardized testing and create more equitable admissions processes.

6. **"The Future of Law School Testing: Moving Toward Fairness and Equity"**
This article examines emerging trends in law school assessment, including the adoption of programs like JD Next. It discusses the impact of these trends on diversity within law schools and the legal profession.

7. **"Legal Education and Social Justice: Addressing Inequities through Reform"**
Focusing on how legal education can become a tool for social justice, this article discusses various reform efforts, including alternative testing methodologies, to support historically marginalized students in their pursuit of legal careers.

These articles collectively highlight the ongoing efforts and discussions surrounding equitable practices in law school admissions, assessment methods, and the implications for the legal profession as a whole.

X-TIMESTAMP-MAP:LOCAL=00:00:00.000,MPEGTS=900000

00:00.055 --> 00:02.085
We've been looking at law school enrollments in the

00:02.085 --> 00:05.645
United States, that have really suffered from,

00:05.675 --> 00:08.205
from problems that, that our legal profession doesn't

00:08.485 --> 00:11.565
represent the American population in terms of race, in terms

00:11.585 --> 00:14.365
of, gender, in terms of ethnicity, in terms of wealth.

00:14.785 --> 00:17.245
And so we realize that law schools are the gateway

00:17.385 --> 00:18.485
to the profession.

00:18.905 --> 00:20.085
And so we have to make sure

00:20.085 --> 00:21.405
that when people enter law schools,

00:21.405 --> 00:23.245
that they are well prepared to succeed,

00:23.585 --> 00:26.565
and that we have a, a method to predict their success,

00:26.975 --> 00:30.885
which doesn't reproduce the large racial and ethnic and,

00:30.985 --> 00:33.045
and class, disparities

00:33.045 --> 00:35.125
that we see in the standard entrance exams

00:35.125 --> 00:37.005
that exist here in the us I sometimes called those the

00:37.005 --> 00:38.085
legacy exams.

00:38.865 --> 00:42.165
And so we saw people when they did come to law school in

00:42.165 --> 00:44.725
that very first year, there was just wildly different levels

00:44.745 --> 00:47.645
of preparation because in the US Law school is a graduate

00:47.645 --> 00:50.365
degree, and there's no particular undergraduate

00:50.365 --> 00:51.525
degree that's required.

00:51.875 --> 00:54.605
Some students could have studied chemistry as an undergrad,

00:54.785 --> 00:57.885
others could have studied, political theory, sort

00:57.885 --> 00:58.885
of everything in between.

00:59.345 --> 01:01.925
And so, we needed a way to level them up

01:02.185 --> 01:05.005
and, to make sure everyone had that same basic level

01:05.105 --> 01:07.685
of preparation for what's a fairly unique educational

01:07.685 --> 01:09.365
experience in the jd.

01:09.905 --> 01:12.525
So we developed JD next as both a course

01:12.905 --> 01:15.725
and an exam tailored together to serve those two goals.

01:15.975 --> 01:18.165
Goals. I think the, the course really is essential

01:18.185 --> 01:19.485
to our method of testing.

01:19.785 --> 01:22.445
It, it grows out of tradition of testing that goes back

01:22.445 --> 01:24.805
to the 1920s with Vygotsky

01:24.805 --> 01:27.285
and others who said, let's not just look at the fixed

01:27.685 --> 01:29.565
capacities that students have, because

01:29.565 --> 01:32.845
after all, in in education, we are in the business of,

01:32.905 --> 01:33.965
of human development.

01:33.965 --> 01:36.405
We're in the business of helping people grow from

01:36.405 --> 01:37.685
where they start to where they end.

01:37.985 --> 01:39.205
So what we really wanna be able

01:39.205 --> 01:41.045
to see is their learning potential.

01:41.145 --> 01:43.565
How much could they grow in a supporting,

01:43.655 --> 01:45.125
supportive learning environment?

01:45.585 --> 01:48.445
So by linking the course to the test, we're able

01:48.445 --> 01:52.285
to give the students all the supports they need, and,

01:52.285 --> 01:54.165
and then when the, in the test, we can see

01:54.505 --> 01:55.725
how prepared they really are.

01:56.265 --> 01:59.275
Whereas in standardized testing with these legacy exams,

01:59.625 --> 02:01.875
they instead say they're, they're largely tests

02:02.015 --> 02:03.675
of intelligence or aptitude,

02:04.135 --> 02:06.155
and the students are left to prepare

02:06.175 --> 02:07.275
for these tests on their own.

02:07.375 --> 02:10.155
And some students will spend hundreds of hours

02:10.855 --> 02:13.555
and, thousands of dollars getting private tutoring,

02:14.095 --> 02:16.195
but of course not everybody has that hundreds of hours

02:16.295 --> 02:17.355
or thousands of dollars.

02:18.175 --> 02:20.155
Um, and so that's one of the mechanisms

02:20.155 --> 02:22.195
that creates the disparities that I'm talking about.

02:22.895 --> 02:26.635
So instead, when we say we provide everything you need

02:26.635 --> 02:28.795
to prepare as part of the course experience,

02:29.605 --> 02:31.955
let's give you the modern pedagogies

02:31.955 --> 02:34.955
with formative assessments and space repetition

02:35.215 --> 02:38.755
and, we're able to bring everybody up to a level

02:38.755 --> 02:41.035
where we can more fairly see their performance.

02:41.465 --> 02:44.675
It's a, a a four hour exam, in,

02:44.695 --> 02:47.115
in various implementations, it can be three or four hours.

02:47.625 --> 02:50.275
That really tests what we've taught in the course,

02:50.275 --> 02:54.035
which is primarily how to read common law cases,

02:54.375 --> 02:56.435
and, how to spot the issues

02:56.535 --> 02:58.795
and understand the legal principles. A lot of

02:58.795 --> 03:01.645
What a lawyer does is extract law from these

03:01.715 --> 03:02.925
long written opinions.

03:03.195 --> 03:05.325
It's definitely what a first year law student does.

03:05.665 --> 03:08.245
So we're just very directly testing their ability to do

03:08.245 --> 03:09.925
so in some domains.

03:09.925 --> 03:12.365
This is called a curriculum sampling exam,

03:12.365 --> 03:14.405
because we're actually taking what we do in the beginning

03:14.405 --> 03:17.805
of law school and pulling it back, pre-teaching it

03:17.925 --> 03:20.005
and testing on it to, as, as

03:20.005 --> 03:21.805
what we theorize will be the best predictor of

03:21.805 --> 03:23.645
how you'll be able to actually do in law school.

03:23.645 --> 03:25.685
Since it's, it's kind of a slice of law school.

03:26.145 --> 03:29.605
We, studied JD next across several cohorts,

03:29.625 --> 03:30.925
in the first,

03:31.065 --> 03:32.845
but for the validity study where we're looking at

03:32.845 --> 03:36.405
how well the test predicts law school performance,

03:36.505 --> 03:38.965
in this study we have, we, in the first year,

03:38.985 --> 03:41.005
we recruited several hundred students from

03:41.005 --> 03:42.925
around the country who were aspiring to law school,

03:43.345 --> 03:45.725
and we actually randomized them into a placebo course

03:46.265 --> 03:47.725
and, a treatment course.

03:48.225 --> 03:50.925
Uh, and then we gave them all the exam at the, at the end,

03:51.185 --> 03:53.045
and we incentivized them to do well.

03:53.385 --> 03:55.805
We incentivized them throughout the course,

03:55.945 --> 03:58.565
and, had a, had a, a lottery based drawing.

03:58.565 --> 04:00.565
The better they do on the exam,

04:00.585 --> 04:02.925
the more money they would potentially receive back.

04:03.305 --> 04:06.125
We, we wanted to simulate some skin in the game,

04:06.125 --> 04:08.445
like a real standardized test will, where it could,

04:08.445 --> 04:10.245
it could shape your attendance in law school

04:10.585 --> 04:12.605
and shape the amount of scholarships you could get.

04:13.225 --> 04:14.965
Our incentives aren't anywhere near that large,

04:14.985 --> 04:17.765
but we wanted to have, at least some skin in the game.

04:18.345 --> 04:22.445
Uh, and then we simply, secured from the, the schools

04:22.455 --> 04:24.285
where they participated in the second year

04:24.585 --> 04:26.485
or from the students themselves in the first

04:26.485 --> 04:27.965
year, we secured their grades.

04:28.185 --> 04:31.165
And so it's a, it's a fairly straightforward correlational

04:31.175 --> 04:35.005
study to just look at whether our exam,

04:35.145 --> 04:37.445
is a valid and reliable predictor of,

04:37.545 --> 04:38.885
of their law school performance,

04:38.985 --> 04:40.365
either in the first semester

04:40.385 --> 04:42.205
or in the first year,

04:42.235 --> 04:43.645
depending on the cohort we studied.

04:43.865 --> 04:45.925
So we found, significant,

04:45.925 --> 04:49.845
predictive power at about the same rate you see on the lsat.

04:49.865 --> 04:52.365
For example, the law school admissions test is sort

04:52.365 --> 04:54.725
of the gold standard in our field for American law schools.

04:55.225 --> 04:58.325
We found that we were predicting, about the same.

04:58.385 --> 04:59.645
And one cohort is a little more,

04:59.645 --> 05:01.005
and another cohort is a little less.

05:01.105 --> 05:03.525
But, essentially the same level of prediction

05:03.525 --> 05:05.485
by this test that's been around and been optimized now

05:05.485 --> 05:06.685
for 40 or 50 years.

05:07.425 --> 05:09.805
So right out of the gate, we saw that as a win.

05:09.855 --> 05:11.685
We're sort of playing on the field,

05:12.425 --> 05:16.005
but the next few hypotheses were, were even more exciting.

05:16.105 --> 05:18.405
And the first one is we wanted to, to look

05:18.405 --> 05:21.725
and see whether we had score disparities by race.

05:22.105 --> 05:24.925
And, on the lsat, according to,

05:24.925 --> 05:28.965
that organization's own data, black law students have about,

05:29.065 --> 05:33.565
black, test takers actually have about,

05:33.705 --> 05:37.125
an 11 point gap, compared to white test takers.

05:37.625 --> 05:39.525
Um, for the scientists that, I'll tell you

05:39.525 --> 05:42.765
that is two standard deviations difference, in score,

05:42.765 --> 05:45.085
which is pretty much humongous from a

05:45.085 --> 05:46.125
social science perspective.

05:46.785 --> 05:49.205
So we saw that as the problem to be solved.

05:49.425 --> 05:53.525
And in our data, that gap, is cut by,

05:53.525 --> 05:56.325
three-fourths and becomes non-significant,

05:56.325 --> 05:58.565
statistically, in a paired study

05:58.565 --> 06:01.125
with the same sample sizes for both the lsat,

06:01.185 --> 06:02.685
and the the JD Next score.

06:03.145 --> 06:05.325
So when you put those two findings together, as we have

06:05.325 --> 06:07.885
as much predictive power as the LSAT without any

06:07.885 --> 06:10.365
of the significant racial disparities, was super exciting.

06:11.185 --> 06:13.325
The third thing we looked at is,

06:13.325 --> 06:15.685
there are actually 200 law schools in the United States,

06:16.125 --> 06:18.285
although they all have essentially the same curriculum.

06:18.745 --> 06:19.885
I'm generalizing a little bit,

06:19.885 --> 06:21.485
some people won't like me saying that,

06:21.485 --> 06:23.245
but they basically have the same curriculum.

06:23.865 --> 06:25.325
Um, but they have

06:25.945 --> 06:28.765
wildly different admission standards from students that are,

06:28.765 --> 06:32.525
you know, 4.0 students maxing out the LSAT at, at a handful

06:32.585 --> 06:36.085
of places to others where students, are, are,

06:36.105 --> 06:37.245
are on the verge of not getting

06:37.245 --> 06:38.485
into any of the law schools at all.

06:39.025 --> 06:42.045
And so we were able to break down our sample to look at,

06:42.265 --> 06:46.245
top 50 schools in terms of the rankings, next 50 schools

06:46.505 --> 06:48.365
and, and the bottom set of a hundred schools.

06:49.105 --> 06:51.205
And we looked at the predictive power in each

06:51.205 --> 06:53.445
of those groupings and, str and,

06:53.545 --> 06:54.925
and not just the correlation.

06:55.465 --> 06:57.845
You wanna know, not just whether the test predicts

06:57.955 --> 07:00.605
performance, but whether it tells you something you didn't

07:00.605 --> 07:02.205
already know, right?

07:02.265 --> 07:03.685
So you want not just validity,

07:03.685 --> 07:05.645
but what's called incremental validity.

07:06.185 --> 07:08.165
And, if it's, and,

07:08.165 --> 07:09.205
and the way you study that is

07:09.205 --> 07:12.445
by having a base regression model, and you take it

07:12.865 --> 07:15.245
and you see if, if your predictive power changes when you

07:15.265 --> 07:17.045
add in, the test score.

07:17.545 --> 07:19.925
And in that area, we were very excited to find

07:19.925 --> 07:23.445
that the JD next had strong incremental validity across all

07:23.445 --> 07:24.445
three types of schools.

07:25.065 --> 07:27.325
Um, but, the legacy exams didn't actually.

07:27.385 --> 07:29.965
And, and that's another huge finding that, that,

07:29.965 --> 07:31.525
frankly we haven't emphasized enough,

07:31.985 --> 07:34.045
but it suggests our current practices we're using

07:34.065 --> 07:36.845
for 50 years or more may not actually be providing

07:36.845 --> 07:39.405
information to, law schools,

07:39.405 --> 07:40.685
outside the very top ones.

07:40.705 --> 07:42.005
And frankly, those are the ones we're

07:42.005 --> 07:43.045
most worried about, right?

07:43.045 --> 07:44.565
When a student is going to,

07:44.565 --> 07:46.805
think about taking on a hundred thousand dollars of debt

07:47.425 --> 07:49.365
to go to a law school that

07:49.575 --> 07:53.325
where the career outcomes are maybe not as strong,

07:53.545 --> 07:55.285
we, that's where we really need the test

07:55.305 --> 07:58.405
to inform the school and inform the student, about,

07:58.465 --> 08:00.405
you know, their likelihood of success in that program.

08:00.785 --> 08:03.965
So, the fact that the JD next works across the board,

08:04.265 --> 08:06.885
without those big racial disparities,

08:07.185 --> 08:08.285
was a huge finding for us.

08:08.455 --> 08:10.605
First of all, law schools need

08:10.605 --> 08:11.725
to start using this test, right?

08:11.725 --> 08:14.925
Because the current exams they're using are creating these

08:14.925 --> 08:16.085
huge racial disparities.

08:16.825 --> 08:19.605
And historically they've tried to solve for that

08:19.745 --> 08:22.845
by having racial preferences, by saying, well, you know,

08:22.845 --> 08:24.565
we need to admit people of this race

08:24.585 --> 08:27.245
or that ethnicity, even though they have lower scores.

08:27.985 --> 08:31.085
But, the, in the us, the Supreme Court,

08:31.105 --> 08:33.245
in a big decision has made that illegal now.

08:33.345 --> 08:36.965
So, so the schools using the legacy exam are in a

08:36.965 --> 08:38.005
bit of a double bind.

08:38.065 --> 08:39.805
On the one hand, the tool they're using is

08:40.165 --> 08:41.285
creating big disparities.

08:41.505 --> 08:43.925
On the other hand, the Supreme Court says you can't look at

08:43.925 --> 08:46.525
race to counter to counteract those disparities.

08:47.145 --> 08:49.925
So, JD Next is coming in at, at, I think,

08:50.005 --> 08:53.125
a really opportune time to provide a, a different solution

08:53.195 --> 08:54.805
that avoids those racial disparities

08:54.805 --> 08:55.805
In the first place.

08:56.345 --> 08:58.685
So the first policy implication is schools need

08:58.685 --> 08:59.685
to be allowed to use this,

08:59.685 --> 09:01.805
and there's some accreditation complications around it,

09:02.185 --> 09:05.085
and they probably really need to be adopting it,

09:05.085 --> 09:06.285
aggressively as more and more

09:06.285 --> 09:07.645
data's coming out about its effects.

09:08.525 --> 09:10.605
I think the second big policy implication is

09:11.165 --> 09:14.205
I think we have, proven this approach

09:14.305 --> 09:16.285
to standardized testing more generally,

09:16.675 --> 09:18.005
that we should be going back

09:18.005 --> 09:19.645
to these theories from the 1920s

09:19.865 --> 09:22.845
and looking at this dynamic proximal curriculum sampling

09:22.845 --> 09:25.605
testing, it's a way of allowing people

09:25.605 --> 09:27.685
to do a tryout essentially for school,

09:27.685 --> 09:30.685
whether it's not just law, but say engineering or medicine

09:30.745 --> 09:34.445
or these other fields, instead of just testing intelligence.

09:34.445 --> 09:37.325
Like it's a fixed concept that, you know, largely you're,

09:37.325 --> 09:39.485
you're born with and is baked into your economics.

09:39.825 --> 09:42.085
Uh, from the very early age, we need

09:42.085 --> 09:44.205
to reconsider giving people a chance to,

09:44.305 --> 09:46.925
to prove their ability through programs like jdx.

09:46.945 --> 09:49.045
So I think that's the next policy implication,

09:49.045 --> 09:51.405
is if we're gonna continue having standardized tests at all,

09:51.905 --> 09:54.725
this approach is probably a more promising one.

00:00 - We've been looking at law school enrollments in the United States, that have really suffered from, from problems that, that our legal profession doesn't represent the American population in terms of race, in terms of, gender, in terms of ethnicity, in terms of wealth. And so we realize that law schools are the gateway

00:17 - to the profession. And so we have to make sure that when people enter law schools, that they are well prepared to succeed, and that we have a, a method to predict their success, which doesn't reproduce the large racial and ethnic and, and class, disparities that we see in the standard entrance exams that exist here in the us I sometimes called those the legacy exams.

00:38 - And so we saw people when they did come to law school in that very first year, there was just wildly different levels of preparation because in the US Law school is a graduate degree, and there's no particular undergraduate degree that's required. Some students could have studied chemistry as an undergrad, others could have studied, political theory, sort of everything in between. And so, we needed a way to level them up and, to make sure everyone had that same basic level

01:05 - of preparation for what's a fairly unique educational experience in the jd. So we developed JD next as both a course and an exam tailored together to serve those two goals. Goals. I think the, the course really is essential to our method of testing. It, it grows out of tradition of testing that goes back to the 1920s with Vygotsky and others who said, let's not just look at the fixed capacities that students have, because

01:29 - after all, in in education, we are in the business of, of human development. We're in the business of helping people grow from where they start to where they end. So what we really wanna be able to see is their learning potential. How much could they grow in a supporting, supportive learning environment? So by linking the course to the test, we're able to give the students all the supports they need, and,

01:52 - and then when the, in the test, we can see how prepared they really are. Whereas in standardized testing with these legacy exams, they instead say they're, they're largely tests of intelligence or aptitude, and the students are left to prepare for these tests on their own. And some students will spend hundreds of hours and, thousands of dollars getting private tutoring, but of course not everybody has that hundreds of hours

02:16 - or thousands of dollars. and so that's one of the mechanisms that creates the disparities that I'm talking about. So instead, when we say we provide everything you need to prepare as part of the course experience, let's give you the modern pedagogies with formative assessments and space repetition and, we're able to bring everybody up to a level where we can more fairly see their performance. It's a, a a four hour exam, in,

02:44 - in various implementations, it can be three or four hours. That really tests what we've taught in the course, which is primarily how to read common law cases, and, how to spot the issues and understand the legal principles. A lot of What a lawyer does is extract law from these long written opinions. It's definitely what a first year law student does. So we're just very directly testing their ability to do so in some domains.

03:09 - This is called a curriculum sampling exam, because we're actually taking what we do in the beginning of law school and pulling it back, pre-teaching it and testing on it to, as, as what we theorize will be the best predictor of how you'll be able to actually do in law school. Since it's, it's kind of a slice of law school. We, studied JD next across several cohorts, in the first, but for the validity study where we're looking at

03:32 - how well the test predicts law school performance, in this study we have, we, in the first year, we recruited several hundred students from around the country who were aspiring to law school, and we actually randomized them into a placebo course and, a treatment course. and then we gave them all the exam at the, at the end, and we incentivized them to do well. We incentivized them throughout the course, and, had a, had a, a lottery based drawing.

03:58 - The better they do on the exam, the more money they would potentially receive back. We, we wanted to simulate some skin in the game, like a real standardized test will, where it could, it could shape your attendance in law school and shape the amount of scholarships you could get. Our incentives aren't anywhere near that large, but we wanted to have, at least some skin in the game. and then we simply, secured from the, the schools where they participated in the second year

04:24 - or from the students themselves in the first year, we secured their grades. And so it's a, it's a fairly straightforward correlational study to just look at whether our exam, is a valid and reliable predictor of, of their law school performance, either in the first semester or in the first year, depending on the cohort we studied. So we found, significant,

04:45 - predictive power at about the same rate you see on the lsat. For example, the law school admissions test is sort of the gold standard in our field for American law schools. We found that we were predicting, about the same. And one cohort is a little more, and another cohort is a little less. But, essentially the same level of prediction by this test that's been around and been optimized now for 40 or 50 years. So right out of the gate, we saw that as a win.

05:09 - We're sort of playing on the field, but the next few hypotheses were, were even more exciting. And the first one is we wanted to, to look and see whether we had score disparities by race. And, on the lsat, according to, that organization's own data, black law students have about, black, test takers actually have about, an 11 point gap, compared to white test takers. for the scientists that, I'll tell you that is two standard deviations difference, in score,

05:42 - which is pretty much humongous from a social science perspective. So we saw that as the problem to be solved. And in our data, that gap, is cut by, three-fourths and becomes non-significant, statistically, in a paired study with the same sample sizes for both the lsat, and the the JD Next score. So when you put those two findings together, as we have as much predictive power as the LSAT without any

06:07 - of the significant racial disparities, was super exciting. The third thing we looked at is, there are actually 200 law schools in the United States, although they all have essentially the same curriculum. I'm generalizing a little bit, some people won't like me saying that, but they basically have the same curriculum. but they have wildly different admission standards from students that are, you know, 4.0 students maxing out the LSAT at, at a handful

06:32 - of places to others where students, are, are, are on the verge of not getting into any of the law schools at all. And so we were able to break down our sample to look at, top 50 schools in terms of the rankings, next 50 schools and, and the bottom set of a hundred schools. And we looked at the predictive power in each of those groupings and, str and, and not just the correlation. You wanna know, not just whether the test predicts

06:57 - performance, but whether it tells you something you didn't already know, right? So you want not just validity, but what's called incremental validity. And, if it's, and, and the way you study that is by having a base regression model, and you take it and you see if, if your predictive power changes when you add in, the test score. And in that area, we were very excited to find

07:19 - that the JD next had strong incremental validity across all three types of schools. but, the legacy exams didn't actually. And, and that's another huge finding that, that, frankly we haven't emphasized enough, but it suggests our current practices we're using for 50 years or more may not actually be providing information to, law schools, outside the very top ones. And frankly, those are the ones we're

07:42 - most worried about, right? When a student is going to, think about taking on a hundred thousand dollars of debt to go to a law school that where the career outcomes are maybe not as strong, we, that's where we really need the test to inform the school and inform the student, about, you know, their likelihood of success in that program. So, the fact that the JD next works across the board, without those big racial disparities,

08:07 - was a huge finding for us. First of all, law schools need to start using this test, right? Because the current exams they're using are creating these huge racial disparities. And historically they've tried to solve for that by having racial preferences, by saying, well, you know, we need to admit people of this race or that ethnicity, even though they have lower scores. But, the, in the us, the Supreme Court,

08:31 - in a big decision has made that illegal now. So, so the schools using the legacy exam are in a bit of a double bind. On the one hand, the tool they're using is creating big disparities. On the other hand, the Supreme Court says you can't look at race to counter to counteract those disparities. So, JD Next is coming in at, at, I think, a really opportune time to provide a, a different solution that avoids those racial disparities

08:54 - In the first place. So the first policy implication is schools need to be allowed to use this, and there's some accreditation complications around it, and they probably really need to be adopting it, aggressively as more and more data's coming out about its effects. I think the second big policy implication is I think we have, proven this approach to standardized testing more generally,

09:16 - that we should be going back to these theories from the 1920s and looking at this dynamic proximal curriculum sampling testing, it's a way of allowing people to do a tryout essentially for school, whether it's not just law, but say engineering or medicine or these other fields, instead of just testing intelligence. Like it's a fixed concept that, you know, largely you're, you're born with and is baked into your economics. from the very early age, we need

09:42 - to reconsider giving people a chance to, to prove their ability through programs like jdx. So I think that's the next policy implication, is if we're gonna continue having standardized tests at all, this approach is probably a more promising one.

Faculti AI can make mistakes