Unequal Results, Unequal Society

Some caveats are in order. First, it’s important to acknowledge that yes, SAT results reflect inequalities in race and social class. Black and Hispanic students and poor students do not perform as well on these tests as their white and affluent counterparts, mirroring state-mandated standardized test results and grading distributions. But this reflects a symptom of larger inequality, not a biased test.

Poorer students and racial minorities suffer in these testing outcomes because of socioeconomic inequalities that dog our entire educational system. Test development includes a process called differential item functioning. In this diagnostic, students from different demographic backgrounds are matched by ability and take the same test items. Once overall ability is controlled for, black students do just as well as white on the same items, as do poor students and wealthier ones.

Racial and class inequalities in the SAT certainly are troubling — but only insofar as they show our persistently unequal society.

Critics of standardized tests often complain that affluent students have greater access to test prep materials and coaching. This is indeed a concern, but the research here is clear: coaching services produce far smaller gains than those advertised by the big test-prep companies, which routinely claim triple-digit improvements.

A 2006 meta-analysis found that students retaking the SAT after coaching resulted in, on average, an increase of about 50 points on a 1600 scale. That’s not an insignificant number. However, as the researchers point out, we can expect some of that gain to occur simply through increased familiarity with the test and, for lower-scoring students — the type most likely to retake the test — regression to the mean. More recent research found that, after using statistical controls to compare similar students, the combined effect of coaching on a 1600 point scale was about 20 points.

It’s also worth noting that affluent parents can hire a wide variety of coaches and tutors for their children to improve their grades, performance in sports, or ability in extracurricular activities, all of which impact their chances of college admission. Coaching is thus far from a problem with testing alone. Yes, the game is rigged, but it’s rigged from the top to bottom — not just with the SAT.

The Flaws of “Holistic Assessments”

Let’s look at the alternative metrics for student performance: “soft” criteria and high school grade-point average (GPA).

Detractors of entrance exams often argue for more “holistic” methods of evaluating students than tests, pushing for greater emphasis on student activities, college essays, and letters of recommendation. They argue that these things allow them to select students that are more than just grades and test scores and build a diverse student body. As Jennifer Finney Boylan put it in a piece decrying the SAT, the only way to fairly choose between applicants is “to look at the complex portrait of their lives.”

But this reasoning goes directly against the stated goal of equality. It should be obvious: affluent parents have far greater ability to provide opportunities for extracurricular (and frequently out-of-school) activities than less affluent parents do.

The student who is captain of the sailing team, president of the robotics club, and who spent a summer building houses in the Global South will likely look more “holistically” valuable than a poorer student who has not had the resources to do similar activities. Who is more likely to be a star violin player or to have completed a summer internship at a fancy magazine: a poor student or an affluent one? College essays are more easily improved through coaching than test scores, and teachers at expensive private schools likely feel more pressure to write effusive letters of recommendation than their peers in public schools.

Favoring the “soft” aspects of a college application is straightforwardly beneficial to the more privileged at the expense of the less.

There’s another reason for people who believe in equality to champion standardized achievement tests: they serve as one of the only bulwarks against the pernicious effects of high school grade inflation.

Everyone has heard of grade inflation in college. Depending on your point of view, it’s either a sign of the continued degradation of educational standards or a mostly inconsequential statistical curio. But far fewer people seem aware that grade inflation exists in high school. In fact, high school grades are inflating at a rapid pace — but unequally, in a way that should disturb us.

Research by Michael Hurwitz and Jason Lee found that, from 1998 to 2016, the average high school GPA rose from 3.27 to 3.38. That may not sound like much, but distributed over millions of students, it’s a large increase. What’s more, the phenomenon is concentrated at the top. Paul Sackett and Nathan Kuncel, working with a data set from over 300 colleges and a million students, found that the modal GPA — that is, the GPA that occurred most often in the data set — was 4.0. The data set only includes college students, screening out many of the lowest GPA. Still, it demonstrates the extent to which students entering college are now bunched at the top of the grade distribution. Even relative measures like class rank fail to really distinguish grades, as high schools vary widely in how grades are weighted and do not always make this information public to colleges. Some high schools now graduate over 100 students named valedictorians.

Why should progressives care about high school grade inflation? Because it perpetuates traditional inequalities.

In research published in the recent book Measuring Success, Hurwitz and Lee find that whiter schools, more affluent schools, and private schools are all seeing far more grade inflation than higher minority, poorer, and public schools. In fact, public schools have seen little grade inflation; the problem is rampant in private schools, where grades are inflating at three times the rate of public.

You might assume this simply means that schools seeing higher grade inflation are doing a better job of educating students. But when Hurwitz and Lee control for ability by including SAT scores in the model, they still find grades inflating at disparate rates along predictable demographic lines. The potential unfair advantage this may confer in college admissions is clear.

The likely causes of these developments seem obvious. Private schools have a direct financial incentive to please parents in a way that public schools do not. White and affluent parents likely enjoy greater social capital and thus greater ability to pressure teachers, a privilege poor and minority parents largely lack. We could argue that the solution is simply to stop grade inflation. But it’s difficult to imagine a policy fix that could work across the country, and the incentives for teachers of wealthy and white students will only become more obvious as the race to the top of the educational ladder gets more intense.

The SAT and ACT aren’t perfect. But much of the folk wisdom about them and their deficiencies is wrong, and though critics mean well, they actually risk deepening inequality by attacking these tests.

To be clear, I am not advocating an admissions system that uses only the SAT. You only need to look at the various social problems associated with China’s Gaokao exam to see the dangers of placing too much emphasis on any one metric. And I should point out that I have always been a strong supporter of race-based affirmative action programs, which remain necessary to combat traditional inequalities. What I am saying is that the casual assumption that de-emphasizing or eliminating the SAT will result in a more equitable system doesn’t withstand scrutiny.

Unlike their rich peers, students who labor under racial and economic disadvantage have very few ways to distinguish themselves from the rest of the pack. A stellar SAT score is potentially one of the most powerful. We should take care not to rob them of that tool in a misguided push for equality.