Does Reducing the Role of Testing Decrease Bias or Increase It?

A new study of Indian graduates provides some more fodder for the debate about the impact of subjectivity in hiring.

Photo by Ayan Ahmad on Unsplash

In 2020, a range of American colleges and universities dropped their SAT and ACT requirements. This was done for a couple reasons: one, the pandemic made it extremely difficult to administer the tests that spring. But many of these schools had long been skeptical of these standardized exams in the first place.

"The main reason we are looking at SATs is because they are racist,” Kum-Kum Bhavnani, the chair of the University of California system’s Academic Senate, said five years ago.

The SAT/ACT debate was part of a much larger debate about how we should handle admissions and hiring in America. Do objective measures like tests carry too much bias — in the words of the speaker above, are they even racist?

Every policy decision involves tradeoffs. The question we should be asking about objective measures is what the alternative looks like.

In February, I reported on how some universities are slowly changing their minds about dropping testing requirements. Dartmouth, for instance, studied the test-optional policy and found that it actually ended up hurting more disadvantaged students.

Sometimes, objective measures can help someone stand out from the crowd; they can help them prove themselves to a school or employer who otherwise might judge them based off of subjective characteristics. Those of you who’ve followed my writing for a while might be aware that I tutor kids on the side.

For a lot of kids that I mentor who don’t come from elite backgrounds, the SAT has been a way for them to show that they’re just as good of a student as someone who maybe spent their summers in Paris or whose parents paid for regular violin lessons.

But how might this apply to hiring? There are so many different ways that people get jobs in this country, and objective measures are often the least important part of the process. It’s not what you know, it’s who you know, goes the old adage. Patronage and informal networks and biased hiring teams often mean that the person who is hired isn’t someone who proved themselves to be the best at the job. Poorer people who lack informal connections are usually the ones who suffer from this lack of objectivity in hiring. Higher-class people, on the other hand, benefit from the subjectivity.

But studying the impact of these informal biases can be difficult. How in the world do you measure an employer’s bias towards, say, fancy vocal affects from a certain candidate?

A new study I happened across came up with a very interesting way of doing so.

Using the Indian caste system to study class discrimination

Photo by Karthik Chandran on Unsplash

India outlawed its caste system — a strict hierarchy where people were born into particular castes and considered lesser-than for the entirety of their lives — way back in the 1950s as part of the Indian Constitution.

Like the Civil Rights Act in the United States, using the force of law to reshape society did a lot of good. The power of caste to dictate the outcomes of your life isn’t the same in 2025 as it was in 1925.

But the law cannot dictate the heart of every human being. Caste discrimination continues to be part of life in India.

The silver lining in this unfortunate reality is that it gave Soumitra Shukla, a whip-smart economist and Research Affiliate at Yale University, an opportunity to study the impact of class discrimination in hiring.

“The caste system in India, it allows you to measure class in a way that’s not really possible in the West,” Shukla explained. “In the West if you say that somebody’s high class, does it really mean income? Does it mean accent? Does it mean dress…it’s possible that you have high income but you have a working-class accent.”

Caste, on the other hand, is less ambiguous. If someone is from a higher caste, they are also going to be from a higher class.

What Shukla did in his study was look at how caste played a role in elite job recruitment in India.

He worked with an Indian college to examine the career trajectories of graduates who went on to work at top firms.

“We know that hiring happens in a very structured way, and we have the opportunity through the lens of the placement office of this college to really get at which of these steps in the hiring process really make class or caste more salient,” he told me.

Much like in the U.S., top firms in India often visit colleges to recruit students. The screening process for hiring includes both objective components, like testing, and subjective components, like interviews.

What Shukla’s study found was that the personal interviews in particular were disadvantaging people from lower castes, even while objective measures seemed more correlated with actual performance on the job.

Why might that be? The interviews provided an opportunity for the interviewers to pick up on the applicant’s class/caste.

“Suppose somebody’s resume says play squash. And then you kind of go at the interview and say well do you play squash at your school’s gym, or do you play it at the country club? That kind of question you can only ask really in an interview,” Shukla explained.

What Shukla highlighted in the paper is this subjective but misleading sense of “fit” that employers often use to choose a potential employee for employment.

“Employers might have some stereotype about fit and performance on the job being correlated. But at least in my setting it seems like there are more objective criteria that predict performance,” he said. “Much like if you were looking at somebody’s college performance on tests, you would think that SAT score is a better measure than whether or not they can play squash.”

So what should we do about job interviews?

Photo by charlesdeluvio on Unsplash

Does this mean we shouldn’t use job interviews at all? Even if it were true that testing or work experience alone was 100% correlated with job performance and far less biased than subjective measures, it would be hard to imagine employers agreeing to forego them. There is a certain human element to hiring that employers tend to crave.

Shukla suggested that we could at least start by reducing the weight of interviews in hiring processes. If our biases privilege certain types of candidates in these interviews, the least we can do is make them a little less relevant.

Another thing we might consider is standardizing our interviews with job-relevant questions. Maybe chatting with an applicant about their professional hobbies isn’t the best use of interview time.

“You might design questions that seem job-relevant…at least at a basic minimum asking the same questions is probably helpful,” Shukla said.

Finally, he suggested that using panels for interviews might be one way to cut down on bias.

“Maybe [try] to have a panel type of an interview where you have more diverse voices, if possible. I think that can also help. Because sometimes a single person might just bring in their own biased thinking,” he said.

Some of these reforms have already been implemented in India’s administrative service.

Back here in the states, the evidence from India should at the very least make us think twice about throwing out the baby with the bathwater. Maybe the SAT and other admissions and hiring exams are not perfect. But these tests probably tell you more about the objective skills of an applicant than their squash club does.