Your Take-Home Interview is Probably Biased
Over the last few years, it seems like the tech industry has started to move away from the traditional whiteboard interview in an attempt to make the interview process more closely mirror the actual job. It makes sense — never in my job have I needed to reverse a linked list while two strangers look on and forbid me from Googling the proper syntax. People work through different problems in different ways, and it is natural to want a method of screening that more accurately reflects that.
Does this mean that we’ve actually moved towards fairer and more equitable alternatives?
Enter the take-home. In theory, it seems better in every way. You’re given a task to complete, rather than an algorithm to implement. Maybe it is data analysis reflective of the role, maybe it is building a simple web app or API. You’re also typically given a timeframe. You may be told an explicit deadline by a recruiter, or you may be given a vague suggestion that “it shouldn’t take more than X hours, send it back to us in Y days”.
What is intended as flexibility, however, is the potential to introduce massive amounts of bias into your hiring process.
There are plenty of think pieces on why take-home exams are unethical unpaid labor. While that is its own can of worms, take-homes don’t just have questionable ethics when it comes to the candidates; They might also be bad for your hiring process.
When interviewing for a role, three different candidates may have three very different levels of availability. One may be currently unemployed, free to spend as much time on a take home as possible. Another may be employed, without non-work responsibilities. A third might be working multiple jobs, caring for children outside of working hours, or have other outside responsibilities.
The issue is that the take-home, aimed at making the technical interview more flexible and reflective of aptitude for a certain job, will evaluate these three candidates very differently. When giving overly flexible timeframes for a take-home’s completion, recruiters open up the door to favoring candidates with more free time available. The larger the time window to complete the task, the more the inequity expands.
The Same Take-Home, Three Ways
Take our three candidates as an example. Each of them are interviewing for the same role at a large tech company. They are given a take-home technical interview and told it will take roughly 4-6 hours to complete, and they have a week to do it.
Candidate A, our currently-unemployed job seeker, spends a few relaxed hours each day poking at the problem. They are able go to the company’s open source projects to see how something is currently done. They can explore external resources. They can complete the take-home with enough time to send it to a friend for review.
Candidate B might be currently working, but similarly enjoys ample time in the evenings. They work on it for 3 evenings before deciding it’s complete, but they give themselves a few nights after that to mull it over before submitting, ensuring they haven’t missed anything.
Candidate C has small children, and is the primary caregiver. They work full-time, and their evenings are spent assisting their children with homework and catching up on housework. They decide they’ll do the take-home on Saturday and submit it on Monday. Come Saturday, they have a 2 hour window while a friend is able to take care of the kids. It doesn’t matter that it isn’t finished. They submit it anyway.
Candidate A may have had to look up every element of their solution (which might be fine!) while the others were not afforded that option. Candidate B may have leveraged a coworker to look over their take-home to solicit feedback. Candidate C may have out-performed the others by a mile, if only they were given the chance.
None of these candidates are inherently more or less qualified for the role, but this method of evaluating them creates a tipped scale.
What is the Answer?
The right balance to strike can be a difficult one. I’m not an HR expert, but I can share what I’ve seen that strikes a proper balance between minimizing candidate stress, evaluating skills, and eliminating bias. Like most things in life, it comes down to a compromise.
Present the candidate with a one week window, and ask them to choose a day and time at which they want the assignment sent to them. Specify up front that, upon receiving the take-home, they will have X hours to email it back. Limit it to 2 hours. They are not building a feature for you — they are interviewing. Their time is precious and you should treat it as such.
Email the candidate the take-home in full at their specified date and time. Set this process up such that it is automated - some candidates may only be able to do the take-home at 1am on a Wednesday. That is fine.
Again, clarify the time window that they have to complete it and the email address at which they can return it.
This process, while not perfect, somewhat levels the playing field. No candidate is afforded extra time over another, while still giving them the benefit of choosing the time at which they complete it and not forcing them to code with an audience watching.
At the very least, being more aware of the bias our processes enable can help make both our candidates experience (and the people we end up hiring) a little bit better.