As businesses search for talent, tech leaders conducting technical interviews have to adjust to new tools, like OpenAI’s ChatGPT, to find the right skills that the business needs.
While technical interviews have drawbacks, the purpose is to get a read on whether a candidate is able to solve problems, if they would fit in reasonably well while doing so and demonstrate their thought process, according to Nathan Sutter, global VP of engineering at CoderPad, speaking during a live event Thursday.
Since ChatGPT hit the scene in late November 2022, there have been fears about plagiarism and cheating in academia. It struck a similar chord in the world of technical interviews.
Technical interview environments can vary from company to company. Some organizations conduct open-book technical interviews where interviewees can complete Google searches and use tools to get an answer, while others prefer a closed-book approach with no outside assistance.
There are upsides for tech leaders to understand how a candidate would act in an on-the-job scenario, but incorporating ChatGPT in interviews requires interviewers to formulate questions that can’t be answered with a simple copy and paste.
For businesses to get it right, leaders have to meet internally to identify how their organization sees ChatGPT use within a technical interview.
Once guidelines are established, businesses should tell candidates what tools can and should not be accessed during the interview, Sutter said. This can mitigate potential fear coming from the interviewee about what is allowed and keeps everyone on the same page.
“You need to be transparent in the beginning to let them know that this a tool that’s there for them to use, and we’re not going to knock points off for them using it,” Muhannad Asfour, VP of engineering at SaaS data analytics company PiinPoint, said during the live event.
What ChatGPT use can tell the business
Incorporating ChatGPT into technical interviews can show the business how candidates would use the tool for daily tasks if hired, replicating close to an on-the-job experience.
But having ChatGPT access doesn’t mean the candidate will be a shoo-in.
“I expect and I've seen anecdotally, people use [ChatGPT] as a crutch and it backfire completely because their confidence level is so high,” Sutter said.
Mathis Hammel, head of developer relations at CoderPad’s CodinGame, considers it a green light if a candidate can use tools efficiently.
“I think we need to put an emphasis on understanding what the tool does for you,” Hammel said during the live event. “If you’re just using the tool without thinking about it, it’s going to cause issues.”
At DevOps platform company CircleCI, Staff Software Engineer Chuck Daminato conducts open technical interviews, allowing candidates to search the internet and use tools to find answers to questions. Candidates are asked to let Daminato know what they’re searching for and where they’re looking.
“We don’t expect everyone to come with all the tools memorized at their command,” Daminato said during the live event.
As part of the process, Daminato closely looks at how candidates phrase input questions and interprets responses. If a candidate were to ask ChatGPT to refactor the sample code and the candidate just copied that, Daminato would poke a little bit more into the thought process. Using ChatGPT to refresh a candidate on the exact format of a closure, would be a more appropriate use case in his opinion.
For a software engineer position, interview questions usually have gray areas purposefully left out by the company to make the candidate think through the problem, Hammel said. This is not an area where AI shines.
“It’s usually a very easy problem, but it has so many layers of needing to dig into the problem and have meetings with people to really understand what is needed,” Hammel said. “This kind of complexity is really needed and that’s the heart of the job.”
For a candidate, one downside of using ChatGPT in a technical interview is that the tool is extremely verbose, possibly creating information overload, Sutter said.
“If a candidate is sort of panicking and not really knowing where to go and has an interview panel that is looking at them for answers, and you have this piece of AI that is telling you a bunch of things all at once, it could be a little bit overwhelming,” Sutter said.
