It isn't going to take long for tech companies to see how quickly people can solve near perfectly leetcode style problems, using GPT4 (especially with bing chat that can access the internet). I've done at least 40 software engineering interviews in my past, as an interviewer. I have given bing chat my interview problems, asked it to expand, explain it's reasoning, and tell me time complexity, and justify rather it thinks it's algorithm is the most optimal. For about 80% of the problems I throw at it, it aces them perfectly. Gpt3 was comparatively much less capable. But GPT4 is for all I'd normally ask as a software engineering interviewer: it is perfect. Now there were some questions I'd ask it that it just got wrong. But 80% or so it got perfectly right. Am, I as an interviewer going to try and find the perfect 20% questions/problems that it's still not good at? What about when those interview questions get leaked? Then it learns them as well. Soon we will have it be able to solve every leetcode style problem you throw at it. ..... What this means is, if an interviewee is using GPT4, it will then become a game of "who is best at being a convincing liar?" , Then we're effectively testing people on their deception and acting skills than on their actual coding ability. .... The only way around this is if we return to all interviews being done in person. Maybe that's part of the reason for RTO? I don't know. But for any remote friendly company, they're going to have a hard time trying to figure out who's actually good at coding and not just good at acting. Maybe this means we will start to see some innovation around interview formats. Maybe we will see Google change course and explore new frontiers. ...... This is going to at the very least make on-sites more expensive. As your false positive rate from phone screens will be far higher. ........ To cap this discussion: those who doubt that GPT4 is this capable, try it for yourself before you speak. Nevertheless, I'd like to channel this discussion to the possibility that it is capable of solving most leetcode style problems, so while I too am skeptical.... let's explore the solution space for what we do in the event that it is (assume the hypothesis for the purpose of discussion). #gpt4 #chatgpt #openai #leetcode #bingchat
🆗
I've heard some are doing phone screens live in person now. Very expensive and someone could always cheat with anal beads (For those confused https://www.buzzfeednews.com/article/kelseyweekman/chess-cheating-anal-beads-conspiracy-hans-niemann )
Maybe for take home assignments. Don’t forget that candidates don’t know the questions as well as you. Not only do they have to be good liars, they have to be good actors too, to fake an initial puzzlement, brute force approach and finally come to an acceptable solution after a careful nudge from you, the interviewer; at which they will first sigh a sigh of relief before coding it away. But you never know!
Yes so we're judging them on their acting abilities. What is this an audition for a play?
They have to lie and fake the whole interview interaction, is what I mean.
Interesting times. Few months may be too optimistic but it will happen at some point in near future.
Phone screens will just be proctored, browsers locked etc. yeah you can still cheat but it would be way harder, just as hard with no proctor but no GPT.
Anyone even a tiny bit capable in tech can run their browser inside a virtual machine. Like it isn't even a challenge. That's not going to work.
It’s just going to be an arms race. More measures put in place to ensure you’re not cheating. Like a camera behind you for example.
If you’ve ever taken an online university exam there’s many proctoring solutions. Sure some candidates can cheat around proctors but do you really need to solve for the small % here? I’m as interested in seeing what a candidate can do with chatgpt
Unless you're meaning a physical proctor, where you go to a satellite office that is local to you, it is only trivial to run any software based proctor inside a virtual machine. Hell I could even fool the VM to think I'm using legitimate hardware. Unless the proctor software is able to get the Host machine's kernel level access (which if I'm using Linux or Mac, they don't even have the ability for software to do that), then there's no way to guarantee against cheating.
https://openai.com/research/gpt-4 OpenAI's own research says ChatGPT got 3 out 45 LC hards. If you've actually tried it, you'd know it can only solve common and old problems. It fails miserably on new problems on LC's weekly contest.
You realize that most interviewers reuse their same problems, cycling through 3 or 4 of them. If we literally have to constantly generate brand new problems every interview, then this is not going to be feasible. Besides I don't ask candidates leetcode hard questions. Medium at most.
You can create file to do fine-tuning. All that means memorising zero value leetcode algo should be gone.
You’re underestimating how easy it is to see what someone is doing and if they’re cheating by asking them to share their screen. Additionally you can ask them to download a proctor extension which tracks their clicks. I did that in grad school a lot on online exams Highly recommend trying this for once
You must be a new grad? If your asking a candidate to do all that, ever considered the interview system is bad? This ain't school. It's almost insulting to do this to a candidate
Not insulting if they want the job
I mean there was no point in testing leetcode since the time leetcode started(2016?). It was good before it became a game and the company "leetcode" started. Interviewers not even taking the time to slightly modify the question, made it useless. People asking dynamic programming it complex recursive questions made It double useless. It's no longer testing your engineer skills. It's at best testing your typing speed...
Why rto? Just do live coding?
Love coding, while you have chatGPT on another screen? Come on.