The other day, I joined a semi-viral tweet chain with I’ve been coding since 1979 and I still have to look up java.lang.String methods all the time. There were a bunch of programmers doing this and I thought it constituted amusing humility while also making a useful point: Remembering the details of any particular API or algorithm is irrelevant. Turns out I was part of a trend, see TheOutline: Programmers Are Confessing Their Coding Sins to Protest a Broken Job Interview Process. Except for, that’s bullshit; I still do whiteboards at interviews, and I don’t think the idea is broken. (Also, they’re not sins.)
Does this mean I ask people to code a bubble sort? Or to show any evidence that they remember the details of any particular API? Obviously not: As many have pointed out, that’s what Wikipedia, StackOverflow, and (most often) my IDE’s auto-complete are for.
These days, when I’m interviewing for senior-developer and development-manager positions at AWS, my questions are of the form “Design X”, where X is something like Twitter or SQS. The whiteboard is an appropriate level of detail; I want to look at the boxes and arrows they draw and see if they come up with a sane decomposition of the problem, and if they spot where the hard parts are. Sometimes, if there’s enough time, I might ask them to sketch in a bit of front-end code, but usually not.
Back when I was at Google, I was mostly interviewing people for “Developer Advocate” positions, and a lot of people somehow got into the process without being able to code at all. So, early on, I’d ask. “You’ve got a list of objects, write some code to select one of them at random. Any language, don’t worry about syntax, assume the built-in random function is good enough.”
That was actually a nice question: If you wanted to dive a little deeper, you could ask the candidate to sketch in unit tests. And if you’re talking to somebody super-technical, ask “Your code is in production and sometimes it’s throwing illegal-index exceptions under heavy load. What’s going on and how do you fix it?” Just because that’s a cool problem, very real-life, and most people smile when they get it.
Apparently DHH started the trend by admitting he couldn’t code a bubble sort on a whiteboard, and I think we can all agree that would be a totally dorky interview question. But TheOutline’s piece goes further, alleging that whiteboarding is anti-diversity. See Aline Lerner’s excellent (and data-rich) You can’t fix diversity in tech without fixing the technical interview. It mostly argues that “logic puzzle” interview questions are bullshit, and I heartily agree. By the way I got one of those in my interview day at Google, and another at Amazon, and I blew them both.
At AWS, these days, I’m on a few hiring loops, and I don’t see anyone asking logic puzzles. But I have no idea what the state of the art is these days at the big tech companies; are there any studies?
I think and hope that the way I use the whiteboard doesn’t make me part of the diversity problem, and I’ll be watching out for data on the subject.
Comment feed for ongoing:
From: eerie quark doll (Mar 04 2017, at 12:09)
I've experienced Amazon and Google in recent times; logic puzzles at neither, but definitely white board coding of methods and classes.
On one hand, sure they have to do something; on the other hand, writing code on a whiteboard during an hour-long-interview is ill-suited to people who either don't write quickly, or whose design cycle usually features a first pass naïve implementation followed a second pass optimization after window and navel gaze.
I feel like if the candidate is obviously not incompetent through dialog and design discussions, has a personality that the interviewers get along with, and has a long successful working history, then that should suffice for a hire.
[link]
From: Paul Hoffman (Mar 04 2017, at 12:56)
"You’ve got a list of objects, write some code to select one of them at random."
"...you could ask the candidate to sketch in unit tests".
I'm fascinated what kind of unit tests you would expect for selecting from a list at random. :-)
[link]
From: Bob Monsour (Mar 05 2017, at 17:25)
I interviewed for a product manager role at Google back in 2015. Among the interviewers (all one-on-one), the engineer asked me to sketch out a bubble sort. I hadn't coded in a while, but recalled it conceptually. He seemed fine with it. One of the product managers threw a queueing theory question at me, in a particular context, I got through it ultimately, but stumbled a bit.
I didn't get the job, but I don't think it had anything to do with these two aspects. The two phone screens with senior product managers went well (2nd better than the first; but then I think that's why there was a 2nd).
I will never be sure if my age had anything to do with it (I'm your age) as I believe I still have a lot of energy and don't quite look my age.
I think whiteboards are a great tool for dialog and I think that the approach you describe makes a lot of sense, for both technical and product people.
[link]
From: you dont know me (Mar 07 2017, at 12:16)
you're missing the point, and possibly its because you havent been in the interviewing trenches (as a candidate) in a very long time.
I interviewed at Google about 18 months ago. Out of the 5 white-board interviews they conducted only 1 of them matches what you described (conceptual breakdown of a software design problem), and even that one still got pretty heavily involved in algorithm minutia once we talked through the high level design. here are the other 4 questions they asked me
1. implement (in syntactically correct code) binary tree sort for trees where each node contains a 3-tuple. Sort based only on the 2nd element of the 3-tuples.
2. implement (in syntactically correct code) an algorithm to determine if input string A is a substring of input string B.
3. implement (in syntactically correct code) a peer-to-peer payment settlement system where a group of N people who each have different amounts of money settle purchase orders for K items made by the various members of the group.
4. implement (in syntactically correct code) a game where Player 1 guesses letters such that the minimum number of letters is guessed to select (from a dictionary of ever word in English) a word chosen secretly by Player 2.
and no, they weren't interested in talking through the design concepts of the problem. passing the interviews meant having working code within 30 minutes. bonus points if it's also optimal in terms of big O time complexity.
this is what whiteboarding interviews are really like. being asked to implement Bubblesort would be a mercifully easy softball of a question.
just because you're not committing the worst offenses in your own interviews doesn't mean they aren't still widespread in practice. the worst part of this is how many companies get Google envy and decide that they should copy the Google interview technique. I interviewed for a second tier e-commerce company (food delivery industry) that conducted interviews just like Google, but they weren't doing massive scale systems or working on novel problems. They had a Ruby on Rails powered e-commerce website. It's all a sick joke.
[link]
From: Adrian Petrescu (Mar 07 2017, at 12:32)
@PaulHoffman: I imagine it looks something like "sample the function a lot of times and measure the distribution of objects you get back", along with the normal edge cases of small-to-empty lists, duplicate objects, etc. Then you can dive a bit deeper on how many samples you'd need, and how much variance to accept, etc.
Seems like exactly the sort of thing where unit testing it is non-trivial enough to be interesting.
[link]
From: Anonymous Opinion (Mar 07 2017, at 13:09)
Dunno if it counts on the diversity side but whiteboarding does certainly select against those with social anxiety and/or introversion. (Rather ironic for an industry where that is the popular stereotype for its members.)
FWIW, I've been in the industry for 20 years and I still get "whiteboard anxiety" attacks and bomb interviews.
[link]
From: John Cowan (Mar 08 2017, at 09:31)
I've done coding phone screens (using CollabEdit) and interviews with a whole bunch of intern candidates this year, and in each and every case I make it clear that I'm *not* a compiler. I don't give a damn about syntax errors or forgotten method names. I don't even care what programming language they use (as long as I know something about it).
Instead, I look at whether they explain what they are going to do before (or as) they do it, whether the code organization is coherent, and whether they get bogged down in complexity. I try to ensure that each interview is a learning experience, definitely for the candidate and preferably for me.
[link]
From: Gavin B. (Mar 17 2017, at 09:43)
A somewhat satirical take
https://aphyr.com/posts/340-acing-the-technical-interview
[link]