Auto Ads by Adsense

Booking.com

Monday, April 05, 2010

Hiring Committee Stories

Google's engineering hiring is unique as far as I know. Patterned after faculty hiring by top universities, Google engineering had no hiring managers. None. This meant that all the usual job search advice by any number of books and web-sites didn't apply to Google. Bypassing HR and trying to get to a hiring manager didn't do any good. About the only thing that could have been useful would have been to get a strong employee referral (i.e., get an employee who thought the world of you to say so when he submitted your resume). The interview process went like this: you would interview with a panel of engineers, who would then write feedback to an internal database, which would then go to a hiring committee (also composed of engineers) to evaluate the feedback and provide a go/no-go decision. Since engineers might have to live with the code (and personality) of a bad hire, the hiring committee tended to be conservative on hires. Phone interviews and in person interviews were conducted almost exclusively by engineers, with directors brought in only if the candidate requested a meeting with a manager explicitly, or if the candidate himself was interviewing for a manager/director position.

For reasons explained in my book, I ended up on the Site Reliability hiring committee. At that time, the hiring committee was composed out of relatively senior engineers: Lucas, Ben, Bogdan, and various engineering big-wigs like Bill and Urs. Frequently, when the committee found feedback on hiring to be ambiguous, it would assign another interview to an engineer well-known to be decisive (i.e., someone who would be willing to stick his neck out and say "hire" or "no-hire"). This happened surprisingly frequently because many people dislike rejecting people, and occasionally, someone would write feedback that wasn't really informative enough.

We didn't always have the luxury of a second-interview, however, since some folks had to be flown in from far-away places. Google was truly a global company, and in its pursuit of talent would consider resumes from literally anywhere in the world. Now, I didn't think that Google's interviews were particularly hard, compared to startups and other well-known firms in the industry. In general, quality companies reject a large number of engineers because most people who call themselves programmers can't code.

A few incidents came to mind as being particularly funny:

One day, I came to the hiring committee and started reading feedback from interviewers. One of them turned out to be a candidate I had interviewed earlier in the week. I was doing as much as 5 interviews a week at this time, so I didn't always remember the candidate by the time I got to the committee. Lucas's feedback for the candidate started with, "I spent the first five minutes of my interview calming the candidate down after his interview with Piaw..." When the others got to this part of the feedback there was a lot of laughter. I think that was the moment I realized that Bogdan and I would get along, because he high-fived me across the table. The candidate was a no-hire, but I don't think it was because I was particularly harsh.

At one point, we came across a candidate who had to be flown over from the other side of the world. Since we knew there was to be no chance of re-evaluating this candidate if the feedback was insufficient, we asked the recruiters to make sure that we had decisive, experienced interviewers for this candidate, who seemed pretty senior. She replied, "How about Piaw, Ben, Bogdan, and Lucas?" When he heard this, Bill put his head in his hands and said, "Why don't we just save ourselves and the candidate some time and just send him a rejection letter now?" The room burst into laughter.

Our committee took hiring seriously. We agonized over many hiring (and no-hiring) decisions for many years, learned the idiosyncrasies of many interviewers, and tried to match them up to candidates as well as possible. It was very high intensity work, and on one or two occasions I had to go head-to-head and argue my case in front of VPs because I felt strongly about one candidate or another. I didn't always win, but at every point everyone's opinions were considered. I'm sure we made mistakes, but looking back, I'm not sure I would have found a better process. I for one think that the decisions the committee made were far better than the decisions each of us individually would have made.

39 comments:

Never said...

Hey Piaw,

A very interesting read - it is good to see the level of effort Google puts into making the right hire!

Having just gone through a Google interview myself (and been rejected at the essay stage), I was wondering whether you could tell me why Google has the policy of "no detailed feedback"?

Cheers

gwern said...

While we're asking methodology questions: do the hiring engineers get statistical feedback? That is, when one recommends no-hire and the decision is to hire, do they learn a few weeks/months/years later whether the hire worked out or not?

Piaw Na said...

It's better than that. Engineers get an e-mail when their candidate is accepted or rejected, along with everybody else's feedback from that interview panel. This lets you see what questions other people asked and whether or not their feedback was similar to yours.

In addition, the hiring committee gets to provide feedback on your feedback as well, in case you have something that's egregiously good or bad.

Piaw Na said...

As to the "no detailed feedback" policy, my guess is that there's a legal reason behind that. I'm not a lawyer so you should ask one.

xs said...

i would love to see the current google employees go through another round of interviews to see if they would get hired again.

in fact, you should do this, to check to see how good your hiring process actually is. i bet more than 50% wouldn't.

obviously, there are some real geniuses at google (just like other companies) that would always get hired. im sure that your best candidates will get hired at any company they apply to because they are simply the best. but a huge percentage of those that got hired once most likely wouldn't get hired a second time, depending on who they got as an interviewer. i know several people who got hired where my coworkers were left scratching our heads, because they were mediocre at best.

the process is wraught with randomness, depending on whether or not the particular candidate knows how to answer the coding questions. even if they eventually figure it out, they will get rejected.

im not complaining, it's obvious that the cream of the crop at google are amongst the best in the world, and you all deserve your successes. but don't think that your hiring process is as superior as some of your products. you are free to choose whomever you want to work at your company, but understand that a process that won't likely produce the same result twice in a row for the vast majority of your employees means that it's not particularly efficient. given how much energy you spend on recruiting and interviewing, there's likely a huge amount of cost savings you can do by making this more efficient and less random.

Piaw Na said...

Very good point. It's been discussed many times internally, but never executed upon.

The reality, though, is that we know that our hiring process is imperfect. We had plenty of statistics and data telling us that. The same statistics also tell us that going the traditional route (manager-determined hiring) does not lead to better results.

And there's a basic problem that while your interviewing process is imperfect, your performance metric for engineers is frequently even more imperfect. I'll touch on that eventually. Needless to say, I have plenty of material to write about.

Bob said...

Google's operational service delivery infrastructure consists of undifferentiated units of capacity. Other than variations in CPU clock speed, RAM, storage, and bandwidth (all of which are accommodated in the job scheduling algorithm so the job doesn't have to know), there's no distinction between a machine that serves the gmail frontend and a machine that stores log data. The machines are completely interchangeable, and there's no inherent reason to prefer one over another. That's part of the genius of Google's operations.

Google's hiring process presumes the same of candidates. The candidate is measured along various capacity axes, and slotted (very very late) for a role. This mechanistic approach works well for some candidates and poorly for others.

Piaw Na said...

There was a decision very early on to hire a certain type of engineer and emphasize that type of engineers in our hiring. You see this tendency stamped all over Google. The philosophy behind this is that our project life-cycles are so short that if we hired for a specific skill set, it would be very difficult to put that person in a different project 3 months later.

The result is that yes, if there's an impedance mismatch between you and our hiring process, you will have a poor experience. While Google does not want you to have a poor experience, I think you'll agree that the primary goal of any hiring system is to hire engineers that match what you want, rather than minimizing mis-fit discomfort.

I say this even though I'm very much a mis-fit at Google!

Louis said...

How quickly does the hiring committee make a decision? I had an interview about 2 weeks ago and I haven't heard back anything from the Google recruiter yet.

Piaw Na said...

The SRE committee met twice a week, and made decisions within a week. The other committees varied dramatically. Some were more overloaded than others.

SciBoy said...

with so many levels between myself and who was making the decisions, no wonder my interview felt like a waste of my time...

Guido said...

Hi Piaw,
I've been on the other side of the barrier 5 times... not 5 interviews, for 5 processes! (2 not passing the dummy hr guy reading the questions from a paper.. cause he/she was not able to formulate/argument properly the request)

I did 1 interview process till the end of it as IT Operations... than, the "hiring committee" decided that they did a wrong job description and they needed a project manager instead... which I'm at the time, but I was not asked for those info.

The 2 other interviews were as SRE... once I was dropped by an SRE interviewing me, and another time by an HR guy which missed the appointment and started arguing with me on the phone.

All the time, the kind of request I failed were the mnemonical ones. I do not care what is the default signal from kill, I use -9 or -1 and in case I look at "man".

The hr process in Google... sucks.

regards,
Guido Serra

http://guidoserra.it

M R said...

Hi Piaw,

when they give the candidate a score of 1 to 4, what on that scale is considered an "Average candidate"?

Thanks.

Piaw Na said...

The average candidate doesn't get hired. :-) In fact, someone who scored a straight 3.0 wouldn't get hired either.

M R said...
This comment has been removed by the author.
Piaw Na said...

They weigh a lot. So much so that 1.0s have gotten hired because of internal recs.

M R said...

wow okay. Thanks a lot. So it sounds like with strong internal recommendations, you just cant bomb. but 1.0s seem like a bomb.

This is truly a system that is hard to understand or internal recommendations really weigh that heavy.

Piaw Na said...

Who the internal recommendations come from matters. A very senior engineer weighing in counts more than a few sales persons weighing in on an engineering candidate.

M R said...

Yea, I was referring to engineering internal recs weighing in on a prospective engineer in which the positions of the people giving the rec correlated to the position that is being obtain.

I think that would only makes sense.

M R said...

What does it mean when the hiring committee request more information. Is that a negative or positive or neither?

When more information is requested, do those cases normally lead to a no hire, and what are some factors that can lead to more information being requested?

Piaw Na said...

It just means you weren't a slam dunk hire or reject. The typical case is that your interviewers were indecisive, which is not your fault.

M R said...

what do you mean by the interviewers were indecisive?

From your experience when they typically ask for more information, does that usually lead to a no-hire?

Piaw Na said...

Interviewers are asked to score each candidate on a scale from 1 to 4. Scores between 2.8 and 3.3 are considered indecisive (i.e., "This guy could be ok.") If you've got 4 3.3s for instance, then the hiring committee could deadlock. (It's better to have 1 4.0, 2 3.3s, and 1 2.0 than to have 4 3.3s, since at least you have one enthusiastic supporter, and studies at Google show that really good engineers polarize their interviewers)

In any case, asking for more information does typically lead to a no hire because you're already a borderline candidate, but people like me are no longer on hiring committees. :-)

Y said...

If a recruiter calls a couple days after an onsite and says "positive feedback! moving on the the next step!" does that mean it's moving to hiring committee at that point?
How does the hiring committee weigh feedback if the interviewers are from the team that's deciding whether to hire the candidate vs an interviewer from a random team?

Piaw Na said...

Yes. The next step after the onsite is the hiring committee. The committee doesn't care unless the team members care enough to say something about it one way or another. "Not good enough for this team but maybe this other team" is a red flag.

EZF said...

In which stage of the process the references are contacted?

Piaw Na said...

After the hiring decision has been made.

Ayush Chopra said...

Hi Piaw,

I am not sure if that is true. These days references are contacted before the file moves on to the hiring committee (or at least that is what is happening with me). My references were contacted for a feedback before my app hit the committee.

techiepm said...

Hi Piaw,

If one has failed in interviews before at Google, does that person get another chance of interview at Google? If previous interview feedback is bad, is that a reason to not even consider candidate after 2-3 yrs?

Piaw Na said...

You get one chance a year. The recruiters are measured on how many hires they bring in, and they know that the hiring is essentially random.

Anonymous said...

Great post Piaw, and definitely good information for potential recruits. I also have been asked for my references before the hiring committee decides. One question for you - I am actually being considered for a Program Manager role, and was wondering how important was GPA in this whole process. I have an MS in CS with 3.5 and an MBA from top 10 school but with a sub 3.2 GPA. Everything else in my app should be very good. Is this a very strong data point in the decision process?

pjl said...

Hi Piaw,
How often are candidates recommended to "hire" by the hiring committee, but then rejected during executive review? Thanks!

Piaw Na said...

I have no clue about PM roles. As far as candidates rejected during exec reviews, less than 1% of those in my experience.

Cat Shark said...

Hi Piaw, I am wondering what is the rejection rate at the hiring committee stage?

Piaw Na said...

It varies dramatically depending on composition. On the SRE committee it was around 70%

Anonymous said...

Hey Piaw!

How does feedback from other Googlers outside of the interviewers (referrals and those who went to school and have worked with you) play into the hiring process? How much weight is given to these centers of influence?

Thanks!

Piaw Na said...

They are weighted extremely heavily. I've had people bomb in the interview and then gotten hired because of a very strong referral.

Sayan said...

Hi Piaw!

Very informative blog here :D

I just wanted to ask, what are my chances to be hired by the Hiring Committee if my HR contact told me that my feedback was "very strong positive"? I'm interviewing for the Account Optimizer role in Hyderabad.

In tenterhooks!

Sayan

Ph1sher said...

This is really interesting. I recently gave onsite interviews with Google. Does the info of all candidates go to the hiring committee or only those who are likely to get hired based on the feedback?