Why you shouldn't hire like Google

2/23/2014

In today's New York Times Thomas Friedman reports on purported hiring practices at Google, as represented by Laszlo Bock, a senior vice president there.

Bock is an amateur psychometrician. He maintains that "GPA's are worthless as a criteria for hiring, and test scores are worthless." Rather, they are looking for "general cognitive ability, and it's not IQ. It's learning ability."

Bock is similarly unimpressed by "expertise." According to Bock, someone with high cognitive ability will come up with the same answer as the person with expertise anyway.

They also value "emergent leadership," which means what it sounds like, and "humility and ownership," which sounds like being a responsible employee; shouldering blame when blame is yours, and trying to learn from your failures.

Everything Bock says is probably not true, and if it were true, it would not work well in organizations other than Google.

Decades of research shows that job performance in many careers is pretty well predicted by standard IQ tests.
"Learning to learn" is nebulous because it's domain-specific, and it's domain-specific because the ability to learn new things depends on what you already know.
"Emergent leadership" and "humility and ownership" are qualities many organizations prize and would dearly love to reliably predict at hiring time. Maybe Bock has something to teach them about this. I kinda doubt it, but you never know.
The idea that smart people can pretty well figure anything out without expertise? Even though IQ predicts job performance (not "learning to learn") experience still matters to performance.

Friedman adds the critical caveat in the last paragraph:
Google attracts so much talent it can afford to look beyond traditional metrics, like G.P.A.

Yes. It reminds me of a conversation I had with a Harvard admissions office who told me "Look, we could fill the freshman class with students who got 800,800 on the SAT. Literally. Every single freshman, 800,800. We're just not interested in doing that."

That doesn't mean that the SAT was irrelevant; you didn't meet many Harvard students with crummy SAT scores. It means that once you're in the 750 range, Harvard figured you're damn smart and whatever "edge" might be represented in the difference between 750 and 800 didn't matter much. They started to look at other qualities. Harvard admissions officers (at least as represented by my friend) were also quite serious in how they tried to do it, and quite humble about their ability to assess them.

Likewise, Google is, I'm willing to guess, selecting from tremendously capable people--capable as defined in standard ways--so it makes perfect sense that further selection is based on other qualities. It doesn't mean that standard metrics are rendered irrelevant.

Friedman is right when (in the last paragraph) he offers this advice: For most young people, though, going to college and doing well is still the best way to master the tools needed for many careers. I would add that I would expect Google's offbeat hiring practices wouldn't work in most places.

Ryan

2/23/2014 02:51:07 am

"Structured behavioral interviews" to assess general cognitive ability, leadership, and humility would have to be very well-structured to overcome issues you mentioned in the previous post on interviewing. I don't get it--Google just asks the same code questions that everyone else does.

Engineer, turned down google

2/24/2014 12:52:26 pm

Google is actually much worse at hiring good people in my experience than other companies. They are generally incompetent and hiring not very competent people.

Somehow, though, google has become a religion, much like microsoft was in the 1990s and a lot of people think they are great no matter how much privacy they betray.

Rational people don't worship google, and they see them for what they are...mostly a joke when it comes to technology.

Joanne Jacobs link

2/23/2014 05:38:02 am

My stepson was hired at Google. His high intelligence was backed by experience -- he started working for high-tech companies in his mid-teens -- and a master's in computer engineering. I don't think they'd consider a very smart person who'd developed no relevant expertise. And I'd be surprised if they hire for humility. Responsibility, yes.

William Hiss at Bates says students who don't submit SAT/ACT scores for admissions do as well in college as those who do. Do you think this is solid? Does it show that measures of ability are less predictive of success than grades, which measure getting the work done, pleasing the teacher, etc. as well as knowing the material?

Catherine link

2/23/2014 05:38:50 am

btw, I don't know whether you've posted about Gergenizer and/or Kahneman, but Bock's observation about people without expertise making better calls than people with expertise reminded me of Gegenizer's work on amateur versus expert predictions, and I wouldn't be surprised if that's what Bock is thinking of.

Ted Maclin

2/23/2014 01:35:13 pm

See also, http://onlinelibrary.wiley.com/doi/10.1111/iops.12001/full

"Many researchers believe that education is the most important component of human capital (McArdle, Waters, Briscoe, & Hall, 2007), but the evidence suggests that the relationship between educational achievement and career success is only modest. For example, in a meta-analysis of 50 studies and 62 samples, Bretz (1989) found that the validity of college grades for predicting earnings and supervisory ratings was erratic with correlations varying between −.25 and .43, and a nonsignificant average effect size of d = .39. Grade point average (GPA) barely predicted starting salary and did not predict salary growth thereafter. In a later study, Judge, Cable, Boudreau, and Bretz (1995), using a sample of 1,388 U.S. executives, found that educational level, quality, prestige, and degree type significantly, but moderately, predicted financial success. Similarly, Pfeffer and Fong (2002) reviewed the literature on the utility of business school education and concluded that neither an MBA degree nor GPA were consistently related to pay and promotions, although the prestige of the institution was somewhat related to these measures of career success. More recently, Ng, Eby, Sorensen, and Feldman (2005) meta-analyzed this research and found that indicators of educational achievement correlated modestly but positively with subsequent financial success (r = .21). Thus, although education is reliably associated with career success, the effects are relatively small; the amount of unexplained variance suggests that other factors may also be important. It could be argued that the modest effects of college grades on subsequent career success are a function of the restriction of range observed at higher levels of career success, that is, the fact that academic/educational qualifications are higher and more homogeneous in more competitive, desirable, or highly-skilled jobs. Thus, once you are “smart enough”—in terms of your academic qualifications—other factors are more important in determining your success levels. On a practical note, GPA is easily obtained and captures years of reliable performance differences between people, which makes it a useful predictor variable. However, as Chamorro-Premuzic and Furnham (2010) point out, “recruiters often ignore GPA when recommending selection and in some circumstances even recommend selecting candidates with lower GPA” (p. 82)."

Dan Willingham

2/24/2014 12:05:34 am

yes, and I should have made plainer the GPA isn't be worth much. . . too many factors contribute, and there's too much variation across schools and even across majors. And we may be moving to a situation where there's not enough variance in grades (because of grade inflation) to pick up any meaningful variation anyway.

Catherine link

2/24/2014 04:34:13 am

Thank heavens we have you.

Bernie Daina, PhD

2/24/2014 05:05:35 am

It's more complicated than it first appears. The cognitive ability to manage complexity is related to where people should be placed in a hierarchy, something that some organizational psychologists such as myself help companies assess, and it is not measured by IQ tests. The ability to learn should be related to this variable and can be assessed in a "process" interview, not via coded questions. The humility variable is probably related to characterological narcissism, which in small doses does some good in a fast-growth company and in larger doses is a horror.

Karl M. Bunday link

2/24/2014 12:16:20 pm

Professor Willingham, you write, "It reminds me of a conversation I had with a Harvard admissions office who told me "Look, we could fill the freshman class with students who got 800,800 on the SAT. Literally. Every single freshman, 800,800."

The Harvard admission officer is the victim (perpetrator?) of a delusional belief. Young persons with that score pattern are just rare enough that it's highly unlikely that Harvard could fill its class only with such young people. (Such young people have become much more common in recent years, but they have never amounted to a surplus for any Harvard admitted class, as Harvard admits about 2,000 students per year to get its yield of 1600 students, and not all students with such scores apply to Harvard.) The annual data on score percentiles

http://media.collegeboard.com/digitalServices/pdf/research/SAT-Percentile-Ranks-Composite-CR-M-2013.pdf

from the College Board make this clear. Only Caltech has a small enough entering class to admit only students with "perfect" SAT scores both in critical reading and in mathematics, and indeed Caltech comes pretty close to doing that, but also admits some students each year who did not perfect both sections of the SAT.

I have heard the same statement, even more blatantly false, from college admissions at colleges much farther down the echelons of desirability, and with much larger entering classes. This mathematically CANNOT be true of more than just one or two of the most selective, most desired colleges. All colleges each year admit students who dropped points on the SAT. It happens every year. On the other hand, in this era of grade inflation, it is literally true that most of the top colleges could fill their classes entirely with students who had straight-A high school grade averages, which is a much less lofty standard.

Now, the interesting psychological question here is why an admission officer would make a statement like that when the statement is not only false, but easily shown to be false. Getting through the public statements of college admission officers to examine whether what they say is what they mean is so far a problem that economists and sociologists have worked on harder than psychologists have, but it is a worthy problem for psychologists to tackle too.

Something similar is going on, methinks, with the statement from the Google corporate officer. Google may do something in reality that is different from what Google officers describe in interviews with the press. I did find it interesting that the Google officer says that Google hires persons who haven't finished college and yet finds that some of those workers perform quite well at Google. That didn't used to be Google's reputation.

Bernie Daina, PhD

2/25/2014 02:21:39 am

I agree with Joanne Jacobs - I'd be very surprised if they hire for "humility". I'd be astonished if they know how to assess this characteristic reliably!

Mike G

2/26/2014 01:29:10 am

Good post Dan. Agree.

My google friends tell me the column incorrectly captures the real hiring process.

Which for senior software guys, is giving someone a coding problem and sitting with them as they work on it. Rinse and repeat later with other staff.

They're looking precisely for expertise, in the sense of "do you know enough about coding to look at a novel problem and recognize enough aspects of it to do something productive." Much like a chess pro uses expertise. Or a cognitive scientist, for that matter.

Moreover, at least my friends say, most of the coders walking around Google have top 2% SAT. Maybe Google doesn't check, but to say that test scores don't matter is more of a hiring process statement that anything else.

If test scores truly didn't matter, we'd see a wide distribution of math SATs at Google. Does anyone really believe that's the case?

Comments are closed.

Why you shouldn't hire like Google

Purpose

Archives

Categories