Can people analytics help firms manage people better?

knowledge@wharton / Editors / 2015-08-12

How an organization makes its people-related decisions has a huge impact on its success or failure. But traditionally, these decisions have largely been based on intuition and biases and therefore have been prone to error. But now, companies are starting to use data and sophisticated analysis in issues such as recruiting, compensation and performance evaluation because they believe it can help in better decision making. Cade Massey and Adam Grant, who lead Wharton’s people analytics initiative, spoke with Knowledge@Wharton about why a data-driven approach to managing people at work is gaining traction.

Read this article in Chinese | Français

Knowledge@Wharton – When we spoke last yearabout people analytics, it was just before your conference and now you have another conference that begins today. It seems to me that during this past year the interest in people analytics has really gone up. Why is people analytics so hot?

Cade Massey – I agree with you that it does seem to be blowing up a little bit. It started in technology more than any other industry and then finance picked it up. And now, we see it everywhere. My sense is that people appreciate that this is a very important function [which] hasn’t been approached in a sophisticated way in the past. All of a sudden, they are realizing that all these tools that we’re accustomed to using in marketing or finance can also be used for hiring and compensating people. .

Adam Grant – I didn’t even know it existed until Cade and I were working with Google about five years ago. They built this whole people analytics team that was a mix of traditional HR folks, consultants, engineers and people like us who study organizational behavior. It was amazing that they were able to take questions that used to be answered based on intuition and actually run experiments and gather data to figure out what were the right choices to make. Google’s got a ton of press for all the great work they’ve done in this area and other leaders have started thinking: “Why aren’t we doing this? Shouldn’t we too be making all of our important decisions based on evidence?”

Cade Massey – We should give Laszlo Bock credit because he’s been like an evangelist for this. He’s the head of HR at Google and he’s always been very willing to say, “Hey, this is what we’re doing. This is what you should be doing.” It is rather uncharacteristic for someone to say that to his competitors.

One of the things I find interesting about people analytics is how a lot of HR decisions that used to be based on intuition are now becoming data driven. If you take hiring as the first point of engagement between a company and an employee, could you give me some examples of how that is changing because of people analytics?

Cade Massey – People are very interested in finding out if they can use objective measures to identify who is going to work well in their firm. Rather than having to bring in [potential employees] and talk to them in person, can we grab their GPA in college, [find out] where they went to school and who they worked for and from these inputs predict how they’re going to be? If you could, that would be great, right? Because you can save a lot of time, you can process all the applications really efficiently. So, that’s pretty promising. It has a lot of appeal. But, there’s more appeal to it than substance right now because it’s really hard to do. If you can pull it off, it would be great. Any investment you make in making that happen is going to have high returns. But it’s still new. There’s no silver bullet. People are drawn to it because it would be great if it happens.

Let’s say the hiring is over and now you’re on to the next step of on-boarding people. What can people analytics do to improve the on-boarding process?

Adam Grant – Google has looked at this explicitly. They ran experiments [by on-boarding people in different ways] to see what actually makes a difference. This is something they’ve been doing for the past couple of years. They recognize the on-boarding process is really important and completely under-explored. I don’t know the details of the study, but there’s one finding that jumps out at me. When they looked at all the things that make a difference in the first few days or first few weeks of your time at Google, [they found that] probably the most critical thing is that you meet your manager on day one. It may also be that as a new hire you’re sent to a lot of different places. But based on the evidence they said, “One of our rules for on-boarding is you’ve got to meet your manager on day one.” And that’s such a critical part of building a bond between an employee and an employer.

It’s not that we didn’t know it was important for you to meet your manager. But I think all of us, myself included, really underestimated how much of an effect that would have on day one. That’s an example of the kind of thing we’ve learned from their work.

Cade Massey – Something else they talk about that seems wise is that a person’s success at a company often depends heavily on who they work for. And yet, who they work for is essentially completely outside their control. This isn’t exactly on-boarding but it is early career, early stage consideration. If things aren’t working out, you need to be wise about it and not just blame that person. And you probably want to see that person in more than one circumstance before you draw out too strong a conclusion. These are both examples of being more systematic and scientific about evaluating your employees or trying to train your employees, as opposed to the old school thinking that we do this because this is what we’ve always done.

Increasingly we find that more and more of us are working in teams. And teams are very often geographically disbursed — across different countries and also across generations. You have baby boomers and millennials having to figure out how to work together. What can we learn through people analytics about the creation and construction of high performance teams?

Cade Massey – An infinite number of things. This is such a rich area. One example is a recent work by research psychologist Chris Chabris and his colleagues on team IQ. They do some really interesting work that looks at the productivity of teams as a function of their individual characteristics versus what they do collectively. They seem to find something like a team IQ which is different from the sum of the parts. It’s not that if you put all the smart people together they produce the smartest work. It’s that they need to be people who understand how to work with each other and work as part of the team. There’s something unique about team level intelligence that’s different from the sum of the individual level intelligence.

Adam Grant – And to build on that, there are probably ways that you can compose a team to enhance the likelihood that they’ll be intelligent as a group. One of the things I hear a lot is that diversity is good. And there’s no question that we get a lot of value from diversity in terms of people bringing unique perspectives, thoughts, skills to the table. But when you look at personality research there are some characteristics on which it’s actually helpful to have similarity as opposed to variety.

If you look at the data, extroversion / introversion is the clearest trait where variety is useful. Your team of whole extroverts essentially never starts working on the task. A team of all introverts often forget to bond. And the data say the most effective teams have a mix of the two. But don’t necessarily stretch that into every other personality trait. For example, if you look at a personality trait like agreeableness — people who love social harmony — one of the worst things you can do is put them on a team with people who are extremely critical and skeptical. Because the disagreeable people feel like they have to walk on eggshells constantly. Meanwhile, the agreeable people have this catch -22 situation: “I can be really agreeable and act disagreeable like the disagreeable people and then hate myself afterward. Or, I can be really agreeable and then it doesn’t quite jell.” [In such situations,] it’s actually helpful to have either similarity in personality or a consistent norm of how we’re going to interact. I think we have to be really more thoughtful about composition than we have probably been in the past.

Cade Massey – One of the things that people analytics brings to that task in general is just the inclination to study it precisely, and ideally, to run experiments around it. So, we’re not just going to take conventional wisdom and we’re not going to take something that’s written by someone who used to run some teams. We’re going to actually collect some data and run some experiments and ask these questions and figure them out.

Let’s talk further on the diversity aspect. For example, there’s been a lot of news, some of it controversial, about women in high technology companies. Has people analytics come up with any evidence that shows how gender roles and even racial roles are related to performance?

Adam Grant – Sheryl Sandberg [chief operating officer of Facebook and author of bestseller Lean In] is the brains behind that operation. I’ve learned a ton from working with Sheryl on gender issues. And she has a wonderful researcher, Marianne Cooper at Stanford, who’s collaborated with us on looking through what the data really shows.

What I would say right now is there’s a lot of academic research that hasn’t been leveraged. We know a lot about how to design, for example, performance evaluations that actually lead people to judge contributions as opposed to the person behind them. We know a lot about how to attract more female applicants in the high tech world; [It] turns out that the recruiter matters a lot. [Professor] Matthew Bidwell here at Wharton has shown this in the finance realm — that one of the reasons there are so few women in finance is that they don’t apply at very high rates. Women actually have slightly higher odds of getting hired because financial services organizations are really trying to solve this gender problem and bring in more women. But where they start to get discouraged is when the bunch of partners who show up to recruit, are all men. They say, “Well, I’m never going to get this job. Why bother trying?”

I think there’s a pretty big gap between what the social science shows on gender and what most organizations are actually doing. It’s great to see the Facebooks and Googles of the world trying to make headway on this. We’ve also seen a growing number of consulting firms make gender a big priority. I know this is a big topic that McKinsey is working on right now. Mercer has a whole initiative about how to create a gender balanced workforce that’s completely data driven and I think that’s only going to grow in the next few years.

What does people analytics have to say about performance evaluations and compensation issues?

Cade Massey – People analytics is mainly just bringing data where data hasn’t been used before. And in kind of a Moneyball spirit, it’s not taking conventional wisdom and saying, we’re going to be evidence-based here. There is an intersection though with some fields like psychology and the biases that people bring. One of the real motivating factors for us, because of the worlds we come out of, is we can actually improve decision making by using these tools.

Performance evaluation is a classic example because there have been so many biases that show up in this. But it’s giving people analytics maybe too much credit because it’s really just a vehicle to bring in some psychology we’ve known about for decades. Because we’re being more rigorous and focusing more on how we can improve this process, we’re able to ferret out some of those biases. For example, trying to keep opinions independent. So, I don’t want to judge Adam after having heard you judge him. We don’t bring irrelevant information about a person to the table because we know from decades of research, that if we know things about them it’s impossible for us to separate that from our thinking.

Adam Grant – Cade touched on something that brings us full circle. We talked about the important role Google has played in stimulating interest in this field, but I think Moneyball and the way sports analytics has taken off was probably the other catalyst.

Cade Massey – Yes, professional sports has been ahead of almost the entire non-sports world in using these tools because its whole existence is [based on] the performance of individuals. And they’ve got better data. They can see inputs precisely. They can see outputs precisely. You can look at what’s going on in sports around analytics now and know that 10 years from now those tools are going to trickle down. And even just the rigor and the scientific orientation of the people who use the numbers are better –literally — because the guys who crunched numbers for baseball 20 years ago were figuring things out.

Google has come up a couple of times in this conversation. Are there any other companies or organizations that you have been impressed with for the work they’re doing in this area? What can other companies learn from their experience?

Cade Massey – There have been some firms who’ve focused on this for a long time. Deloitte & Touche has had a workforce analytics practice for a long time. Goldman Sachs has undertaken a big initiative over the last couple of years. Credit Suisse has been very interested with some full time people. Johnson & Johnson is involved. Adam’s got a longer list, I’m sure.

Adam Grant – I’ll just add a few from other industries. Teach for America, particularly on their admissions domain. They’ve been tracking for years what they need to do to assess in the hiring process to figure out who’s going to be a star teacher and who’s going to stick with them. JetBlue has made a lot of headway in this area as well. They’re doing work in several different domains. One of my favorites is recognition. How do you build a science of recognition so that you know when it’s important to give people a sense of gratitude and appreciation? Do you make recognition public? Do you make it individual?

Cade Massey – Teach for America is such a good example because as we’re talking about Google and Goldman, you wouldn’t necessarily go to the not-for-profit world. But they are the best we know on hiring. If I could name one firm that knows the most about their hiring practices, it’d be those guys. Now, they’re perhaps ideally suited for it because they see 50,000 or 60,000 applications for the same kind of job and so it’s a perfect stream to get good at. But I can tell you, the Wharton MBA Admissions Group has learned and improved their processes because of the way Teach for America hires people.

What’s the secret ingredient of how you choose the best people to hire?

Cade Massey – To recognize that you’re never right and that you’re never going to be done. Those are the two ingredients. At Teach for America, they say they are never going to be done. It is not a one-time project. It is not a one year project. And the other thing they say, which is amazing, is: “These are our metrics, these are our objectives. We know they’re wrong.” This leads to a continuous conversation about how they can be refined. That kind of humility is a great counter balance to analytics because you can get pretty confident about your model. You need the humility that says we’re wrong, we know we’re wrong.

Adam Grant – I think it’s part of the founding of the evidence-based management field that people analytics probably belongs to. Jeff [Jeffrey] Pfeffer and Bob [Robert] Sutton [authors and professors at Stanford University] have long said that if you want to do analytics right, you need an attitude of wisdom, which in their definition is basically the willingness to act on the best information you have while constantly doubting what you know. And it’s easier, as Cade points out, to lose sight of the doubt part.

There’s one other thing from the hiring perspective that’s been very eye-opening to me. There is a bunch of data by Rick Jacobs [professor of psychology at Pennsylvania State University] and his colleagues suggesting that the cost of a bad hire is usually about triple the benefits of a good hire. I think a lot of selection is actually more about screening out than it is screening in. You’re always going to have false positives and false negatives. But it’s much more risky to bring in somebody that you have to later replace or do a lot of cultural damage repair around. And that’s where, I think, you probably want to put more of the emphasis.

If I were to switch gears and turn to the two of you as researchers, what are the big questions that you are trying to answer? And what has surprised you most about what you’ve learned so far?

Cade Massey – One of the things that I’m working on right now is exactly on this topic — how to get people to be more open to analytics. In some tasks we need to forecast what’s going to happen with the market or price, or the performance of an employee. You might have some algorithm and in most cases you need to blend that algorithm with some expert judgment. It’s not [about] person or computer — it’s best if you can blend these things. And yet, people are reluctant to take inputs from computers – especially if they are experts in the field. We’re trying to understand the psychology of that. What leads people to resist those inputs and what can we do to help break that down?

Adam Grant – One of the things that I’ve gotten increasingly interested in is the problem of collaboration creep, so to speak, where we’re constantly having to go to meetings and answer emails and there’s this massive explosion in inter-dependence and nobody knows how to handle it. Everybody thinks collaboration is great but everybody is overwhelmed with the amount of collaboration that they do.

I’m currently working on a project with Rob Cross [associate professor of management, University of Virginia McIntire School of Commerce]that looks at the following question: If you do a network analysis of an organization, who do you depend on for critical knowledge and advice and expertise? What Rob finds is that there’s a certain number of people that can write your name down. After that number you’re at serious risk for burn-out and overload. The unanswered question is exactly where does that number fall? It turns out to actually be quite deadly to have everyone depending upon you for expertise. I’m interested in how do you redistribute the help, the insight, the connections so that it’s not all bottle-necked in one or two people.

That’s really interesting. One final question. If you were to look at people analytics and the state of knowledge in the field today, where are the biggest knowledge gaps and what should be done to fill them?

Cade Massey – I would say, in being effective — not just running better numbers but actually making change in an organization. It’s one thing to have a fancier model or have some insight numerically. It’s a very different thing to actually translate that into action. And it doesn’t matter how good your model is until you get good at that translation. Right now, everyone’s enamored with the models and the data and the analysis. But it’s not going to matter unless they can actually persuade and change an organization.

Adam Grant – From my perspective, probably the biggest unanswered question for people analytics is what Cade’s working on right now, which is: Why don’t more organizations do this? And how can you get senior leaders to realize that just because sometimes these variables are hard to measure doesn’t mean you shouldn’t bring better science to them? And what does it take to open the minds of leaders to recognizing that if we had more data it won’t replace our jobs. It’ll actually give us the tools we need to make better judgments.

This article was first published by Knowledge@Wharton, under the title “Can people analytics help firms manage people better?” Copyright Knowledge@Wharton. All rights reserved. Translated and reprinted by permission.