8 Fantastic Questions for Reference Calls
Insights from “Scaling People” by Claire Hughes Johnson
Effective hiring is one of the hardest leadership responsibilities. Even the best executives and boards mishire frequently, leading to strategy mistakes costly time lost in the market, and expensive turnover. It’s easier when you can hire from a qualified pool of candidates you already know - but as companies scale, most leaders outgrow their known network and must hire more widely.
I found helpful advice on the best questions for reference calls in the book Scaling People, by Claire Hughes Johnson, Chief Operating Officer of Stripe.
Often, after a long search, detailed interviews, and the building pressure of work-to-be-done, reference calls feel like a checkbox to be passed. But it’s a critical step to catch issues and avoid disaster. Many reference calls illicit general feedback because people are uncomfortable (or legally concerned) about providing critical references. But Hughes Johnson asserts that the *right* questions can get more specific feedback that will help you evaluate fit and areas of concern. Here are the questions she recommends:
Where do you see this person in three years? This question gets the speaker thinking about ideal shorter-term outcomes and helps you understand the candidate’s trajectory.
When was the last time you didn’t see eye-to-eye? This question highlights a specific issue and helps you see how the candidate handles conflict.
What are some ways you’ve seen them be helpful to others? This can give you insights on the candidates’ collaboration and relationship-building skills.
Tell me about a time when you coached them on something. This question can surface a development area and lets you see how the candidate responds to feedback.
How would you rate the candidate on a scale of 1-10, you can’t answer 7. Most people want to answer 7, but framed this way, the speaker must select between slightly above average, or nearing exceptional.
What’s a skill you’ve seen them grow? This gives you a sense for a development area, but also how the candidate learns and how self-directed they are.
What advice do you have for me, as a manager, about how to make them more successful in this role? This again surfaces development areas.
Would you say this person is in the top 50% of the people you’ve worked with? Top 20%, 10%, 5%? This is another version of question five, but is effective at the end of a call once you’ve built rapport to get even more specific about the candidate’s effectiveness and potential. While people are generally hesitant to be negative in a general sense, when asked for data, people are more likely to be honest and accurate. Hughes-Johnson warns that being in the top 20% is not a ringing endorsement - Stripe operates with an even higher bar of excellence.
I wish she’d answered a few other questions on my mind — how many references should one conduct? Where is diminishing marginal return? How do you balance many positive references with a negative reference?
Hughes Johnson, in this part of the book, was addressing candidate-provided references. Like most leaders, I’ve found great value in “back door” references from people the candidate may not have hand-picked. However, you also must weigh the motivations, context, and trustworthiness of these additional references - in the best case scenario, you know these back-door references’ quality well. Strong networks of strong people are invaluable.
Questions that help you get to specifics are critical, but even more important is the courage to deny the hire if you encounter concerns. While I’ve made some exceptional hires, I’ve also made several bad hires. Some haunt me still. Almost all of the bad hires showed hints of their issues in the hiring process or salary negotiations. Sometimes, we felt like the issue “wouldn’t be that big a deal,” or “was better than everyone else we’d seen,” or we were too desperate for help to wait longer - maybe our quality bar was unreasonable… But making the wrong hire in the end was a very big deal. I’m much more careful and courageous after the painful experiences.
I nodded along as Hughes Johnson concluded: ”A lukewarm reference is a NO - do NOT hire that person.”
Carilu Dietrich is a former CMO, most notably the head of marketing that took Atlassian public. She currently advises CEOs and CMOs of high-growth tech companies. Carilu helps leaders operationalize the chaos of scale, see around corners, and improve marketing and company performance.
CHJ has very good insights for a similiarly privileged or fortunate individual