On Reference Calls

On Reference Calls

I've been doing a bunch of reference calls lately for due diligence on hires and strategic partnerships. As part of this, I've looked around to see if there were any helpful tips or guides for doing reference checks. To my surprise, there isn't too much out there, especially from the operator/entrepreneur side of the house, so I thought I'd share my approach and thoughts.

It takes commitment.

By the time I’m making reference calls, I’m already a bit biased and predisposed to hiring the person. Like everyone, I delay my reference calls until the point where I’m about to extend an offer or engage in a partnership. This can create a paradox if I receive some unexpected and undesirable feedback, because by the time I actually call people, I already want to make the hire.

This dilemma is why so many reference calls end up as formalities with softball questions. It’s because we don’t want to hear the truth! I’ve been there, but this is the key point, and it’s why I think more needs to be discussed around reference calls. There’s a ton of lip service around “hire slow, fire fast” in startups. Part of hiring slowly is not cutting corners at the finish. I see it as my job to seek as much information and evidence that supports my learnings throughout the recruiting and interviewing process. I always assume that I’m going to get great information, but I also want to be sure I haven’t missed anything.

Every reference call is actually a networking call.

I've ended up partnering with companies or hiring people who I originally met through a reference check. Always be recruiting, right? In a list of references, you have a highly-qualified list. Treat it as gold and with great respect.

The simple mind shift of treating reference calls as networking calls makes all the difference. Instead of it just being a box I'm checking at the end of a long process, I'm talking with this person not just for their feedback on my hire but also as a potential future partner, teammate, Advisor, or Board Member. I never know what I might learn or where the call might go, so I act accordingly.

I can tell a lot about a person or company by the references they provide.

Not all lists are created equal. Everything is a signal.

Once I receive the list, I will sometimes ask my candidate, “How did you decide on these people as your references?" Or, "Who did you decide to leave off your list of references? And why?" The answers to these questions are often telling, and they help me to start filling in some names for possible back channel references.

Who is and isn't listed?

  • Did the person list their previous managers, at least from their past few jobs?
  • Did they include a good sampling of people they have managed and their peers?
  • Have they included any customers, strategic partners advisors, or mentors? Board members? VCs?
  • Who have they excluded?
  • How many people have they listed? Too many makes it a laundry list. Too few creates questions about their ability to create and maintain alliances.
  • Is their list a good mix of men, women, and Under Represented Minorities (URMs)? Diversity and inclusion are important at all stages!

What information is provided?

  • Have they shared a short summary on how they are connected with the person?
  • Did they provide both email and phone number?
  • Have they let you know the preferred way and time that the reference wants to be contacted?
  • Did they link to the person's website or LinkedIn profile?

What's the format of the list?

  • Is the list provided inline in an email?
  • Is it attached as a PDF, Word (.docx) file, Google Doc? (I could write another post about this alone.)
  • Is it clean and organized to make it easy to review?

Back channel references are golden.

In order to better reference check people, I focus on getting back channel or "off-list" references to complement the “on-list” references. The simplest way to do this, is that I ask the “on list” people for other people they think I should connect with with in order to get a fuller picture. From there, my web grows.

Back channel references are not as likely to be buddies with the candidate, not as likely to be coached, and they are not as likely to be loyal or accountable. The conversations are often a bit more raw and contain some hidden nuggets that I don’t get from the “on list” folks.

Another way to get back channel references is through LinkedIn. I search through people who used to work for and with the person. This then surfaces names of people either to be references or to suggest references. If their list was too homogenous, then I work to find a mix of genders and backgrounds to call.

For senior-level hires, I also make back channel references from people in the community. I ask VCs, customers, competitors, or even people who had a similar job in the same general community. They may not know the person directly, but they often have heard of them, competed with them for a job, or they have a general sense of their reputation. It turns out that CMOs often know other CMOs, and the same is true for sales, finance, engineering, etc. Two degrees of separation almost always holds up!

I have a pre-determined list of questions, but I don't always follow it.

My list of questions changes based on the person I'm calling, but I try to not follow a script. I use it as a guide to make sure I cover the topics I want to cover. I try to ask probing questions and then use the conversation to guide follow up areas. We've all been on reference call where the person is just reading from their list of questions. I avoid doing this!

Here are some topics I recently used for one of my reference calls for a C-level executive hire:

  • How do you know [this person]? (Because I’ve done my homework, I already know the connections, but it’s always great to hear it in their words, and it's a nice starting point.)
  • Can you share an example of a challenge [this person] faced? (e.g. A difficult conversation? A missed quarter?) Everybody has some areas for improvement and blind spots. It would be helpful to know these coming in to understand where to focus before they join our team. What areas should I focus on with [this person]?
  • Can you walk me through a time where [this person] impressed you?
  • Describe [this person's] leadership and management style.
  • In your experience, is [this person] good at... Fundraising? Being a spokesperson? Sales? Team builder and motivator? Building partnerships? Communicator? And can you share examples?
  • These usually overlap, but is [this person] more of a numbers person or a people person? Or something else?
  • If you were managing, coaching or mentoring [this person], what would be the top 3 things you would likely be discussing?
  • A common thing I’ve heard about this person is [this general thing], can you share your thoughts and experience regarding this?
  • Would you work with [this person] again? And why? (This is only helpful to ask for back channel references. Usually all of the "on list" references want to work together again.)

When I call provided references, I'm actually looking for stellar feedback. I expect to get glowing references, unless the candidate has told me in advance something like, “I’m listing [this person] as a reference. We had many great moments working together, but we also had some tough times. You’ll get a balanced perspective from them, but I want you to know that we had a conflict or friction over [this thing], and that will likely be discussed. I'll look forward to hearing your thoughts.” This kind of reference is the rare case. Usually, the references that are provided are hand-selected to say the best possible things.

During the call, I take notes.

This should be obvious, but you'd be surprised to know that most companies don't do this. After the call, I fill in my notes with more details, and I highlight key words or phrases that catch my attention. I use one document for all my notes for each reference call. This helps me to see patterns.

Each call builds upon previous calls.

Depending on the role, I may check 3-10 references. As the picture and details build, I use the information from each call in each subsequent call to validate points I'm hearing.

After a few reference calls, I have a picture of some of the areas that need to improve. Most of the time, these areas are minor and are likely to be common. What I’m looking to see is how these areas overlap with me and my team’s strengths and weaknesses. I’m working to validate the positives, but I am also trying to catch any last “gotchas” before extending an offer.

I deflect the hard questions by saying, “What some people have said about [this person] is that they are very success oriented, but they sometimes struggle in collaborative, team environments. Some people have shared they are a great individual contributor, but they have challenges building consensus and teams. What’s your experience here?”

For certain roles, I have multiple people do reference calls.

It's critical that the hiring manager make at least 2-3 calls, but for certain roles, HR or other teammates can make a few calls. One reason I’ve found this to be effective is that it helps to get balanced feedback when more eyes and ears are working together, and it is far easier to maintain a good working relationship with the candidate when somebody else asks the tough questions.

For very strategic hires, I've had my search agency, VCs, and even third-party services help me with reference calls. There are professionals who do this for a living, and they are very good at it. Not only are very sophisticated and very thorough in information gathering, but they provide very helpful reporting.

I circle back with my candidate with feedback.

If I'm about to hire this person or engage this company, it's helpful to be clear of the things I'm hearing. This is the foundation for effective 1:1s and partnerships going forward. And, by the way, I assume that all of my conversations will go back to the candidate, so this information will likely already be known. This is normal, so when I'm on reference calls, I don't ask or dwell on the negative or share any information I’ve learned that shouldn’t be shared.

Everybody has blind spots.

After writing down how they know each other and all of the validating details, I ask, “Everybody has some areas for improvement and blind spots. It would be helpful to know these coming in to understand where to focus before they join our team. What areas should I focus on with [this person]?” Then I listen, and, again, I write down the key phrases.

I may ask a couple different versions of the "blind spots" question to try to get to an answer. It may take a couple times, but there's always a blind spot to discuss. On certain occasions, I'll dip into talking about cognitive psychology and the Johari Window technique. To make it a safe environment and to encourage sharing, I'll sometimes will first share some areas where I'm not as strong.

How to interpret references.

Interpreting the information learned on calls is not always a slam dunk. If anything comes up that I feel needs some investigating, I first try to validate through my web of contacts. If I can validate the feedback through a couple other calls, then I reach out to the candidate and have a candid conversation.

It’s a rare event to have this happen at the reference stage, but reference checks can lead to not moving forward with a candidate or partnership. The types of validated feedback that are non-starters for me are attitude problems, unnecessarily creating friction, or lacking fair judgment.

In my experience, it’s better to know the good, bad, and ugly before before having someone on my team.

Hire right. Hire slow. Hire smart.

Header photo from Pavan Trikutam. Thanks!