How to conduct user interviews

As product designers and product managers, we spend a lot of time thinking about developers’ needs. We look at the data and analytics on how they’re using the products we build.

But unless we get out of the building, we’ll never get the full picture of how and why developers use our products.

Our product and design teams speak with developers every single week. But with DigitalOcean’s rapid growth over the last few years, we realized we need to ensure that the whole company – not only product teams – had a consistent view of developers’ needs and their Jobs to be Done. An analytics console or SQL queries weren’t enough: we needed to get in front of developers, face to face.

So, in 2018, the Product Management, Product Design, and Customer Success teams kicked off a formal research effort called the Developer Landscape project to reach this understanding. The goal was twofold:

1. Deepen our understanding of the needs and “Jobs to be Done” of the developers and teams that use our products, and;
2. Analyze and synthesize what we learn, and communicate that understanding to the rest of the company.

By accomplishing these two things, we hoped we’d be in a great position in the coming years to give developers products and solutions that help them make their own products and solutions even better.

Today, we want to share one of the processes we developed as part of the Developer Landscape project. It’s the playbook for how we found users to interview for the project – and how we ran those interviews.

How to find users to interview

There’s a multibillion-dollar market research industry loaded with for-fee tools, services, and consultancies. But if you’re on a startup budget, don’t worry. There are lots of ways to recruit users for interviews – for little or no cost. Here are a few examples:

• People who are conference speakers and panelists are often very interested in connecting with others in the industry (and they often like to talk!). So why not reach out with a nice, personalized email and see if they’ll talk to you?
• Press releases from companies in your space (including competitors) can also be a place to source names of people who might be open to becoming research subjects. If your competitor quotes a VP of Engineering in a press release, and you hypothesize that a VP of Engineering is a potential buyer of your product, this person could be a great interview subject.
• Reaching out (appropriately) via social media is always a solid option.
• You can set up a dedicated page on your site for potential participants to opt in and sign up, just like our Research at DigitalOcean page.

If your audience is very broad – maybe your product is a social media app or a photo sharing service – and you have a modest research budget available, then tools like UserTesting.com or Lookback.io are easy ways to find participants. Ethn.io is another useful tool for targeting actual site visitors.

If you’re targeting a hard-to-reach user base – or you’re looking to rapidly ramp up the volume of user interviews you’re doing – recruitment services like respondent.io might be a good choice. We used this service because we needed to conduct a large number of interviews in a short period of time. We found that it worked well when targeting a specialized international audience (like technical-minded software engineers interested in IoT), but not if screening by specific job titles or roles. If you’re mining your customer database, your CRM or LinkedIn might be the best way to find these folks.

What worked best for us? Personal connections and connections of our colleagues ended up making up the bulk of our user interviews. The takeaway is that individual tools and methods are only one part of a larger strategy for recruiting users. Even if you have a research budget, you should still ping friends and colleagues for help to broaden your network of participants.

No matter how you approach recruiting, you’ll need to screen participants if you haven’t prequalified them. Here are some good tips for writing screeners so you can avoid spending time interviewing people who aren’t really your users.

Once you source users, you then need to get prospects to actually agree to your request.

There’s no one way to do this, but we kept these in mind when writing emails:

• Make them feel special! Point out that they’re in a select group. (“You were selected from a list of our top customers.”) Or personalize your outreach. (“Hey, I saw that we went to the same university.”)
• Use simple, accessible language and short sentences. Make it easy for them to say “Yes.” This can be as simple as asking the user to reply to your message, or including a link to a scheduling service like Calendly.
• They’re doing you a solid, so acknowledge that fact. You might also offer them a “thank you” after the interview in the form of cash or a gift. We offer DigitalOcean credits or gift cards – the longer the session, the higher the value.

If you’re not getting the participants you need, try increasing the types and/or amounts of your incentives to better entice users.

How to prepare for interviews

You’ll have limited time with each user, so careful preparation is essential.

Before each interview, we made sure the user had everything they needed to connect with us on the big day. We issued a lookback.io link via Google Calendar with a reminder configured. Lookback gave us the ability to record the interview, and let our teammates observe and comment in real time.

At the beginning of the project, we also gave our team access to a short discussion guide we prepared. The guide was stored as a Google Doc, allowing the team to make a copy of the file for each interview and easily add notes.

Here’s a snippet from our discussion guide:

Intro
* Thanks for joining the call
* Intros
* Purpose of call – to understand tools/solutions/processes you team is using so that we can better serve our customers
* Reminder recording – internal purposes only, not to be shared
* Reminder DO credit/incentive after filling out the Google form questionnaire

Background & Current Infra
* Company & Role: Tell me about your company and your role. What do you do? Who are your customers?
* Skill: How did you get to where you are today?
* How would you describe your current infrastructure? (Scope/State/Scale)

How to conduct interviews

Beginning the call informally in nature can help establish a level of comfort with the user. And if there are remote colleagues sitting in to listen, be sure they remain in a listen-only state to avoid overcrowding your subject. When an interviewee is comfortable, they are far more likely to answer questions about their current challenges with their role, team, and company – openly and honestly. This is what that process looked like for us:
* We started with warmup questions to help build a rapport and put the users at ease with the process. * We avoided leading questions, and instead focused on open-ended questions to solicit deeper insights than pointed questions would. * We didn’t follow the order of questions verbatim. If the participant started deep diving in a specific area, we went along with them. In some cases, we found that certain questions might not be applicable to an individual. In that case, we skipped the question and moved on.

At every stage, we were careful to ask questions to get the answers we needed.

Who are they?

Again, we started with easy, open-ended questions. We asked them to tell us a little bit about their company and their role. We asked how they ended up in this job at this company. And we asked them to tell us about their users and customers.

What’s their day to day like?

We wanted to understand what jobs users are typically trying to do, and how they get those jobs done.

• What’s your process for getting code from development to production?
• What tools or services do you use to do that?
• How do you feel about those tools, services, and processes?
• Why did you pick those tools and services in the first place?
• What was that process like?

What do they do when the going gets tough?

You can learn a lot about a user by honing in on a difficult situation they’ve faced. By asking them to drill into a specific challenge they’ve recently faced, you can follow up:

• Where do you look for solutions?
• Whom do you ask for help?
• Would you have preferred to select a different solution were it not for some other constraints?
• What were you nervous or anxious about as you were selecting or implementing a solution?
• How did you feel afterward? How do you currently feel about how things played out?
• Are there more incremental improvements you plan to make? Why?

What does the world look like when things are going great?

We learned a lot by asking users to tell us about a tool, solution, or process that worked really well for them. Knowing what excites and makes them happy is important when trying to figure out how to build a product they’ll love. To dive deeper into these happier scenarios, you can use some of the questions from the previous section as well.

Why? (Why? Why?)

Ask “Why?” as a followup question – frequently.

• “Why is that important to you?”
• “Can you tell me why you think that?”
• “Just so I understand, why do you feel this way?”

Asking “Why?” is a great way to demonstrate empathy. And it’s a great opportunity to dig deeper and gain insight into how your ideal user thinks about problems and solutions.

Close the interview by giving a user a floor for aspiration
Ask what their dream process or solution would look like if money and resources were no problem.

And, most importantly, ask if there’s anything else you should know.

This is a great final question because it lets the customer share exactly what is on their mind. Maybe there’s something they wished you asked them, or something else they want to share. Either way, you might be surprised by what you learn when you cap off the interview with this open-ended question.

After interviewing

Send the participant a thank-you followup as soon as possible after the interview, and give them instructions on accessing the incentive you promised. If there was anything you wanted to dive deeper on from the interview, this is a great time to ask.

We sent a short survey with simple followup questions to each interviewee, along with a note letting them know we’d send their incentive once they completed the survey (which we informed them about prior to the interview).

Next, share the raw notes from your interview with your team. Don’t worry if they are messy! You might even want to give a tl;dr version in your daily standup.

How to analyze, synthesize, and distribute interview results

If you’ve run a good user interview process, you’ve completed several interviews, and have notes and recordings from each. Now you need to identify common themes throughout your interviews and synthesize the data into actionable conclusions for your team.

The synthesis method you choose varies by the type of user research you’re running. For instance, if you’re running a usability test, a list of observed usability issues and pain points prioritized by severity and frequency will be more helpful and actionable to you than a word cloud.

To get to the next level, present the insights (along with the customer quotes that contributed to them) in your team or company all-hands meeting. And include those insights in the onboarding materials for new hires. In many cases, it’s not hard to convert the customer’s journey into a single slide that’s easy to understand for both new and tenured team members.

Go forth and research!

We hope this helps you start your own user research project – or provides a few tools and suggestions you can apply to existing projects.

If you have questions or would like to share how you’re incorporating user interviews into your product development process, please leave a comment below. We’d love to hear about your user research needs and will try to answer any followup questions you might have.