User Experience Surveys: What Questions to Ask

Written by Moritz Dausinger, CEO of Refiner

Are you looking for new ways to boost user experience? Wondering what questions to ask in your next user experience survey to collect the most valuable feedback?

You know, I sometimes find it crazy how vital user experience is and how little attention we sometimes pay to it. Consider this – 90% of users admit to stopping using an app due to poor performance. 

Now, just imagine losing most of your users because your product doesn’t provide a good UX. 

I admit that it’s a highly unlikely thing to happen, of course. But for SaaS, losing even a small batch of users may mean a severe hit to our MRR.

Hence this guide. In this article, we’ll discuss the different types of feedback you can collect with user experience surveys and what kind of questions you should be asking. Finally, we’ll also cover some best practices for using surveys to evaluate the customer experience. 

So, let’s get right to it. 

What is a User Experience Survey?

The simplest way to explain UX surveys is as a strategy for collecting customer feedback to make sense of their behavior. 

In short, user experience surveys help you figure out how customers interact with and experience your product. With such insight, you can discover what works for them and which aspects of your product might need more attention. 

In terms of the user format, there are different types of surveys you could run to evaluate the user behavior, preferences, and attitudes towards the product:

  • Customer satisfaction surveys like NPS or CSAT that evaluate how satisfied and loyal your customers are.
  • Customer effort score (CES) measuring how much effort does it require for a customer to complete a specific task or action. 
User experience survey question example.

(An example of a user experience survey created with Refiner)

Why use surveys for user research?

I admit that I get asked this question a lot. Surveys aren’t the only way to evaluate customers’ experience with a website or a product, after all. You could gain similar insights by evaluating heatmaps, running usability testing, or watching session recordings, for example. 

So, why bother with creating surveys? Well, there are several reasons. 

  1. Surveys deliver consistent data from the user’s perspective. Your feedback does not rely on your interpretation of the user’s behavior. Instead, with a UX survey, you allow a customer to hint and explain their issues to you.
  2. You can then combine this information with other participants’ answers, collecting rich insights directly from your users. 
  3. You can focus surveys on specific areas of the product and collect detailed insight into how users feel about that particular experience. 
  4. Not to mention that surveys are reusable. You can run the same survey, again and again, building out the pool of insights from customers. 

What Questions to Ask in a UX Survey

I won’t deny it – The hardest part of creating a survey is figuring out what questions to ask. Sure, there are other elements of the survey that you need to consider – the survey type, delivery type, any conditional logic that you might want to include, user targeting, or frequency, for example. 

All of those elements do help with the survey’s success. They can increase the response rate, for example.

But the fact remains that the quality of insights you receive is closely tied to what you ask your users about. 

So, let me share with you some ideas for good questions to ask in a user experience survey. 

3 Types of UX Survey Questions

#1. Task-driven questions

These questions focus on the person’s experience either with the app in general or its particular aspect. You could also use them to probe deeper into the person’s product preferences. 

For example: 

  • “Tell us about your experience with Refiner.”
  • “What has been your experience with customer research platforms to date?”
  • “What has been your experience with our onboarding process?”

#2. Questions that ask about customer attitudes directly

These questions ask the person to tell you more about their attitudes, preferences, or experience with the product or its particular features. They ask for opinions, impressions, experiences, and more. 

For example:

  • “What is your favorite feature of Refiner?”
  • “What features do you use the most in our app?”
  • “How do you find our [NEW FEATURE]?”
  • “How well does the [NEW FEATURE] help you in your work?”

#3. Follow-up questions

These questions compliment rating-based surveys, like NPS or CSAT. Their purpose is to learn more about the person’s motivations for rating your product the way they did. 

For example: 

  • “What is the primary reason for your score?”
  • “Why would you NOT recommend us to friends or family?”
  • “What could we have done differently to provide a better experience?”
An example of a follow-up question.

(Example of an NPS follow-up question.)

A couple of points about those questions

Can you see certain similarities between the examples I shared with you above?

  • They’re all open-ended questions. Close-ended questions, ones that customers can answer with either yes or no, provide little information. They can suggest a problem or help you evaluate your user base’s general attitudes true. But it’s almost impossible to extract any deeper UX-related insight with them. Open-ended questions work much better when you’re evaluating the user behavior and experience. 
  • These aren’t leading questions either. Those questions don’t assume that a person has had a positive or negative experience. They take a neutral approach and allow the person to first decide and then express what they feel. 
  • These questions do not impose any answers on the person either. In other words, these questions do not prompt the user to give a pre-determined answer. Once again, they allow the person to express themselves fully. 

How to Get the Most of User Experience Surveys: UX Survey Best Practices

To close off this guide, let me offer you some pointers to make your next user experience survey a success. All the advice below comes from my personal experience – from planning and building Refiner, a dedicated survey software tool for SaaS, and from researching our users’ behavior to continuously improve our UX. 

So, without any further ado, here’s what I recommend you pay attention to when creating UX surveys:

#1. Avoid bias

It’s so easy to fall into the trap of designing a survey only to confirm your beliefs. You might be thinking that changing a particular feature or interface element would make a difference to UX, and so that’s what you ask about in the survey. 

A good example of doing so is asking users whether they’d feel they’d be more likely to click on a button if it was bigger (or in a different color). 

The problem is that, by approaching the UX survey this way, you’re not acquiring any valuable insights. You’re just confirming your own ideas, and that’s not what these surveys are for. 

Another common way to instill bias is by asking leading questions, ones that prompt a specific response from users. We talked about this just a moment ago, and you’ve seen that none of the examples I gave used that approach. 

#2. Even if you run a rating survey, add an open-ended follow-up question to it

Sometimes you may want to research more general attitudes towards your product or the user experience. And so, you launch NPS or CSAT surveys and ask users to rate a particular aspect of your product or service, company’s operations, sales, support, etc. 

But even then, add an open-ended question at the end of the survey. It won’t affect the rating scale, but it will allow you to understand and probe deeper into the person’s motivations for their score. 

An example of an open-ended question added to a UX survey.
An NPS survey featuring an open-ended follow-up question.

#3. Focus on one UX issue at a time

I know; it sounds obvious. Then again, I’ve seen companies using a single survey to evaluate more than one issue. The result? Well, on the one hand, it helps to keep the survey research simple. You run one survey only, after all. But on the other, it often leaves customers confused about what feedback to offer and leads to unusable insights. 

#4. Don’t leave any room for interpretation

This is one aspect of running UX research that I find to cause the most problems. You see, it’s easy to be biased when creating a survey, but it’s also similarly simple to ask too vague questions. 

For example, asking “what do you like about user research software” might cause the person to talk in the context of another tool, not yours. Now, if that’s the purpose of your research, to evaluate their general attitudes towards such products, the question will work fine. However, if you focus on your product’s user experience, then, unfortunately, the question might provide invaluable insight. 

#5. Be careful not to run too many surveys one after the other

I can guarantee it; once you’ve run one UX survey, you’re hooked. The data you receive is so phenomenal and helps drive so many aspects of the business forward that you’ll want to do it again and again. 

And that’s the problem.

You see, useful as such surveys are, they can be irritating to users if you launch them too often. 

Plus, with constant surveys, you might not get enough time to process their findings before new data arrives. 

So, manage your survey frequency wisely. Set up intervals between surveys. This way, you won’t be bombarding customers with more requests, and you’ll have the time to digest and plan how to act on the data. 

And that’s it…

Now you know what questions to ask in user experience surveys and how to make them a success. 

Good luck!

Related posts