Collection Pace
Introduction
Soliciting your users with too many popup surveys within a short time frame is something you might want to avoid. Besides providing a bad user experience, surveying users too often ultimately leads to low response rates and poor data quality.
The same is true for your team. Collecting too many survey responses at once might not be necessary or overwhelm your team. And sometimes, a couple of survey responses might be enough after all.
Refiner offers various Throttling options addressing those issues. The Throttling options described below give you full control over how many users will see a survey and how many surveys one single user can see in a give timeframe.
Sampling
Survey sampling is a method to reduce the number of responses you collect while keeping the survey results statistically correct. Instead of showing surveys to all users in your Target Audience, only a subset of users will see the survey. The number of users who will see a survey is determined by the percentage number you define in the sampling options. Users are chosen based on a unique identifier which we attribute to them randomly.
Throttling
Throttling per survey
By default, Refiner shows a survey as a Trigger Event occured for a user in your Target Audience. In some scenarios, it makes sense to slow down the collection speed though.
Our throttling options give you fine-grained control over how fast in-product surveys are delivered and how many responses should be collected in a given time period.
When enabled, you can define for a given time window:
- a how often a survey should be shown in total
- how many responses you want to collect in total
- how often one single user can see a survey
- how many responses one single user can provide
- how much time needs to pass until a user can see a survey again
You’ll find the option control collection speed in the Targeting tab of the survey editor.
These throttling options are prioritized over other survey trigger options, such as recurring surveys and our Follow-Up mode.
The following parameters parameters are available for throttling per survey.
- Time Frame: This parameter defines how far we will look into the past when determining if a survey can be shown or if we wait a little longer
- Max. survey views: Determines after how many survey views in the given time frame we should stop serving surveys. If you enabled Follow-Up Mode, each attempt counts towards the limit.
- Max. survey responses: Determines how many survey responses you want to collect in the given time frame.
- Max. survey views per user: Determines how often one user can see the survey. This option is only relevant if you are using Follow-Up Mode or Recurring Surveys. If you enabled Follow-Up Mode, each attempt counts towards the limit.
- Max. survey responses per user: Determines how many survey responses one user can provide in the given time frame. This option is only relevant if you are using Follow-Up Mode or Recurring Surveys.
- Gap time per user: How much time needs to pass until a user can see the same survey again.
Environment wide throttling
In addition to throttling options on a survey level, it’s also possible to define survey throttling in the global settings of your environment. You can find the global throttling options under SETTINGS > SURVEY SETTINGS.
Global throttling options are grouped into three levels as described below. Each group has the following parameters:
- Time Frame: This parameter defines how far we will look into the past when determining if a survey can be shown or if we wait a little longer
- Max. survey responses: Determines how many survey responses you want to collect in the given time frame.
- Max. survey views: Determines after how many survey views in the given time frame we should stop serving surveys. If you enabled Follow-Up Mode, each attempt counts towards the limit.
They three groups of throttling options are:
Throttle globally
Sometimes you might want to control the speed of data collection globally, taking into account all surveys.
A popular use-case for global throttling is to make sure that your team doesn’t get overwhelmed with too much data and can follow up timely on each response (e.g. feedback received from a NPS survey).
When enabled, this settings is prioritized above all other throttling options.
Throttle per user
When you are running multiple surveys in parallel with overlapping target audiences, you might want to control how many surveys or respond to in total.
If enabled, the number of survey views and collected responses is limited for each individual user. You can for example decide that a user should never see more than one survey per week, no matter how many survey campaigns are currently activated.
This group has an additional “Gap time” parameter which defines how much time needs to pass before a user can see another survey.
Throttle per survey
This setting allows you to define a default behaviour for all surveys in your account. The setting can be overwritten for individual survey as described above.
Disclaimer
Please note that survey throttling should be considered as approximate methods.
When choosing a time frame longer than one day, view and response limits are normalized to a 24 hour window. We do this to achieve a more equally distributed collection of survey data.
On high traffic websites and apps, it can happen that you’ll see slightly more survey views and responses than indicated in your settings. We recommend to start throttling with lower numbers and work your way up.
Response Limit
Sometimes you might want to collect only a certain number of survey responses and then stop the survey once this number is reached.
In the Targeting tab in the survey editor, you can enable the option “Limit Responses” and set the number of responses you are aiming for. When enabled, Refiner stops showing your survey once the number is reached.