Understanding what is important is important. Whether you are trying to improve services, build a new product, develop a strategy, or develop persuasive communication, you need to understand what is important to your audience.
In this article, we will provide a guide to some of the best ways to measure what is important to your market in a survey. The approaches we discuss are used both to understand the relative importance of different features of your product or service (within analysis), and the importance of different services and products (between) analysis) to your customers.
Areas we cover include:
- What to consider when measuring importance
- Which factors impact importance
- How do you find what to measure
- What to do if everything seems important
- How do you measure importance
Why Do We Need to Know What is Important?
The main reason we need to understand and measure importance to consumers is that we have limited resources. If we had unlimited resources, we could improve all features or run campaigns that targeted all potential message areas. With limited resources, we must prioritise where we invest to have the biggest impact.
From the customer’s perspective, we need to understand what is important to them because this is what they use to evaluate their experiences and make decisions.
In this article, we look at what is important and what is its relative importance.
In another article we have written on Kano Analysis we also look at how improvements to features can have very different effects than expected.
What things can we include in measuring importance?
Importance studies can cover a broad range of topics, but most deal with understanding what product or service delivery features drive choice or satisfaction.
In economic or strategic studies, we can look at broader context factors like demographic change and composition, economic factors, competition, prices (interest rates, currency value, etc). This type of importance analysis is mostly undertaken using econometric techniques similar to the driver analysis indirect approach discussed later in this article.
What things should we include in measuring importance?
Before quantitatively measuring importance, you need to first undertake a discovery phase to surface all the factors that are likely to be important to your market. This can be done through analysis of past research, literature reviews, competitor/ similar provider analysis, analysis of complaints and enquiries, and internal workshops with subject matter experts.
In tracking and large-scale studies, undertaking a pilot or benchmarking phase to ensure you are measuring the right things correctly is critical. Otherwise, you will have limited flexibility in making changes if you have missed something that is very important to your market.
Who we ask is important in understanding importance
Before determining what is important to our market, we need to ensure we are measuring importance among the right people. To know who we should research, we need a clear idea of what behaviour we are trying to influence and how they link up to our organisation’s objectives. For example, if we are trying to reduce attrition, we should focus our efforts on those closest to that point in the customer journey.
We need to be clear about whose behaviour we are trying to change because what is important depends on the decision being made, the decision-making context, and the person’s proximity to the behaviour. Focusing on the people whose behaviour you want to influence is a basic tenant of marketing and behaviour change.
Importance is relative
A common misconception is that what is important to a person is immutable and unchanging. At the broadest level, this may appear true, but it is more likely we are misremembering. In the well-established field of attitude-behaviour, researchers have long known that when there is a discrepancy, we will realign our beliefs to reflect our behaviour.
At the personal experience and decision-making level, what is important and its impact on satisfaction and decisions are relative. The very idea of importance is relative. Something is more or less important than something else.
So, what impacts your customers’ relative importance on services and features?
- What is important depends on people’s goals
A person’s goal is central to what they want. In finding a solution, they weigh options based on how likely they will achieve that goal. What is important and its relative weighting is used to make that decision.
An example is the difference between a parent looking at childcare options for five days will see different things as important compared to a family looking for two days of care. Although they may agree that some things are important in both situations, such as the variety of development activities, the relative importance of that training will also change. Likewise, if a person’s goal in using bay wipes is to mainly clean a child’s face compared to the nappy change of a newborn, the importance they place on absorbency will change.
Take Out: When asking about what is important, frame the questioning around a goal or ask what their main goal is, so that we can later segment customers based on goals, understand how changes in goals impact what is important, and design strategies that leverage these insights.
- Importance is relative to other things being considered
When consumers think of what is important, it is always relative to things they think are more or less important. By using other things in framing their decisions they are more able to accurately understand how likely something is to help them resolve their need and have them feel more satisfied. Seeing importance relative to other things helps to make decision-making easier. This relativeness is rapidly and often without a person being aware of this process.
Take Out: Be aware that importance scores and rankings in your research can be affected by the set of alternatives. Making sure you have included key features, decision factors, or services in your study will reduce the impact of changes.
- Importance changes with decision context
Context can have a very big impact on what is important to a person! The proximity a person is to a decision, the strength of their need, their experience with the area they are researching, and the influences of others can all dramatically impact what they value. This is because context can change goals, even without a person’s awareness, and affect their awareness of how different factors affect their decisions and experiences.
As an example of how context can change what is important and how a major impact on strategy, in a study on the pain category, when asking people what is important, people not in pain often value naturalness, limited potential side-effects, and recycled packaging. While people in pain more strongly value speed, strength, and ease of using a medication, with limited value placed on risk. They want what will take the pain away and fast!
Take Out: Ensure you understand what contexts can impact your market’s decision-making and, depending on the nature of your project, either control for context variation or measure that variation to provide deeper insights.
- Importance changes with decision scale
From an organisation’s perspective, the unit of analysis scale impacts what is important. That is, analysis at the personal level at a point in time will give different results to market-level analysis over several years.
Unit of analysis refers to what it is you are studying. A unit can be a person, family, household, community, social group, market, region or other grouping. You may need to choose between a person or household-level sampling in surveys.
For example, when studying family planning across different projects, we found what drives a woman’s decision focuses on being in a relationship with a partner they expect to be with for life and feeling ready. At a population level, factors like housing, debt, time since marriage, partners’ income stability, job stability, age at marriage, and the average age of first-time mothers in their community were all important.
Take Out: Determine the level of analysis before you start your research to make sure you are measuring importance at the right level for your decision-making.
The most common problem when measuring importance: Everything is important
A common issue in studies that measure importance is that everything appears important, giving you limited insight into what is important and what to focus on.
Below are some common reasons you are seeing relatively little difference in importance scores.
- Averaging across different people
At the total level, you are averaging across different people. You may have different segments with very different results, but when you combine your results, you are just seeing the average. A solution is to segment based on what is important or look for segments within your market with different results.
Take Out: Segment your results to find the differences.
- There is no goal!
If you don’t provide people with a context for judging something’s important, they will use their own or general goals. If they use their own goal and you have not provided a way to know what those goals are, or they use a general ‘made-up’ goal, people answering your survey will give general answers. In most cases, people will try and infer from the other questions you are asking them to try and find out what goal you want them to use.
Take Out: Provide a goal or decision context for them to use in judging what is important.
- You are asking the wrong people
If you are asking people with little or no experience in what you are measuring, they will provide general answers. These are likely to either reflect generic ideas of what is important (like price) or be random.
Take Out: Make sure you are doing your study with the right target audience and have measures for later segmentation.
- You are measuring what is important
If you have done your previous research right, you are likely to have only included what is important. To move to the next stage, you need to ensure that you gave the right approach to discriminating relative importance.
Take Out: Make sure you use the right measurement method to give you deeper insights.
The two types of ways to measure importance
There are two broad approaches to measuring the importance of a survey: Direct methods and indirect methods.
The best approach for your project will depend on your goals and resources.
- Direct methods
Direct methods for measuring importance ask people directly to tell us what is important and how important it is. Asking people to select what is important, importance scales and ranking are examples of direct methods for measuring importance.
Pros: Simple to use and report. Can work with small sample sizes and in both qualitative and quantitative studies.
Cons: People are not always aware of what is important or how important it is. People will tell you what is important based on what they think you may want to hear or they want you to believe.
Direct methods are the most open to manipulation in community or policy studies, with people trying to ‘game’ the research to have policymakers focus on a particular issue.
- Indirect methods
Indirect methods for measuring importance either infer importance from modelling results or ask about importance using an indirect approach. Indirect approaches include statistical driver analysis, choice modelling, and maxdiff scaling.
Pros: Show how much impact a factor has on what you are interested in. Flexible for looking at different levels of factors. It can include context and different types of factors in a model.
Cons: Requires expertise in designing and interpreting models. It can provide unintuitive results that need deeper understanding to interpret. Require quantitative and sometimes larger sample sizes for reliable results.
Indirect methods are harder for people to manipulate when completing the survey.
Direct methods of measuring importance
There are a wide range of direct ways for measuring importance in a survey. Outlined below are the common direct ways of measuring importance in a survey.
- Select Areas of Importance
The simplest approach to measuring importance is showing people a list of options and asking them to select those that are important to them. This approach is used when you have a large list of options, early exploratory stages or projects, and limited space.
The select areas approach can also be used before the scale approach to reduce the number of areas and to only show those areas that have some importance. This approach will provide the least amount of insight and variation over time and across contexts and has the highest risk of providing a list where most things are selected as important.
- Card Sort
In the card sort approach, people are asked to sort all options into important or not important groups. The card sort approach is essentially the same as a select areas approach. The advantage is that a person is forced to make a decision about each item, ensuring that all items are considered. The downside is that it takes more time. The approach is also the same as the importance scale approach, but with only a limited binary scale: Important vs Not Important.
We use this approach in exploratory phases of studies and in qualitative interviews to allow us to ask follow-up questions about the decisions.
- Select Top Three (or more)
A variation of the select approach is to ask people to select their most important areas. This approach can help focus people on only the most important areas and potentially highlight the priority areas. However, while the approach appeals to managers as something they normally do in making decisions, consumers don’t think this way about topics, which can create confusion in selecting between equally important options or adding options just to complete the number required.
We use this approach in business-to-business studies when there are between seven and fifteen options and limited survey space. When there are many options, answering this question becomes confusing as a person needs to keep thinking whether or not to include or exclude an option and if they should deselect another option to include it. On mobile devices, this question is even more difficult for a person to answer as they need to scroll back and forth to answer the question.
- Rank Importance
An evolved and more effective approach to selecting the top-ranked options is to have people rank all the options of up to a set number. This approach has the same issues as the ‘select top’ approach but with the benefit of ensuring all options are considered. Because this approach takes a lot more effort to answer, you should only use it when you have around seven or fewer options.
- Importance Scale
The importance scale approach to measuring importance in a survey asks a person to indicate how important, on a unipolar scale, something is very important or not important. Importance scales work best when there are only a few items or in combination with performance evaluation. Such as asking how important some aspect of a service or product was and then asking how satisfied they were. To reduce the survey length, people are only asked to evaluate areas they consider to have some level of importance.
- Constant Sum
Constant sum questions are a more sophisticated version of selecting your top areas on an importance scale. This approach asks people to allocate a set number of points across different areas. For example, 100 points across different features should be allocated to show their relative importance in making a choice.
A variation of this approach is to have people allocate a budget of $100 across options based on where they believe an organisation should invest their efforts. This approach can work well in surveys with businesses and staff that are more used to this way of thinking and in contexts where there can be a trade-off. The approach also works best with between five and fifteen options.
Indirect Methods of Measuring Importance
A more sophisticated approach to measuring importance in a survey is using indirect methods. With indirect methods, the level of importance is inferred from changes in choice, satisfaction or behaviour from performance evaluations, usage or some other way of measuring the areas we want to include in our analysis.
The main downside of indirect methods is that they don’t have a simple measure to say what percent of people see options as important. Instead each of the approaches use statistical and econometric scores, such as utility scores, to show importance.
- MaxDiff Scaling
MaxDiff scaling is both a direct and indirect approach to understanding the relative importance of different features or offers. With this approach, people are shown a series of sets with features randomly assigned and asked to select the most and least important from that set. This task is repeated several times to ensure all options are shown across different sets.
MaxDiff scaling provides strong differentiation between options to show relative levels of importance clearly. Choosing only the most and least important feature in any set, is quick and easy for a person to complete. MaxDiff scaling is a more sophisticated card sorting approach.
- Driver Analysis
Driver Analysis refers to a broad range of statistical modelling techniques used to understand how changes in results for performance areas drive changes in choice or satisfaction. Driver analysis can tell you that a specific increase in performance in an area can how a known increase in overall satisfaction. When done correctly, this is a powerful tool for business decision-making.
An advantage of driver analysis is that different types of areas can be included in the same model to account for changes in context or market type.
Traditionally, driver analysis uses types of multivariate linear regression, but other methods like logistic and neural modelling (AI) are also used depending on the project’s needs.
Driver analysis is a type of econometrics technique.
At a more sophisticated level, driver analysis uses structural equation modelling (SEM) to show the interrelationship between items in determining change. Unlike other approaches, driver analysis can also be used to understand what drives different drivers. For example, is customer service is important to overall satisfaction, driver analysis can then be used to understand what drives customer service perceptions.
- Choice Modelling
Choice modelling is a powerful tool for determining the importance of different features on choice or satisfaction. Choice modelling is a statistical and experimental design technique in which different scenarios are created that include or exclude features, and different levels of features. A person is shown a random set of options and asked to either choose their preferred option of rate them. A person is then shown other sets, and the process is repeated several times to calculate relative importance and how different features interact.
Choice Modelling is part of the conjoint modelling family or approaches. Originally developed to understand transport choice, choice modelling has been used successfully across a wide range of scenarios that cover product and service design and policy design. Choice modelling can also be used to understand what features are likely to increase of decrease satisfaction.
A downside to choice modelling is increased survey design complexity and analysis time. Studies can also online include a limited number of factors, otherwise the choices become overwhelming for people completing the survey and require large sample sizes for robustness.
What is the best practice for measuring importance?
There is a wide variety of methods for determining what is important and the relative importance. Each technique has its advantages and limitations. This article has provided guidance on how we approach which method to use. To understand which option is best, the most important questions you need to ask are what do you need to do with the results and what are the study constraints. While the simpler approaches can tell you what people think is important, the more sophisticated approaches tell you their relative impact and how changing them can drive changes for your organisation.
We have used all of the above approaches at different times in our studies. Choosing the appropriate tool based on the study’s needs and constraints.
To learn more about how to uncover what is important to your market, contact us to discuss how we can help you uncover the insights that will transform your strategies for change.