Mobile devices are increasingly used to complete surveys that were traditionally completed on a personal computer. In a number of our recent general population studies the proportion of people completing a survey with a mobile device is 25% to 30% and with 69% of adults having a smartphone the proportion using their smartphone to complete a survey is likely to increase. Survey completion times for mobile phones can take twice as long compared to using a computer. For one study a 20-minute survey was taking an average of 55-minutes by mobile phone.
Mobile market research offers many new opportunities and changes to the way we undertake online research. Like any data collection method, we need to understand who is responsive to an approach to get the best results and avoiding costly errors. Even if you are not targeting mobile phone or other mobile device users for your research, people are increasingly completing surveys using their mobile devices; smartphones and tablets.
For most survey research, tablets are basically the same as a laptop. However, mobile phones provide a very different experience. Before discussing the issues and benefits of smartphones for your research, let’s first look at who is using mobile devices to complete more traditional online surveys; ones sent by email with a survey link.
From research we have conducted that covered a general population of people aged 18 years and over we found that younger people are more likely to use their smartphone to complete a survey. This pattern is consistent across age groups with a steady decline of mobile phone use to complete a survey as people get older. Interestingly, operating in the opposite direction is older consumers being more likely to use a tablet.
While tempting to assume mobile phone use for a survey is youth-driven, there is more to the story which we need to know to account for smartphone use in our research design. The chart below shows that the type of employment is also a significant factor. People in part-time employment and the unemployed are also more likely to use their smartphone. Even when we control for age we still find the type of employment influential in what device they use to complete a survey.
Why are People Using Mobile Phones to Complete Online Surveys?
The employment pattern helps to better understand why people choose different devices to do a survey. It is about access at the time of the invitation. People who have access to a computer when they receive a survey invitation and can use that computer without much interruption will use the computer. There is no incentive to switch devices. People working part-time or unemployed are more likely to not be sitting at a computer. The email invitation is more likely to be seen on their phone, these people are also less likely to have ready access to a computer and potentially having to wait until they get home to use a computer.
To better understand what drives mobile phone choice, I’ve also spoken to participants and a couple of other themes emerge. In terms of employment, people who work in retail or non-office roles are unlikely to work near a computer, making the phone the only real option. The age difference was driven in part by older people being more likely to be in full-time employment and in an office role with computer access and their declining eyesight making mobile phone less attractive. The sudden halving around 45 years nicely aligns with the decline in eye sight (Macular Degeneration) with people over the age of 40.
The main driver of mobile phone use is access at the time of invitation. The main factor limiting its use is screen size. The higher use among youth affect is likely to be due to a combination of smartphone penetration, less access to a computer at the time of a survey, and better eye sight than simply and unhelpfully a youth culture thing among the GenZ digital natives.
Design Issues with Mobile Market Research
Longer survey completion times for people using a mobile phone is both an issue for setting expectations for completing a survey and a consequence of other issues. Below are some survey design issues relating to mobile phones that you should consider when design a study and interpreting its results:
- Small Screen. Even with the larger mobile phone screens their smaller screen size of a phone places limits on wide range of survey design features. Participants are less able to view stimulus material (images), read long survey invitations and question instructions, and complete scale and grid-style questions (e.g. service and image batteries).
- Divided Attention. The nature of completing surveys on the go means surveys are more likely to be done between customers (retail and personal service), on public transport and other environments means survey participants are likely to be paying less attention to the overall questionnaire flow and longer questions.
- Less Survey Functionality. Combining the smaller screen size with software differences mean not all question types that are available for a computer can be rendered on a mobile phone. Such as choice and conjoint questions, and some gamification style questions.
- Access Issues. The wide range of platforms and coverage can mean some users cannot access a survey using a mobile phone. This is particularly important for regional, rural and remote participants.
Survey Design Implications
Reflecting the nature of mobile phone usage patterns, their physical and software design there are four key things to keep in mind when designing a survey which is either targeted at mobile phone access or where it is an option.
- Know Your Participant and Their Context. The time, day, holiday/non-holiday can all affect both your response rate and the device used to complete a survey. Similarly for occupation or demographic segment targeted studies you need to be mindful of what device they are likely to use. For large studies a small pilot study will show what device is being used.
- Shorter Surveys. Moving from telephone and face-to-face lead to an explosion in survey length; moving to mobile will mean shorter surveys. This may mean the overall questionnaire is shorter or a split survey design is used where not every question is asked of every participant.
- Test Complex Questions. Not all questions can be shown on all mobile phone platforms due software differences and the way they render a question on a smaller screen. Some questions types like best/worst, choice and conjoint, and heat-map questions may require a change in design to accommodate mobile phone use, if possible.
- Prevent Phone Completion. If removing mobile phones from the survey would not bias your results and if you are using methods that do not work well for mobile then you should block mobile phone access and encourage the person to access the survey when they are on a computer.
There is a temptation to make all surveys available to all people for all devices, while this social equity approach may be needed in some contexts, when you are paying for research and using that information to make decision, quality should be your driving concern.
Opportunities with Mobile Market Research
The above comments were about the use of mobile phones for surveys designed for completing on a personal computer, mobile phones, however, also offer many opportunities for studies designed with them as the targeted device, each one worthy of their own article. Some of the approaches use existing apps on a phone while others either use the browser to access software or require participants to download an app.
- SMS Text Survey. These are surveys sent using the text facility and require only a single response. This can be as simple as single letter or number (‘Y’ for ‘Yes’, ‘U’ for sales are ‘Up’) or a longer text response.
- Example of When to Use: Industry, customer, student and employee polling and simple feedback (e.g. Was the delivery on time?).
- Geo-activated Surveys. Using the geo-location features inside an application or survey tool to activate the survey when a person is in a targeted area.
- Example of When to Use: Research for events, media and PR events, tourism destinations, wayfinding and location research.
- Diary Completion. Using closed and open ended responses to sequentially document an experience rather than give all the information after the event as in traditional studies.
- Example of When to Use: Service experience, customer journey and path-to-purchase, new staff/ student orientation experience, export knowledge mapping, and new product evaluation done in situ.
- Visual Documentation. Use of the photo and video tools to show what a person can see or the context they are in.
- Example of When to Use: Ethnographic studies, shopping, environment/ store design, critical incident/ problem identification, product usage and media research.
- In-Context Studies. Similar to geo-activation except a person is asked to provide the information within a given context. Survey and tools may only be active during defined time periods.
- Example of When to Use: Cultural and sporting events, during shopping
The ever expansion of devices that people use to communicate and use provide an increasingly wide range of tools for research use that add to existing methods. Each with their own advantages and implications for research design. For mobile phones used in conjunction with personal computer based research care is needed to ensure biases in who, when and an in what context a study is undertaken are used to either enrich the study or are understand and controlled to ensure you get accurate and reliable results.
Finally, while tempting to design all studies for the smallest screen, this is not always the best option for getting the insights you need and can lead to false findings about a customer segment when the insight was about the device.