Having research that is representative of your community starts with a design that is inclusive and accessible.  Representativeness is crucial for making informed decisions.  A representative study reduces sampling biases to provide an unbiased view of both the survey completers and the types of responses they give.

Having representative community research can mean different things:

• Representation of the range of views and experiences
• Representation of types of people in your community

Ensuring the representation of diverse types of people helps ensure that we get a true representation of their views.  With no one view being over-representative and drowning out the broader views in a community.

Improving representation starts with reducing sampling bias.  Sampling bias refers to how survey design, or any research, either encourages or discourages certain types of respondents or discourages others from participating.  For example, a telephone survey often systematically increases the proportion of older and female respondents.  Studies that rely on community panel engagement can also be biased towards the views of those most active in a community and highly engaged in council activities.

In community research focused on feedback and problem identification, representation can focus on user groups more likely to encounter issues or impact resourcing.

To reduce sampling bias, we design studies to remove barriers and improve accessibility by using multimethod designs, a breadth of approaches to sourcing and engaging.  For details on examples and design options for increasing representation in community surveys and research, this resource provides more information on how to ensure your community research and surveys are representative and reflective of your communities, providing reliable and actionable insights.

A survey with too few participants provides unreliable results, while larger surveys can be more expensive.

An important question when setting up any community survey is how many people you should survey.  For smaller local governments and some studies, the question is more often, “Do I have enough interviews for a reliable survey?”.

Understanding what the right sample size is important.  The number of people you include in your community survey can significantly impact costs.  The number of people you survey also affects study credibility.  If the survey has too few, then the results are likely to be dismissed, meaning the money saved in having a smaller study is wasted.

To determine the right sample size for your needs, you need to address the following:

• Risk and Confidence. The more important the issue and its consequences, the more certain you will need to be that your survey results are reflective of your community.  In statistics, this is generally referred to as ‘confidence’ and how wide (plus/ minus) the range you need.
• Population Variation. The wider the variation is in your population, the bigger your survey sample size needs to be to capture it accurately.
• Effect Size. For evaluation and monitoring studies, the bigger the change is, the smaller the sample size needs to be to detect that change.
• Measurement Sensitivity. The better your research design is at reducing ‘noise’ and is sensitive to measuring variation, the fewer people you need to include in your community survey.
• Population Size. The size of the population is not a big factor unless it is less than a thousand people.  With small populations, a ‘finite population correction factor’ is applied, which means fewer people need to be surveyed.
• Smallest Group. Are there specific groups within your study for which you need reliable results?   In random sampling research, it is often these smaller groups that determine the required sample size.  For example, if a group accounts for only 10% of residents and you need 100 participants in your survey, then you would need to interview 1,000 to find 100.

In practice, the right sample size will also be determined by resource availability, method limitations, and the community’s willingness to engage.

For a more detailed explanation of sample size, decision-making, and the consequences, read this short article.

As a simple guide, based on a 95% confidence interval (academic standard in social sciences), the maximum expected sampling error is shown below.

• Small councils with fewer than 10,000 residents or studies that are not risky a sample of n= 200 to 250 (±7% error range for total results).
• For medium-sized councils (11,000 to 49,000) or studies where a sample of n=300 to 350 (±5% error range for total results)
• For larger councils and those studies that need more room to look at larger community groups or demographics, the sample size is n=400 to 450 (±4.5% error range for total results).
• For studies that have many subgroups that each require accuracy, then sample sizes of n=1,000 or more may be required.

With good design and planning, larger sample sizes are possible and within budget.

Across our projects, we often employ multiple approaches, utilising either the survey method or community engagement to enhance community representation, facilitate understanding of needs, and reduce costs to the council.

• Self-complete surveys
  • Online survey
  • Mail or paper-based survey
  • Kiosk survey, where the survey is completed on a mobile device or kiosk at a designated location.
  • IVR surveys (interactive voice surveys) that have people select a number on their phone to give responses.
  • Chatbot (AI) surveys that use AI or rule-based systems for survey completion
• In-Person surveys
  • Intercept survey where a person approaches people at a location or locations, such as on the street or in a library.
  • Door-to-door survey where the interviewer goes from house-to-house or building to interview people.
  • Central location interviews are like intercept interviews, but with the interviewer in one place and people coming up to them to complete the survey.
  • Telephone surveys

In conjunction with the above method, a wide range of approaches is available for engaging and recruiting people to participate in the survey.  These approaches can be done in isolation or using multiple approaches.

• Mail out invitations to an online survey
• Partial telephone survey with online follow-up
• In-person recruitment with online or telephone survey.
• Location recruitment with QR code, geotagging or other approach
• Social media or website recruitment
• Email or SMS invitation
• In-stream recruitment

In-stream recruitment is a more sophisticated approach that identifies individuals in one survey and invites them to another survey relevant to their experience and needs.  For example, a library survey is then linked to a youth survey for people in the target age group.

By using the right engagement and recruitment approach, a community survey is more representative, increases the sample size, lowers costs, and is perceived as more relevant to your community.

This resource provides additional information on selecting the most suitable survey method.

• Direct method. You directly ask what is important or its relative importance.
• Indirect method. By observing how changes in perceptions or experiences impact overall evaluations or behaviour.

Direct approaches ask people in your community research directly what they feel is important.  Importance is defined as being important relative to some goal, such as satisfaction, improving one’s community, economy, or a particular aspect of a service.  Direct questions include asking how important people feel something is, ranking, and constant sum (where people allocate points across areas).  While it is simple, it can add significantly more time to a survey and does not accurately reflect the importance of context.

The direct importance often doesn’t show how changes make an impact and cannot be used for directing investment or prioritising improvements.

Indirect approaches infer what is important by statistically modelling or using a proxy measure to determine importance.  Statistical modelling looks at how changes in satisfaction, intention, behaviour or another area of interest change with what people think about what you provide.  This can be done within a community survey, such as community satisfaction surveys, it is often called ‘driver analysis’.  When done across surveys and time, it is called causal modelling.

Additionally, you can employ an experimental design in your community research to gain a deeper understanding of its importance.  This approach is referred to as Choice Experiments or Conjoint Analysis, and it involves manipulating what service features and levels are offered, including costs or fees, to understand their importance in driving preferences.  Within study Within survey experiments, are often faster, cheaper and less risky than actual changes to a service.

While understanding importance is often quantitative (surveys and data analytics), understanding it with qualitative research will add greater depth of insights on why something is important and how changes will affect your community.

You can learn more about approaches for determining relative importance with this resource.