Collecting Evidence

Introduction

Before diagnosing the problem and selecting the SBC approaches to address it, you need to collect Social and Behavioural Evidence.

This process will help to ensure that all challenges and decisions are rooted in evidence and local insights.

Why collect Social and Behavioural Evidence? 

Human behaviours are complex. 

There are thousands of variables that influence people’s choices, from context to culture and everything in between. The determinants of social phenomena are often even more complex. Collecting Social and Behavioural Evidence is a way to make sense of this complexity, by providing information and data on why people do what they do, or why a society is structured by certain rules and interactions.

Evidence needs to be collected systematically. The more information you have on participant groups, the better positioned you are to design programmes that effectively address their needs, interests and strengths. 

Social and Behavioural Evidence is important because:  

1. It helps us empathise with the affected population – their day-to-day lives, what motivates and frustrates them, who they trust and what they believe and desire.
2. It illuminates the barriers that are impeding change, what positive levers can be activated, and what should be addressed first. This can be anything from power dynamics and social norms to structural factors like accessibility and quality of services.  
3. It allows us to situate the problem within larger systems, as local situations are likely connected to national political decisions, economic systems or international issues.  
4. It makes us aware of existing actions and previous initiatives that focus on the same problem, so that we can learn from them and create synergies.  
5. It helps us confirm which stakeholders should be in the lead particularly those within the  government and civil society.   
6. It helps determine whether our strategies will work in the future, by using continuous monitoring to determine if and how barriers are shifting.  
7. It helps us mobilise resources and prioritise where to invest them through an analysis of sustained progress that indicates why a programme should be funded or scaled up. Throughout this exercise, we highly recommended that you select a model (see Why do people do what they do?) to structure the research and avoid blind spots and biases.

Identifying the evidence you already have

Collecting Social and Behavioural Evidence can be a time- and resource-intensive task. Before you begin collecting new data on barriers and levers, consider the evidence you already have. Potential sources include:

• Grey literature: Research that has not undergone a third-party or peer-review process. This includes government reports, white papers, case studies, conference proceedings, etc. 
• Academic publications: Qualitative or quantitative research that has undergone a third-party or peer review process to ensure reliability.  
• Social listening reports: Reports on discussions, key words and questions asked on social media. These help you tune in to public opinion and learn about the types of rumours or misinformation circulating on the ground.  
• Country information systems and administrative data: Many governments and organizations have databases that provide regular, real-time information on their population.  
• Country census reports: Nationwide surveys that gather information on demographics, economics and social data at national, provincial and regional levels every 5-10 years.  
• Multiple Indicator Cluster surveys (MICs) and Demographic Health Surveys (DHS): Household surveys that monitor the situation of population groups, provide information on inequities and measure progress toward Sustainable Development Goals (SDGs).  
• Knowledge, Attitude and Practice surveys (KAP): Studies on a specific population that gather information on general practices and beliefs.  
• Media landscape surveys: Surveys that focus primarily on the capacity of media channels to reach the population and their geographic and demographic targeting abilities.

Selecting research methodologies to fill the gaps

If the evidence reviewed is not recent, does not cover your key populations or provide insights on your challenge, you will need to conduct your own primary research. This section covers six common research methodologies you can use to fill gaps in your research.

1. Interviews 

What are interviews?  
Structured or semi-structured conversations between a researcher and one or more respondents. These conversations are guided by a series of questions that are designed to reveal a person’s attitudes, perspectives, knowledge and intentions on a particular idea, programme or situation. 

There are two types of interviews: Key Informant Interviews (KIIs) and Focus Group Discussions (FGD).

1. KIIs are interviews with only one respondent at a time. Respondents are typically people with  particular technical knowledge or a relevant position (e.g., government staff, local leaders, frontline workers, etc.) or people who have direct lived experience with the challenges you are addressing (e.g., parents of young children, teachers, commercial sex workers, etc.)
2. FGDs are interviews with a purposefully selected group of respondents, around 6-10 people. Respondents are asked a set of open-ended questions.The composition of the focus group should be carefully considered, to bring out the most honest reporting. For example, to encourage women to share their candid views on a topic, consider hosting separate FGDs for men and women. FGDs can help build your initial understanding of the context and culture early on in the research process and help you make sense of survey responses and fill in gaps in knowledge.

Why are interviews useful?

Interviews provide context, offering a more complete picture of the data you have collected. They bring out the nuances and complexities of people’s lives and the stories behind the statistics. They help you build empathy with the people you seek to serve and a deeper understanding of their situation.

What are the limitations of interviews?  
Interviewees’ responses are subjective and prone to biases. Participants may choose to leave out information that could reflect negatively on them or feel pressured to say what they think the researcher wants to hear. Subtle cues from a researcher’s tone, reactions or facial expressions can influence how people respond to them. People also have personal interests in local situations, which may impact their responses. It is important to triangulate findings from interviews with findings from other methods, to see whether certain statements hold true.

2. Observations

What are observations?  
A way to see and hear things people may not tell you. They allow you to document and study how people behave in real-life situations (e.g., in their home, health clinic, community, or school) and how they respond to certain challenges. Observations can be gathered passively, using tools like time-lapse video to capture the traffic to latrines. Observations can also be collected through interactions, like asking people why they do certain things.

Why are observations useful?  
Observations can help reveal influences of behaviour that people may not reveal in interviews. They also offer a first-hand visual experience of the problems that people face and the services that they must navigate. Observations help to bridge the gap between what people say and what they do.

What are the limitations of observations?  
People often behave differently when being watched. To reduce the impact of your presence on their  behaviour, allow yourself enough time to conduct observations so that people become desensitized to your presence. You should also triangulate your findings with other evidence to validate your observations.  

3. Human-centred design research

What is human-centred design research?  
A broad term for the process used to better understand the underlying and sometimes hidden desires, needs and challenges of participant groups when co-designing a solution to a problem. It involves having open conversations to cultivate empathy with the people you are designing with, generating ideas and building prototypes together, sharing what you have created and, eventually, testing these solutions out in the world.

Why is design research useful?  
Human-centred design research is a practical and cost-effective tool for evidence generation which ensures solutions continuously consider participants and their needs.

What are the limitations?  
Human-centred design research is rapid, cost-effective and can lead to insights more quickly than other methods. However, critics argue that the process relies heavily on intuition rather than facts.

4.    Social mapping

What is social mapping?  
A visual method that helps us study and understand how members of a community interact and affect one another’s thoughts, feelings and actions by mapping groups, people and their relationships on a diagram. In some cases, these diagrams can map the physical location of households relative to important community social structures and other key locations.

Why is social mapping useful?  
Social mapping is useful because it sheds light on influences and power differentials. It uncovers which stakeholders are important to partner with and which social inequities need to be addressed. The spatial analysis also provides a different angle, helping to uncover blind spots.

What are the limitations?  
This process can become fairly complex in large, interconnected communities and groups.

5.    Surveys

What are surveys?   
A set of structured questions that help us understand various dimensions of a problem. Surveys can reveal cognitive or social drivers of behaviours such as attitudes, beliefs and social norms. Surveys often provide close-ended response formats, such as by rating levels of agreement on a scale. For example, a survey might ask about a mother’s likelihood of breastfeeding her newborn on a scale of 1 (not likely) to 7 (extremely likely).  

Why are surveys useful?   
Surveys are useful for quantifying parameters, and later for measuring change. If sample sizes are large enough, they can also help disaggregate the data for different segments of a larger population.

What are the limitations?  
Surveys are only reliable if respondents report accurately. People can be forgetful or unaware of the unconscious drivers at play in their behaviours and social interactions. They may also feel pressure to respond in a socially desirable way. Surveys provide more thorough scoping and exploration and help quantify the findings uncovered by qualitative measures, however, they are less helpful for understanding the problem itself.

6.    Experimentation

What is experimentation?  
A procedure where a group of people is exposed to a pilot intervention to observe and measure its  impact. The level of experimentation you choose will depend on the time, resources and capacity available to you. Randomized controlled trials (RCT) are the most rigorous type of experiment. In an RCT, people are randomly assigned to either an intervention group or control group and researchers measure any differences in results between groups. Other common methods of experimentation include pre-post studies and A/B testing.  

Why is experimentation useful?  
Experiments are useful because they help you understand if an intervention will have its intended impact. They help ensure a solution’s effectiveness before any major investment occurs and can also reveal unintended impacts.  

What are the limitations?  
Experimentation is a final step that comes after the problem is well understood. Experiments can be lengthy, so it may take a while to get results. Conducting rigourous experiments often requires a significant investment of time and resources.  
 

Remember to triangulate 

Social and Behaviour Change is all about making sense of complex situations. To achieve this, you should use a variety of research approaches.

If you do not have a clear understanding of the problem, consider qualitative research methods such as interviews, observations, human-centred design research and social mapping. This type of formative research will later help you design your quantitative tools and ensure they are measuring the right things.  
Because they typically involve a large number of people, quantitative research methods like surveys and experiments, can help validate and quantify qualitative findings.

Refer to these rules to help you triangulate your findings:

Qualitative methods should follow quantitative methods when…  

• You want a deeper understanding of people’s responses.  
• You want to build more empathy with participant groups.  
   

Quantitative methods should follow qualitative methods when…  

• You want to know whether your findings can be generalized to a broader population.  
• You want to test a hypothesis based on information collected through interviews.  
• You want to measure the parameters you have uncovered or the impact of the work.

For more information on mixed-methods approaches and how to combine qualitative and quantitative data, visit this website.   
 

How to select a research methodology  
 

For guidance on how to select a research methodology, refer to the BetterEvaluation Rainbow Framework.

Resources 

Evidence mapped at global level  

• A summary of research on social, behavioural and community engagement interventions for reproductive, maternal and newborn child health  
• A searchable database of behavioural science evidence and results
• Data on community feedback, social listening, infodemic and risk communications and community engagement activities

Tools for understanding different research methods   

• A one-stop resource with hundreds of tools, methods and processes  
• Guidelines for interviewing children   
• Guidance for when to use in-depth interviews
• A guide for conducting in-depth interviews
• HCD for health resources   
• A field guide to Human-Centred Design  
• A social mapping toolkit  
• Tools for data collection  
• An A/B testing tool  
• A toolkit for randomized controlled trials in development settings