Feedback and Accountability Systems

Introduction

Whether or not you are the kind of person who consistently leaves reviews or religiously reads them before deciding on dinner plans, you know that customer reviews have become an integral part of modern life. The client feedback mechanism does more than offer people useful information on where they might have an unforgettable meal. It provides transparency about services and holds service providers – whether they be chefs or shop owners or ministries – accountable to providing services that meet the needs and standards of those they are intended to serve. It also works to empower and offer choice to users and communities.

This tool offers useful tips on how to create offline and online accountability mechanisms that provide transparency and accountability from public service providers, especially those that serve the most vulnerable or respond to humanitarian crises. Fundamentally, feedback mechanisms aim to professionalize service delivery and treat communities as empowered citizens, partners or clients deserving of high quality, professional services.            

Benefits and social/behavioural objectives

It would be difficult to find a strategic priority that does not involve the design of services in some way. The list below is just a sample of UNICEF priorities that can improve with community feedback:

• Educational enrolment and retention, particularly of girls
• Uptake of vaccination, maternal and child health services, or any other health services
• Birth and vital events registration
• Handwashing with soap
• Prevention of open defecation
• Exclusive breastfeeding, breastfeeding and nutritional support
• Registration/uptake of social safety net programmes

Implementation steps and checklist

Participatory approaches have long been a part of UNICEF’s work, but we can sometimes forget to systematically ask for feedback or respond to it in meaningful ways. Here are a few steps, from UNICEF’s Accountability to Affected Populations Handbook, to keep in mind when designing and implementing your programmes.

Prepare to listen

Being accountable to affected populations means listening to and acting on feedback and complaints to ensure that programmes and responses are effective, relevant and do no harm. We need dedicated systems that allow communities to share their views and feedback safely and ensure that the information is collected, analysed and used correctly. It is critical that these mechanisms are implemented systematically. UNICEF and its partners should be open to receiving as much feedback as affected populations need to provide, through dedicated and informal mechanisms. Staff should be trained to welcome and respond to any view expressed. Feedback is an opportunity for advocacy, transparency and improvement.

It is important to note that giving and receiving feedback, particularly negative feedback, is not always common or socially acceptable in many cultures. In these cases, a feedback mechanism may be seen as artificial, imposed or inorganic. By understanding what works best for communities, we can establish appropriate mechanisms and build trust, familiarity and skills to amplify their use.

A well-functioning complaints and feedback mechanism provides insight on:

• Quality and relevance of services, projects or supplies

• Accessibility of services by affected populations

• User satisfaction

• Organizational performance, including the behaviour of staff and partners

• Responsiveness of programmes

• Types of rumours and misinformation affecting service uptake

By acting on the information provided by feedback mechanisms, we can become more accountable to affected populations. The stages for implementing an effective feedback mechanism are:

• Agree on the mechanism’s key features, including protocols and channels to be used
• Establish the feedback management process
• Integrate feedback collection into regular monitoring and evaluation activities to inform programme design and implementation 

Agree on key features          

Establishing feedback mechanisms takes time. Below are key actions to take when developing a mechanism:

• Secure the support of leaders to ensure the mechanism is adequately resourced and promoted
• Sensitize staff so that everyone has ownership of the mechanism and understands their role
• Collaborate across sectors, to reduce duplication and confusion across different services (see Chapter 13 in the AAP Handbook)
• Consult with affected populations on different features of the mechanism to make sure it is understood, accepted and trusted
• Choose communication channels that suit the socio-cultural context and preferences of the local population and consider the language(s) and communication styles (including written vs. verbal) of different groups, including children
• Consider barriers to access and how to overcome them
• Identify suitable tools for each stage of the feedback management cycle: data capture, management, analysis, visualization and reporting (see Chapter 11 of the AAP Handbook)
• Develop operating guidelines and procedures for ethics, confidentiality and data handling
• Establish a robust, secure referral system for complaints related to sexual exploitation and abuse (SEA), gender-based violence (GBV) and fraud, and maintain an up-to-date list of local service providers, including services for both child and adult survivors of SEA
• Train staff in the required standard of behaviour when engaging with people
• Publicize the mechanism, so that people understand its purpose, how to access it and what to expect when they use it

Decide on communication channels

A single feedback channel cannot accommodate every group within the affected population. Use multiple communication channels, based on the population’s context and preferences. It is also important to consider the availability, acceptability, trust and user-friendliness of different channels. A mixture of analogue and digital channels is recommended to increase the reach of your mechanisms.  You should also consider the resource implications of the channels you use. For example, hotlines require trained staff to take calls and collect data on paper. The information is then transferred to a spreadsheet, which requires a significant time investment from volunteers. 

Manage feedback and complaints

Receive and manage feedback

• Record feedback in a tool linked to a centralized database, using predefined categories (e.g., quality of services, complaints, perceptions, rumours)
• Address cases that can be dealt with immediately
• Refer those with sensitive feedback (SEA, fraud, corruption, etc.) to appropriate entities using the referral protocol and established pathways

Analyse and share data

• Clean the data to remove any corrupt or inaccurate records, as necessary
• Analyse and extract important trends from the data
• Visualize data using a real-time dashboard
• Identify issues and potential solutions
• Report the results, using visuals and a narrative for clarity
• Share the results with senior management regularly, ideally as a standing item at management meetings

Inform decision-making

• Discuss trends, issues and proposed solutions
• Agree on actions to respond to feedback, such as changing activities that are not working
• Allocate time, roles, responsibilities and timescales to ensure corrective actions take place in the short term
• Use the results to inform the next planning cycle
• Use the results as evidence in discussions with donors, governments and other stakeholders

Take action and close the loop

• Track how feedback has been addressed, including through referrals
• Track how feedback has been addressed, including through referrals
• Communicate the actions taken to affected populations
• Evaluate the complaints and feedback mechanism, to ensure it accurately represents the target population and that all actions taken in response to feedback are appropriate

Establish protocols 

Affected populations are likely to use the same channel to provide feedback and complaints. Therefore, a mechanism that is confidential, reliable and trusted is critical to ensure that complaints, particularly those related to SEA or other sensitive matters, are addressed quickly. Mechanisms unable to address such complaints must be upgraded. Any feedback given by an affected person, whether it be positive, negative or neutral, should be listened to and acted upon. Complaints that express unhappiness, dissatisfaction and concern about something or someone, should be given particular attention.

Manage referral pathways 

Feedback related to UNICEF can be handled directly by the relevant sector. In cases where feedback relates to the work of other agencies, there are robust referral systems between different sectors, organizations and governments to ensure nothing is overlooked. If you receive feedback that does not relate to programmes run directly by UNICEF, it is still important to listen to the person providing the feedback and close the feedback loop.

Be transparent and honest when a referral is not possible. Acknowledge the person’s feedback and explain why it cannot be resolved at this time. While some feedback cannot be addressed directly, we can advocate for a solution from other partners or governments.

Protect personal data

Organizations with responsibility for handling personal data must ensure the safety and privacy of every person that provides feedback. The following steps help to ensure their security:

• Assign a trained and qualified data protection focal point, responsible for implementing, monitoring and evaluating data protection measures
• Conduct a privacy impact assessment to identify and minimize data protection risks
• Develop risk mitigation strategies
• Establish formal data protection agreements with partners and third parties
• Train staff and partners on data protection
• Raise awareness among affected populations on their rights in relation to personal data and informed consent
• Assign categories of consent to the different types of data collected, especially any data that requires a referral, to ensure that the most sensitive data is protected

Measurement

Where should you focus your monitoring efforts? Do you measure whether your programme has been effective and appreciated by the community? Or do you measure the success of your feedback systems?

The answer is both. According to UNICEF’s Accountability to Affected Populations Handbook, you should be measuring: 

1. The impact of your SBC interventions, as part of your overall programme’s monitoring systems
2. Levels of community participation, feedback and transparency with respect to how the programme is delivered to ensure accountability to community standards

Specific things to measure about your feedback system include:

• Do people know about the service offered?
• Do people know how to provide feedback?
• Are feedback systems meeting response times?
• Were people consulted at the start of the programme?
• Do people feel they can participate in programme decisions?
• Are issues raised in community meetings documented and acted upon?

In order to measure programme and feedback efficacy, you will need a mix of formal monitoring data, informal feedback gathered through community meetings and discussions with community members or other stakeholders and comments posted on digital platforms like social media. Remember that the whole programme team is responsible for acting on monitoring and feedback data.

Partnerships

Consider recruiting community, digital, telecom and design partners to help you design feedback systems and loops. For example, CSOs, NGOs and volunteer groups like IFRC or National Societies collect offline feedback from communities formally and informally throughout a project cycle. Digital companies like Facebook or telecom companies like Vodafone and Orange can help you reach target groups with important questions. Human-centred design partners can help you design engaging processes that seek out and incorporate user feedback each step of the way.

Case studies and examples 

• LEBANON: Feedback for education in emergencies UNICEF Lebanon developed a complaints and feedback mechanism for education in emergencies. A call centre staffed by UNICEF was supported by a hotline team of trained education partners able to respond to complicated cases promptly. An online portal and dashboard were established for real-time monitoring of different indicators. Feedback categories specific to girls and women provided data to address the low enrolment of girls and parental attitudes towards education. After implementing an intervention to improve enrolment rates, monitoring indicated a 40% increase in demand for education among the targeted communities. Feedback analysis became a key tool for senior managers to advocate for further educational resources, which led to funding for the call centre’s work plan for the upcoming year. During the second phase, ownership of the complaints and feedback mechanism was transferred to the Ministry of Education.
• SINGAPORE: Feedback to improve an established service The Work Pass Division (WPD) processes work permits for foreign workers, who make up about 40% of the workforce. In order to address low customer satisfaction, the WPD teamed up with IDEO to unpack and restructure the customer experience. After changing how staff communicate with clients across the board, customer satisfaction ratings increased to 5.7/6. Additionally, the centre was able to process over 95% of visitors within 15 minutes. WPD concluded that improving the experience can also improve efficiency. Combining business process re-engineering and design thinking has made WPD a world leader in efficiency and customer experience.
• SINGAPORE: Feedback to shape changing community needs Over the next 15 years, Singapore plans to double its rail network and open over 100 new stations. Existing stations were designed with technical constraints as a starting point: What’s the available land size? How many fire exits are required? This time, the engineering team asked a different question: How do you imagine a different future, using design? They learned that the stations needed to respond to community needs. Their designs included childcare centres, bike parking, spaces for community collaboration and more. This exploration led to key design archetypes for future stations that will guide the design process.
• INDIA: Using multi-component interventions including increased accountability from school communities to improve health and wellbeing. A cluster-randomised trial in Bihar state, India, explored how promoting quality school social environments could offer a scalable opportunity to improve adolescent health and wellbeing. The study found that engaging the school community (i.e., adolescents, teachers, and parents) in school-level decision-making processes; promoting social skills among adolescents; providing access to factual knowledge about health and risk behaviours in the school community; and enhancing problem-solving skills among adolescents, had substantial beneficial effects on school climate and health-related outcomes when delivered by lay counsellors.
• NIGERIA: Adapting community conversations to promote more meaningful participatory interventions for child health in Nigeria. Formative research for a cluster randomised control trial in Jigawa State used 36 community conversations to generate debate and discussion around key social and relational processes that would influence the success of participatory groups and action around child health. The conversations gave communities the opportunity to explore and shape the pillars of the intervention. Researchers explained how the process will likely increase the sustainability of the intervention as it ensures local relevance to community groups prior to rollout and evaluation. The study found the need to increase the ownership that target communities have, not only over outcomes, but also process, to reduce the impact of tokenistic participation.
• PAKISTAN: Using community engagement to build accountability of religious leaders during COVID-19 in Pakistan. Religious institutions and leaders who are trusted sources of information play a key role in promoting voluntary compliance with public health measures in the context of a health emergency. With an eye to investigating how communication from religious leaders may influence pro-social public health behaviour, an RCT in Pakistan measured the impact of one-on-one engagement with local religious leadership on the compliance of protocols (e.g., mask wearing) at their mosque. The study found that persuasive scripts can be used to increase the accountability of these leaders, with those imams who received the persuasion scripts being 25% more likely to advise their congregants to wear a mask to prayers.

Key resources

• A Red Cross Red Crescent Guide to Community Engagement and Accountability (CEA)
• UNICEF: Accountability to Affected Populations HandbookDesigning for Public Service
• A Design Solution for Designing Better Customer Feedback: A blog post explaining why people may not respond to well-intentioned requests for feedback
• Designing Great Feedback Loops: An article discussing how to influence behaviour through well-crafted feedback loops.