The need to understand Evaluation use by NGOs
This blog draws from research conducted by Elizabeth Asiimwe, a Twende Mbele PhD Fellow at the Mbarara University of Science and Technology, Uganda, in 2017. The study is premised on the assertion that, although there is a growing discourse on evaluation utilisation for evidence-based policy making in the public sector, very few studies have focused in evaluation utilisation in the NGO sector in Uganda. This is despite considerable investments by funding agencies such as the Department for International Development (DFID) and the United States Agency for International Development (USAID) in Uganda. More information on the USAID evaluation policy can be found at: https://www.usaid.gov/evaluation, and for DFID, in the Uganda Operational Plan 2011-2016 https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/389293/Uganda.pdf.
In essence, NGO work provides important social change work that should inform policy making. In this regard, continuous improvement of NGOs’ work through monitoring and evaluation use is fundamental for enhancing evidence-based policy making – provided the system is set up to receive this information. However in Uganda there is a huge gap on studies that explore utilisation of evaluation results. Asiimwe’s study aimed at contributed towards addressing this empirical gap.
Focus of the Study
The study assessed evaluation utilisation among non-governmental organisations in Uganda, aiming to generate evidence to guide policy makers, project planners, evaluation planners, commissioners and implementers on determinants of evaluation utilisation. A mixed methods approach was utilised to collect data from a random sample of 404 respondents across 101 NGOs, using an online tool (SurveyCTO), focus group discussions and key informant interviews. The study adopted a conceptual framework developed by researchers that include Fleischer and Christie, (2009) http://www.managingforimpact.org/sites/default/files/resource/evaluation_use_results_from_a_survey.pdf. The framework illustrates three major factors that influence evaluation utilisation as: evaluation implementation characteristics, organisational and evaluator’s characteristics.
What were the key research findings?
The respondents where from national, community-based and international NGOs working in the areas of health, governance, agriculture, water & sanitation, education and child support. National and international NGOs reported having a monitoring and evaluation department, M&E staff, an M&E strategy, and had conducted at least one evaluation in the past five years. The study established that 80% of the respondents had conducted an evaluation in the last five years (Figure1) and of these, 68% reported that they had utilised evaluation findings (Figure2). Evaluation implementation factors that strongly determined evaluation utilisation were: the relevance of that particular evaluation to the organisation’s current strategy, and the credibility and quality the evaluation. Although the timeliness and quality of communicating evaluation results were seen as determinants, they were not reported as very crucial. Organisational factors identified included: the organisation’s receptiveness to new ways of doing things, the need for evaluation information to support decision-making, the organisation’s trust of the results presented from a particular evaluation and perceived implication on the results on continuity of projects/programs. Evaluators’ characteristics included evaluators’ experience and methodological expertise. The evaluators’ relationship or communication with staff did not have a strong influence on whether their evaluation results were utilised.
Figure 1: Proportion of organisations that have done evaluations
Figure 2: Utilizing of evaluation findings and Non-use of evaluation findings
A proposed way forward
The study clearly illustrated that evaluation use is determined by a cocktail and interplay of different factors. Not all factors that influence evaluation utilisation elsewhere (including interest of other stakeholders in the evaluations, relationship of staff with the evaluators, evaluator’s communication skills) actually determine the same among non-governmental organisations in Uganda. It is therefore worthwhile for commissioners and/or evaluators among NGOs in Uganda, to keenly consider those factors that would influence utilisation as they plan and execute evaluations. It is imperative that intended users are considered and are receptive of evaluation findings for utilisation to happen. Importantly planners, donors and policy-makers ought to mainstream evaluations for policy and programming and to consider sufficient budgets for evaluation to allow generation of evidence that supports decision-making as well as informing programming in general. A full version of the study will be available on the Twende Mbele website soon.
CreativeBox
