The Growing Practice of Rapid Evaluations in Africa – Insights from Three Countries
In a 2017-19 research study by Twende Mbele to assess the state of M&E culture in five participating countries which included, Benin, South Africa, Uganda, Kenya and Ghana a question regarding the ‘[timeliness of] information provided to decision-makers’ was asked to more than 462 managers.
According to a respondent, information needed to make key decisions does not make it to them on time and when it does, it is not relevant anymore. This suggested that there was a problem with key evidence being available when needed to make decisions. This and other reasons led Twende Mbele to begin an initiative to look at rapid evaluations to plug this gap.
Rapid evaluations are intended to reduce the costs of evaluation projects and the time they take (DPME, 2020). These are evaluations which can produce a result that can feed into policy and practice quickly, but yet is sufficiently robust to provide good guidance for decision-making.
They address the need to quickly assess policy/programme/strategy/function delivery, and establish the main performance data, with main recommendations for improvements (Hercules, 2019). And they also help to understand and learn from what works, what doesn’t, when and for whom.
The last couple of years has seen the advancement of methods for doing Rapid Evaluations, many African countries have taken up the opportunities to conduct this type of evaluation for quicker results. This blog gives a short insight and update of three Twende Mbele country governments in doing the type of evaluation.
At the start of 2019, Twende Mbele designed a Rapid Evaluation Toolkit to provide a framework to think about rapid evaluations, and example questions, indicators and tools to do rapid evaluations. This toolkit is intended for use by officials in evaluations, research and program delivery working in all spheres of government.
In South Africa, the Department of Planning Monitoring and Evaluation (DPME) introduced a Rapid Evaluation Guideline in May 2020. The guideline was developed collaboratively between the DPME, the Western Cape Department of the Premier (DoTP) and the Twende Mbele. The Guideline is an introduction to doing rapid evaluations, either conducted internally or externally. It was designed to provide guidance for implementing rapid evaluations in a government context.
Borrowing from the South African Rapid Evaluation Guideline, Benin adapted the guidelines to its existing national guidelines. A training of National Evaluation System (NES) actors took place in June 2020, with the aim of developing the capacities of national actors in this new form of evaluation for use in the event of an emergency, or as part of a preliminary analysis to help determine priorities, identify emerging problems and trends, and enable decision-making to support adjustments to an intervention. With that said, the Bureau of Evaluation of Public policy and Analysis of Government Action BEPPAAG will be undertaking a Rapid Evaluation of the recent effects of COVID-19 on the informal sector.
In Ghana, the Ministry of Monitoring & Evaluation conducted their first rapid evaluation with IDInsight in 2019. This was the Rapid Evaluation of the One Village One Dam (1V1D) project, an intervention which seeks to increase access to reliable source of water for livestock watering, domestic activities and dry season farming as a means of contributing to poverty eradication and addressing the various forms of inequalities with particular emphasis on rural and deprived communities.
The aim of the Rapid Evaluation was to assess the progress of implementation, emerging outcomes and potential impacts of the 1V1D intervention. It also sought to document emerging issues and challenges that will require immediate attention of the Cabinet, Ministry for Special Development Initiative (MSDI) and other stakeholders.
For critical decision-making, Rapid Evaluation have been used in response to unplanned or planned actions by governments for urgent information needs. Inevitably, there is a trade-off for one to use Rapid Evaluation instead of more rigorous evaluation in the public sector. While there is potential for a greater use of findings when they are timely, a longer evaluation allows for more data points, deeper investigation of the literature, etc.
Second annual gLocal Evaluation Week – 2020
With the ten-year countdown to the Sustainable Development Goals underway, Systems to evaluate the impact of policies and monitor their progress are more important than ever. To sustain the momentum of global efforts to promote monitoring and evaluation capacity the CLEAR Initiative will convene the second annual gLOCAL Evaluation Week from June 1nd to 5th of this year. To take part in this year’s gLOCAL Evaluation Week, simply register and submit your proposal. Applications close 6th March 2020. Click here to register….
The Republic of Ghana Public Sector and M&E Culture
In 2017 the Government of Ghana added a new tool to its development process by establishing the ministries for Planning (MoP) and for Monitoring and Evaluation (MoME). The MoME and the NDPC (National Development and Planning Commission) and the local M&E Association – the Ghana National Monitoring and Evaluation Forum (GMEF) – have developed a National Monitoring and Evaluation Policy. Once cabinet approves the Policy, it will be an important instrument in fostering the culture and practice of evidence-based decision-making in the public sector. The institutionalisation of M&E calls for better planning, use of resources, implementation and learning from outcomes about what worked and what did not.
With the establishment of the MoME and the recent launch of GIMPA’s (Ghana Institute for Management and Public Administration) Masters programme for evaluators, a better coordinated ecosystem for evaluation appears to be forming. However, the government system still faces many challenges, such as:
- weak M&E capacities,
- low demand for, and utilisation of M&E results, limited resources and budgetary allocations for M&E,
- non-compliance with M&E reporting timelines and formats by MDAs (Ministries, Departments and Agencies) /MMDAs (Metropolitans, Municipalities, Districts and Agencies)
- inconsistent data quality,
- data gaps, and
- limited management information systems.
Given that building evaluative thinking and systems is a relatively new endeavour, research into the current M&E culture in the public sector would prove useful. It is for this reason that the Ghanaian government partnered with Twende Mbele to undertake a study to establish a baseline of the M&E culture in the public sector that can be used to measure changes in practices and attitudes over the coming years. The study included interviewing 43 senior management officials from 14 ministries and two agencies using a formulated survey. The results below are based on the Report on the Ghana M&E Culture Baseline Survey.
All (100%) respondents to the survey indicated that evaluation reports are not structured to hide results. Futhermore, most (95%) reported that evaluation results of poor performance are not ignored and 92.50% that senior management does not reject evaluation reports with findings of poor performance.
The MoME is currently working on frameworks and policies to standardise and improve use of M&E across government. This includes the development of a mechanism to obtain information to understand the cause of poor performance. The above suggests a positive management attitude and environment within which to practice M&E in the public service.
According to survey results, recommendations are implemented and learning outcomes are documented and used to improve future results – 92.50% of respondents stated always. The implementation of performance management and performance contracts with managers of the ministries has yielded some results as performance measurement and results are now a basic requirements by Senior level Managers, Ministers and Cabinet for decision-making. There was however, a perception that the officers responsible for poor performance are not sanctioned, (32/38 of respondants stated never), which needs to be addressed in order to motivate staff. The report therefore recommended that a rigorous sanction and rewards regime must be implemented to regulate staff performance in the Ghanaian public service.
An enduring M&E system cannot be developed overnight. However, the Ghanaian government has proven that it is committed to pursuing a desired change based on the work of existing institutions including NDPC, MoME and GIMPA. It has been a steady process and one of the building blocks required was behaviour change. There is also the need for a paradigm shift towards the incorporation of M&E into the routine work of public service staff in addition to deepening M&E processes by which public service activities are conducted in order to build a stronger, learning-focused M&E Culture in Ghana.
By: Rendani Manugu
Twende Mbele, South Africa, Country Coordinator
News from our collaborating partner country-Ghana
By Dr Nana Opare-Djan
Introduction
Over the past year the M&E landscape in Ghana has seen tremendous measures towards institutionalising development evaluation in the entire public sector. Contributions from several stakeholders at different fora, held across the country, gave indications of the high degree of interest in M&E especially the need in drafting of a National Evaluation Policy (NEP) to guide the conduct of evaluation for evidence based decision making at all levels of government.
M&E architecture in Ghana – NDPC/Ministry of Planning and Ministry of M&E
Given the mandate under articles 86 and 87 of the 1992 Constitution, National Development Planning Commission (NDPC) continues with the preparation of the Annual Progress Report (APR) within the framework of the Cross-Sectoral Planning Groups (CSPGs). The process involved the engagement of technical experts with the relevant background and knowledge to review, analyse, and report on progress in the implementation of policies, strategies and programmes using agreed sets of national indicators. The Ministries of Planning, Monitoring & Evaluation (MME) has been created by the amendment of the Civil Service Act, (PNDCL 327) with an Executive Instrument E.I 38 to support government results delivery.
Official Launch of the Postgraduate Diploma in Monitoring and Evaluation Programme
The Center for Learning on Evaluation and Results (CLEAR) at the Ghana Institute of Management has been playing a pioneering role in professionalization of Monitoring and Evaluation in the West Africa Sub region. As one of our key partner institutions in the M&E professionalization journey, we successfully hosted of our first official Postgraduate Diploma in Monitoring and Evaluation Programme as of February 2018.
National Public Sector Reform Strategy (NPSRS), 2018-2023
NPSRS is a five (5)-year project conceptualized to improve public sector performance, especially the delivery of services to citizens and the private sector. A stakeholders’ consultation meeting was held to finalize work on the Public Sector for Results Project (PSRRP) 2018-2023. The objectives of the meeting were to review and finalise project objectives, scope, design features and results framework with key public sector stakeholders; review the proposed components of the PSRRP and agree on details of coverage; discuss financial and procurement arrangements, including necessary assessments; and to define and agree on a preparation programme, key dates, and the budget.
National Evaluation Policy (NEP)
The Steering Committee members however met recently to update the roadmap for drafting the national evaluation policy. The committee recognized the need to have other experts to comment on the draft NEP and provide policy guidance and identify lessons learnt from other contexts for inclusion. A Reference Group responsible for providing the technical and strategic guidance is to be established to support the Steering Committee.
New Twende Collaborators
The Twende Mbele African M&E Partnership has agreed to collaborate with Ghana, Niger and Kenya with the founding countries of Benin, South Africa, Uganda.
In Ghana we note the emerging emphasis on M&E and have agreed to support Ghana to participate in a range of Twende activities, and to support them to take forward their National Evaluation Policy and strengthen relationships between key players in the national evaluation system.
We will involve Niger in Twende Mbele activities, but we note that while a pioneer in evaluation in Africa in the past, it has faced challenges. We note it has had a draft national evaluation policy since 2010, an active VOPE, and we would wish to support them to reactivate the process of adopting and implementing the national evaluation policy.
We will continue to involve Kenya in Twende Mbele activities. We note many elements of M&E are in place but not necessarily well linked, and will endeavour to support the emergence of national and local M&E systems, the Kenya Evaluation Week, and training with Parliament.
Meanwhile we will continue to involve other African countries to share Twende Mbele and their experience.
CreativeBox
