DPME Evaluation Design Clinic – 6, 7 September 2022
As part of the National Evaluation System (NES), a system of Evaluation Design Clinic is used to facilitate stakeholder participation in the conceptualisation and design of the evaluations prioritised in the National Evaluation Plan (NEP). The last Design Clinic was held in September 2021 with custodian departments and proposing parties on the evaluations identified for the periods 2021/22 and 2022/23. The Design Clinic facilitated stakeholder engagements on problem analysis, programme theory building or refinements where it existed, development of terms of reference (scope, focus, evaluation purpose, questions and methodology), and identification of key stakeholders for each evaluation. The DPME will be hosting a Design Clinic in 2022 for the NEP evaluations planned for 2023/24 and 2024/25.
The objectives of the evaluation Design Clinic is to ensure that the following are developed and achieved:
- A draft theory of change for the programmes/ policies being evaluated in 2023/24 and 2024/25,
- Elements of a draft TOR including the evaluation purpose and evaluation questions are developed, and
- Workshop participants are introduced to the 1-1.5 pager summary for evaluations included in the National Evaluation Plan.
Below are some blogs written with regards to the design clinic. Click the link below for a quick read:
Are Rapid Evaluations Helping African Governments Generate Quality Evidence Faster?
In 2019 Twende Mbele partners introduced rapid evaluations into the existing Twende Mbele countries’ national evaluation systems in an effort to address time lags in providing results to decision-makers. Adopting a rapid evaluation approach is envisaged to deliver timely, accurate, and insightful evaluation results, leading to enhanced use of results in decision making.
Twende Mbele partners have so far adapted a collaboratively generated rapid evaluation guideline to their country context, trained government officials in practical approaches to rapid evaluation, and supported the implementation of pilot rapid evaluations. This work has highlighted a few challenges that contest whether the public sector is ready for rapid evaluations, or if use of evidence in this form will be hindered by the same challenges as regular evaluations.
Why rapid evaluations and not traditional evaluation approaches
Rapid evaluations are intended to be used for a variety of evaluations questions, where results are needed to feed into programmes and strategies quickly (and on a smaller budget). While rapid evaluation approaches are useful for government decision making on certain evaluative questions (particularly around implementation and learning), more traditional evaluation approaches are needed for complex policies and programmes, or where there is a degree of political sensitivity and/or a need for a very rigorous understanding of impact.
Rapid evaluations can also be distinguished in terms of their intended purpose, for example, being conducted in real-time, or alongside larger evaluations, to support innovation, intervention, development and implementation.
Demand and use for rapid evaluations by African governments
Just like a more traditional evaluation approaches, weak evaluation culture hinders good governance and evidence-informed decision-making. Creating an evaluation culture requires buy-in from government ministries and agencies, to parliaments, to the grassroots level. Building and strengthening an evaluation culture should enhance the effectiveness, efficiency and accountability in the management of development policies and programmes. There must be a steady supply of high quality evaluations (rapid or otherwise), and demand for these evaluations in order to ensure their use.
For this reason, with the help of other partners, Twende Mbele continues to advocate for the use and benefits of rapid evaluations and evidence use as a whole.
Benin, South Africa and Ghana have conducted rapid evaluations with inputs from the Twende Mbele toolkit. These rapid evaluations ranged from 6 to 8 months to complete, and although the time taken to complete these evaluations are longer than envisioned in the rapid evaluations toolkit and guidelines, this is still a shorter turnaround time as compared to traditional evaluations. More positively, majority of the evaluation recommendations have fed into improvement plans and change in programs.
Quality of rapid evaluations by African governments
Twende Mbele has also supported a Masters research project intended to clarify our understanding of the use of rapid evaluations, the conditions under which they are commissioned, the level of research rigour they carry, and the reason for commissioning them. The results of that study found that:
- Commissioners working in Sub-Saharan Africa commission most of the rapid studies, confirming that evaluation evidence tend to follow development aid activity globally.
- Rapid evaluations are found to be more narrowed in focus, generally studying single outcome interventions or phenomena.
- he study found that over 75% of the rapid studies are either intended as a situation analysis or a baseline study, and rarely as an evaluation of programme effect or impact evaluations.
- The paper has also found that externally conducted rapid studies, rapid studies with high levels of participation by beneficiaries, and rapid evaluation intended to make a “go/no go” decision; all take at least three times longer to conduct (measured in weeks).
- The research also found a positive correlation between research rigour and duration of rapid evaluations.
In all, the research paints a picture of rapid evaluations as a tool requiring better planning and implementation to ensure the quality and the timeliness of the evaluation. The lack thereof, could impede on its rigour and robustness. As such, more work is needed to improve their quality and robustness.
Conclusion
There is no need for African governments to have to choose between the use of rapid evaluation and more traditional evaluation – as rapid evaluations can be used in conjunction with more traditional evaluations- more time allows for more data points, more interviews, more literature, and more time in ensuring that the process is good quality, involves stakeholders etc. And more money would ensure access to resources. A rapid evaluation approach can help shorten some of these processes, especially where they are planned for.
Call For Proposal for a literature review
Call For Proposal For a Literature Review
The purpose of this TOR is to invite proposals for a literature review focusing on providing options for practical tools used to strengthen government budgeting systems by improving the use of evidence (with particular focus on M&E).
Application Deadline: October 31st 2021
Location: Remote
Use the link below for the full terms of reference: bit.ly/3beR6tn
Send applications to [email protected]
Atelier De Validation Du Rapport De l’Etude Diagnostic Sur La Collaboration Entre Le Gouvernment Et La Societe Civile
Atelier De Validation Du Rapport De l’Etude Diagnostic Sur La Collaboration Entre Le Gouvernment Et La Societe Civile
INFOSEC, COTONOU LE 17 AOUT 2021
Présentation du rapport provisoire de l’étude diagnostique sur la collaboration entre le Gouvernement et la Société Civile en matière de suivi et d’évaluation
New Twende Mbele Co-Chairs
Meet Our New Co-Chairs
Aloyce Ratemo is the Director in the National Treasury and Planning currently heading the Monitoring and Evaluation Directorate within the ministry.
Timothy Lubanga is the Comissioner for Monitoring and Evaluation of Central Government in the Office of the Prime Minister of Uganda, under the Directorate for Monitoring and Evaluation.
CreativeBox
