Author: Dominique

Diagnostic Study On Demand And Supply Of Evaluators In Uganda Validation Workshop

By Doris Kembabazi

Uganda, as with many African countries, exhibits a need to expand its existing pool of evaluators to ensure supply matches local demand and enable emerging evaluators to enter the market with fewer restrictions. This has been the focus of a recent Twende Mbele study, looking at the factors that influence change demand for national evaluations and supply of evaluators in the three core country partners. As part of this study, a validation workshop was held in Kampala recently to finalise the results of the diagnostic study and consult on next steps.

Key stakeholders welcomed the study on the demand and supply of evaluators and the validation workshop attracted 52 participants.

Government of Uganda, over the last decade, has undertaken initiatives to build capacity within the national M&E system – specifically with the Evaluation Capacity Development project supported by Gesellschaft für Internationale Zusammenarbeit (GIZ) . This targeted both the suppliers and users of evaluations in the public and non-government sector. Through the Uganda Policy Window (supported by 3ie), a number of evaluations have been undertaken, eg UPE (School Facility Grant) Family planning, youth livelihood program etc. That has informed policy decisions in the respective sectors and also gave emphasis on the use of evidence in decision making.

Results of the diagnostic in Uganda show that on the demand side, there are no national government system requirements for eligibility to provide evaluation services. The study also established the specific professional skills required from evaluators by both Government and CSOs including the minimum academic qualifications, experience, core skills and other qualities or attributes. Survey findings from most clients show that the supply of evaluators is not sufficient to meet the demand.

Quality is a key aspect of the supply of evaluators, and should be guided by established standards to ensure quality of the evaluators’ product. In this regard, the Uganda Evaluation Association has put in place the National Evaluation Standards.

The study also outlined a number of opportunities for local evaluation capacity building in Uganda at  universities, training institutions and professional networks.

To strengthen the evaluation capacity and supply of evaluators there is need to strengthen the culture of management that understands, values and uses evaluative information to achieve results and organisational performance. There is also need for Government to interface with and support the development of professional evaluation associations such as the Ugandan Evaluation Association. This ties into the need to define key competencies for evaluation that form part of the functional organisation of personnel requirements.

To better match the supply to demand of evaluators, there is need for a deeper analysis of the profession of evaluation in Uganda more especially organizational capacity gaps at Ministries, Departments and Agencies (MDAs) level in Uganda.

The one-day validation workshop came up with an action plan for the next months for Twende Mbele as;

• Support Office of the Prime Minister to establish a Centre of Excellency in evaluation that would establish an efficient Uganda information system on evaluation findings and evaluator availability and opportunities
• Support development of guidelines, regulations, standards and competency requirements (knowledge, skills and abilities) for evaluation positions in Government of Uganda as well as ongoing professional development
• Support a study on deeper analysis of the profession of evaluation in Uganda
• Support training all private and government organisation managers to gain evaluation skills
• Identify champions of evaluation to promote evaluation in government ,and private programs and projects etc (for more details see attached report).

Participants appreciated Twende Mbele for supporting the continued improvement of quality in the national evaluation system. The workshop was concluded with drafting the action plan for the next Twende Mbele quarter, and the Uganda Evaluation Association spearheading some future work.

Evidenced Based Policy Making and Implementation Course: The Ugandan Chapter

By Doris Kembabazi

There is an increasing pressure on policy makers to develop more effective policies to best direct and manage resources in more focused and efficient ways that result in improved implementation and outcomes. Evidence-based policy-making is an approach that has become increasingly prevalent in recent years in Africa. It is based on the premise that better policies and better decision-making result when these are based on sound empirical evidence and solid rational analysis. It is also critical to use evidence to improve implementation. Evidence-Based Policy-Making and Implementation (EBPM&I) therefore focuses on establishing rigorously objective evidence as a key informant of policy, but also for improving implementation of public services. Evidence policy making and implementation in Uganda plays an important role, especially in resource-constrained settings where informed decisions on resource allocation are paramount. Several knowledge translation models have been developed, but few have been applied to health policymaking in low income countries like Uganda hence a big challenge to the policy makers.

The desire to use evaluation findings and the importance of credible evidence in decision-making is emphasised in Uganda’s National M&E Policy, the National Development Plan (NDP) and international frameworks, such as the Sustainable Development Goals (SDGs). The rationale for the use of evidence is exemplified by better Government decisions and effectiveness in their implementation; rational decisions in resource allocation in the choice of policies and programmes to implement; and provision of feedback to influence future policies and programmes.

As part of the Twende Mbele African Partnership Programme, the Evidence Based Policy Making and Implementation course in Uganda was an executive course for strategic leaders and top managers in the public service. It is adapted from a University of Cape Town course; designed to assist participants to use evidence to make well informed decisions about policies, programmes, projects and services and to improve government’s impact on society. This was the first time the course was run outside of South Africa and was adapted and piloted in collaboration with UCT.

The Jinja course was officially opened by the Minister General Duties Honourable Mary Karooro Okurut who gave opening remarks on behalf of the Government of Uganda. She acknowledged the benefits of the Twende Mbele initiative and the efforts that had been put in place. She noted that in Uganda, evaluations are beginning to be viewed in a positive light, with growing demand and ownership by Ministries, Departments and Agencies (MDAs) and other stakeholders outside the public sector. In this regard a number of evaluations of strategic areas have been undertaken successfully over the last decade.

The course represented great coverage, as 35 Directors of the Ugandan Government attended, equalling more than 80% of the full cohort of Directors. The course was covered in two days with Facilitators from the Department of Planning and Evaluations (DPME) South Africa, Makerere University and Office of the Prime Minister, Uganda.

On the first day, participants were introduced to EPM&I approach and cycle, and to the Diagnosing a Problem tools and day two built on this by inspiring participants with case studies of evidence processes and then exposing them to the Theory of Change and evaluation tools which can be drawn on in subsequent stages in the EPM&I cycle.

This course was the first of its kind to bring together the directors from MDAs and they had a number of suggestions, firstly, they all appreciated the opportunity to have one voice in future policy-making and implementation. Another suggestion was to create a platform to continue sharing and learning from each other on a day to day basis. The participants also decided to work with the Office of the Prime Minister to strengthen M&E systems for example Human Resources and to write a briefing paper specially on the practical strategies to institutionalise use of evidence.

This course will be translated to French and run in Benin in the second half of 2018.

Etude Diagnostique De L’offre Et De La Demande D’evaluation Au Benin

By Prosper Houssou

L’étude diagnostique de l’offre et de la demande d’évaluation au Benin, est focalisée sur le marché de l’évaluation avec ses forces et faiblesses, la capacité de l’offre en matière de services d’évaluation, la taille du marché de l’évaluation ainsi que les moyens pour parvenir à un équilibre entre l’offre et la demande d’évaluation, notamment dans un contexte de demande croissante des évaluations.

Les constats portent dans leur ensemble sur les points suivants :

Au niveau de l’offre, on remarque une offre quasi-exclusivement masculine, variée et de niveau académique et professionnel adéquat (minimum BAC+5 et 10 ans d’expérience pour les consultants principaux) et une faiblesse du dispositif de formation initiale et continue.

Au niveau de la demande, un total de 43 évaluations répertoriées (24 évaluations du Gouvernement et 19 chez les PTF) avec une tendance à la hausse au niveau de la demande et un besoin de dynamisation de l’institutionnalisation de l’évaluation (visibilité en légère baisse, élaboration de l’avant-projet de loi sur l’évaluation toujours à l’étape d’intention, implication peu suffisante des niveaux périphériques et du Parlement ralentie, léthargie du Conseil National de l’Évaluation). On note aussi par ailleurs une faible systématisation/capitalisation de la pratique de l’évaluation en général résultant en une indisponibilité des rapports et des informations sur les évaluations passées et à venir et un nombre important de défauts d’organisation imputés surtout aux services gouvernementaux.

Partant de ces constats, Il en résulte les conclusions ci-après suivant des aspects majeurs que sont :

Le genre. Les femmes sont presqu’absentes du marché de l’évaluation. Les efforts pour les impliquer dans des évaluations sont souvent peu productifs. Cette situation pourrait s’empirer dans un contexte de demande en hausse, ce qui les marginaliserait davantage.

La formation initiale et continue. Plus que par la formation initiale qui garde son importance, c’est grâce aux formations de courte durée que plusieurs développent leurs compétences. Elles préparent des techniciens sectoriels à devenir des évaluateurs confirmés.

L’ancrage institutionnel de l’évaluation. Le gouvernement a toujours tenu le lead mais son élan s’essouffle de manière remarquable, notamment pour l’extension aux niveaux décentralisés et des institutions. Les services en charge de l’évaluation changent trop souvent de tutelle ce qui ne facilite pas la cohérence de l’action et la collaboration dans la durée.

La faiblesse de la systématisation de l’organisation des évaluations. Tant du côté gouvernement que des PTF, la faiblesse de l’organisation a rendu impossible la reddition des points sur les évaluations exécutées et même celle à venir. La base de données  du BEPPAAG se profile comme une solution durable à ce défaut majeur.

Le rapport entre l’offre en services d’évaluation et la demande. Au moment de l’évaluation, l’offre satisfait la demande en évaluation. Dans la perspective ou l’évaluation reprendrait de la vigueur avec l’implication de nouveaux demandeurs, cette offre risquerait de se révéler inadéquate. Cette situation appelle au renforcement des capacités d’évaluation.

Gender responsiveness in South Africa M&E system

By Rendani Manugu

Research in Kenya illustrated the challenges in reaching men for male HIV testing through clinics, and so researchers decided to test whether there would be a better up take of HIV testing if men received self-test kits through their spouses. This was important to reduce the spread of mother to child transmission of HIV. When pregnant women visited the clinic for their routine appointments, they were given two oral self-test kits. One for them and the other for their male partners. They were trained on how to broach the subject with the spouse and were shown how to administer the test. The study also had a control group, consisting of men who were invited by the Ministry of Health to attend clinic visits with their spouse for testing. The study found that from the two groups, double the men administered the test given by their spouse, and where there was a positive status, the men confirmed the results at a clinic. This study is an example of a gender responsive programming. It applies a different method of application to men and women, as it responds to the different needs of men and women.

(more…)

Made in Africa Evaluation: Africa’s novel approach towards its developmental paths (Part 1)

By a Twende Mbele Masters Scholar: Mokgophana Ramasobana

(more…)