The Department of Planning, Monitoring and Evaluation (DPME) recognized from the inception of its national evaluation system (NES) in 2011 that there is limited evaluation capacity within government, and externally. This has been supported by the results of the survey conducted by DPME in partnership with the South African Monitoring and Evaluation Association (SAMEA, 2020), which reveals a critical deficiency of evaluation capacity in South Africa. The National Evaluation Policy Framework (NEPF) of 2011 provides for capacity development elements to support the implementation of the NEPF. Between 2011/12 and 2018/ 2019 financial years, DPME and partners made significant progress on evaluation capacity development (ECD). However, despite the notable achievements, ECD still remains one of the main problematic components of the NES. The DPME’s capacity development plan has been broad and included the following elements:
1. Country Peer to Peer Learning
During the early years before the inception of the NES, DPME made significant efforts to learn from the experiences of other countries around evaluation, in order to avoid re-inventing the wheel. In 2011, study tours were undertaken to Mexico, Colombia, the United States and Australia to learn from their experience in evaluation, and the lessons gained from this exercise enabled the DPME to progress much quicker in its ECD endeavours. Likewise, DPME has hosted several countries or received invitations internationally, to share South Africa’s experience on the NES. As part of peer-to-peer learning, DPME has been working with Evaluation Partners and at some point, hosted the Presidents of M&E Associations of Uganda and Kenya. Twende Mbele is a peer-learning partnership around M&E, involving South Africa, Benin, Uganda, Kenya, Ghana and Niger. Using a peer learning approach, this partnership, has allowed the DPME to further strengthen individual and institutional evaluation capacities.
2. Community of practice through inter-Departmental and Provincial Peer to Peer Learning
DPME and Provincial Offices of the Premier recognized a need to create platforms for knowledge–sharing and peer to peer learning. At national sphere, the national monitoring and evaluation forums, chaired by DPME, were held quarterly, whilst provinces also created their monitoring and evaluation forums catering for provincial departments. Since 2015, DPME has been organising annual national evaluation seminars and brownbag sessions as key ECD and learning initiatives. These sessions have provided platforms for peer to peer learning and knowledge sharing. This learning practice is still applicable in the new evaluation discourse and has since included state-owned enterprises (SOEs) and municipalities.
Support through direct experience of undertaking evaluations, in particular the inclusion of capacity development components in all evaluation terms of reference and service level agreements, is one initiative championed by the DPME. Another initiative entails ensuring interns and junior officials get to work with the service providers for the duration of the evaluation. DPME introduced evaluation interns since 2012 and most of them have secured permanent employment both in the public and private sectors.
DPME has also allocated evaluation directorates to departments and provinces specifically to provide technical support as and when requested. DPME also serves in evaluation steering committees across government in order to provide similar support. Given the extension of the NES scope to SOEs and municipalities, DPME has provided technical evaluation support for the evaluations done in these new spheres of the NES.
4. A Suite of Evaluation Courses
A suite of eight courses were developed aimed at supporting the implementation of the NES. Most of these courses were rolled out through partnership with the National School of Government (NSG). This standing partnership is effectively managed through a Memorandum of Agreement (MOA) between the NSG and DPME. This MOA is currently being drawn following the expiry of the previous one in March 2019. A total of 1 989 government officials at national and provincial spheres undertook training of these courses between 2012/13 and 2016/17. The majority of officials were trained in Theory of Change, Evidence–Based Policymaking and in the Managing and Commissioning of Evaluations. Other courses include Evaluation Methodology, Implementing Programme Planning, Technical Evidence-Based Policy Implementation and Deepening Evaluations. Overall, the evaluation of the NES found the training provided very useful, however, the revision of the NSG courses is paramount as most of the courses have been delivered for a long period without being updated and revised. DPME commissioned an evaluation of evaluation courses in 2020 and one key recommendation that emerged was the need to update and revise the course to align with context specific examples. To date, DPME has revised the Evidence Based Policy Making (EBPM) executive course and is almost at the point of revision completion. The other courses, namely, the 1) Managing and Commissioning Evaluations (2) Deeping Evaluation (3) Selecting Appropriate Evaluation Methodologies have been revised in the 2021/2022 financial year through the Twende Mbele support.
The launch of the NEPF, 2019 brought on board rapid evaluation approaches. Twende Mbele assisted DPME in developing the rapid evaluation guideline. Subsequent to that, DPME has partnered with the NSG in developing the rapid evaluation course. The course provides requisite skills for rapid evaluation approaches to be conducted.
6. Partnerships with Twende Mbele, CLEAR-AA, SAMEA and PSETA
The suite of evaluation courses referred above were developed, co-funded and offered by DPME and CLEAR-AA in 2012/13 – 2014/15 – and later outsourced to the NSG. In 2012, DPME signed a MOA with the South African Monitoring and Evaluation Association, (SAMEA) to collaborate on capacity development, evaluation standards and competencies and co-hosting SAMEA conferences. This MOA has since been renewed on a continuous basis. In 2017, both parties commissioned a Feasibility Study on professionalizing evaluation in South Africa culminating in the Road Map for South Africa. In 2017, DPME and Public Service Education and Training Authority (PSETA) signed a MOA aimed at improving the evaluation skills in government. The PSETA have established an M&E bursary programme for government officials in various spheres to study for a Postgraduate Diploma in M&E at the University of the Witwatersrand. To date about 50 Government officials have accessed the bursary and have acquired Post Graduate diplomas from University of the Witerswatersrand and Rhodes University.
A process is underway to ensure that SOEs and municipalities do benefit from this bursary scheme.
7. Evaluation Guidelines
DPME has developed 22 evaluation guidelines and 9 templates. Overall, these have been very helpful to departments and provinces and often used as resource material during training and evaluation process. The evaluation of the NES found that newer evaluating departments and provinces find the guidelines and templates as difficult to put into practice for new entrants. On the other hand, departments and provinces with more experience suggested that the guidelines need to be more flexible, and that additional guidelines are needed for undertaking complex evaluations and different kinds of evaluations.
The launch of NEPF in 2019 presented an opportunity for the revision of all guidelines and the introduction of new guidelines. DPME has introduced new guidelines and those evaluation approaches are a key highlight of NEPF 2019. The new guidelines include the 1) Rapid Evaluation guideline 2) Gender Responsive Evaluation guideline, and 3) Sectoral Reviews and Evaluations guideline.
Guidelines on emergent issues such as Equity and Climate change have been developed through assistance of SAMEA’s 2021 hackathon process.
The blog is based on the evaluation capacity development (ECD) initiatives she shared at the recently held UNDP National Evaluation Capacities (NEC) Conference in Turin, Italy from the 25-28th of October 2022.
CreativeBox
