By Rebecca Metz
For some time, it has been clear that many non-profit organisations that work within the ECD sector need support with developing their Monitoring and Evaluation (M&E) systems, and DGMT has received a number of requests for help with this. M&E within the ECD sector is complex as there are numerous elements that need to be tracked (for example, health, nutrition, learning, parental practice, amongst others) and a number of the tools used to do this are either too complex (requiring specialist skills), too costly, or not practical for many implementing organisations.
In response to this pressing need, in April 23 individuals from 18 ECD organisations met in Cape Town for a workshop to discuss this tricky topic – M&E – that has become an important game-changer in a constrained funding environment. From the outset, participants were encouraged to see it as an opportunity to talk about challenges that they are all facing, to share and to learn from one another – free from the judgment of funders.
Besides some introductory discussions on ‘what is M&E and what are good M&E practices?’, we spent time looking at the Theory of Change, and unpacking our understanding of impact. However, it was the case study presentations that provided the greatest opportunity for learning. Amanda Watermeyer from Ntataise presented a Theory of Change for a programme that provides vulnerable children with access to ECD and related services. Amanda demonstrated the benefits of having an in-depth and well thought through understanding of how ones programme works, and how this can lead to well implemented M&E and relevant research questions. She presented an interesting example of how Ntataise interrogated a number of assumptions underlying their programme, and found that one key assumption did not hold true i.e that staff could facilitate access to important social services through a referral system. In fact, staff could not do this because they had been provided with no (or very little) information on these services. By uncovering this false assumption, they were able to take action and hence improve on their programme by making this information available to staff, and hence to parents of young children.
Organisations also presented case studies on measuring ECD outcomes using indicators and tools, in particular relating to learning, which was identified as an area of important need. Andy Dawes shared two examples: the Home Observation for the Measure of the Environment (H.O.M.E) and the Early Childhood Environment Rating Scale – Revised (ECERS-R). These tools measure the quality of the learning environment in the home and for site-based early childhood programmes respectively. Shelley O’Carroll from Wordworks explained how they had effectively developed their own early literacy tool in response to limited access to affordable tests. This test was especially designed to be easily understood and implemented by volunteer tutors. Rejane Woodroffe from Bulungula Incubator explained how they had used the Aptitude Tests for School Beginners (ASB) – a test not requiring a psychometrist – to measure school readiness of pre-school learners on their programme. After hearing the presentations, another 7 organisations shared indicators and tools relating to learning in ECD. It was clear that there is a significant amount of M&E knowledge that currently exists in organisations attending the workshop.
In sharing these tools, there were a few aha moments in the room – and a few frustrated moans. Some of this sharing would have been useful some time ago, before organisations attempted to develop their own tools! An important lesson that we took away from the workshop was that sharing in the M&E field is critical for bringing down costs, for saving time and for replication efforts where similar work already exists.
This was reinforced when organisations had the opportunity to brainstorm a way forward. Some of the points raised included:
- There is interest in further sharing indicators and tools relating to ECD – the DGMT M&E website (which is about to be launched) is a useful platform for this.
- There is a need for more tools to be developed for the assessment of the 0-4 year cohort and for these to be made available through public institutions. There is possibly value in having these tests standardised and normed.
- The collation of M&E lessons (published and unpublished) needs to be more easily accessible to organisations for learning from best practice. NGO staff also need time to write-up these lessons and findings – with a view to publishing them.
- It would be useful to share Theories of Change from the ECD sector.
- It would be useful to train up a group of assessors whose skills could be pooled and shared geographically – this also provides a possible career path for particularly experienced ECD practitioners.
- More exposure to and training in M&E! Quarterly workshops, training in specific tools etc.
- And lastly, donors, keep reporting demands in mind! Different funders have different requirements for reporting and M&E. This can be extremely time-consuming and costly. More standardised processes would help.
The M&E workshop provided a platform for organisations to develop and reflect on their M&E systems, learn from one another’s experiences, and provide a space to brainstorm solutions. It is clear that organisations need a safe space to question these challenging issues away from the competitive pressure to perform for funders, and it is through this that sharing and learning can take place. We intend to continue the momentum beyond this single workshop.
Rebecca joined DGMT in 2013 as a consultant to assist in the development of an M&E component to the Community of Partners portal. She completed her Honours in Economics from the University of Cape Town in 2010, and has worked at the Abdul Latif Jameel Poverty Action Lab for Africa (J-PAL) as a Research Analyst working on designing and implementing Randomised Control Trials in the areas of education and health. She has a particular passion for researching innovative solutions to poverty and inequality in South Africa that have the potential to have beneficial impacts at scale.
