Evaluation Research Programme
Since Rob O’Donoghue’s 1986 paper entitled “Environmental education and evaluation: An eleventh hour reconciliation”, we have been exploring the relationship between environment, learning and evaluation. This accompanied an early probing of research methodology appropriate for environmental education contexts (Rosenberg, 1995) and its extension in recent years with social theory and a realist under-labouring (Lotz-Sisitka, Schudel, Olvitt, et al.).
At the heart of our evaluation research programme is a desire to support learning, and to limit the use of extractive evaluation methods that are more intimidating than helpful in shaping better educational practice and environmental learning outcomes and impacts. This practical and intellectual quest quickly takes us to a discussion on objectivity and whose insights counts, what knowledge is valid, and how best to produce it in contexts characterized by long lags in open, complex systems.
Mindful of the limitations of positivist models such as the ‘gold standard’ of quasi-experimental designs, we have explored the differences and links between evaluation of EE/ESD; evaluation in and for EE/ESD; and EE/ESD as a form of evaluation, i.e. a reflexive process of comparing current and desired practices and socio-ecological outcomes (see O'Donoghue, Rosenberg, Joon and Krah for a recent paper). Our research projects with an evaluation focus have included narrative approaches such as the Most Significant Change Story model developed by Davies and Dart (Rosenberg and Burt) and Appreciative Enquiry (O’Donoghue); Wenger, Trayner and De Laat’s non-linear Value Creation Methodology (Durr); and Pawson and Tilley’s realist framing that compares Context, Mechanisms and Outcomes (Songqwaru, Ward, Mtati). Current students are exploring the links between reporting and learning in organisational contexts (Mudau); the role of environmental monitors as citizen scientists contributing to evaluation (Mtati), and Sen’s capabilities approach (Human), among others. A short course is in the planning and we are consulting interested parties. (Please follow this link for an orientating questionnaire in the form of a hand-out for a round table discussion at CIES2019.)
Evaluation research at the ELRC has contributed to the evaluation of a number of small and large scale EE and ESD projects including national and regional EE/ESD teacher education programmes, and to the development of participatory monitoring, evaluation and learning frameworks for natural resource management (NRM) and social learning programmes in South and Southern Africa (Rosenberg, Ward,, Human, Mtati, Cockburn and Biggs, with Pollard, Ison, Kotschy, Mudau et al). In these latter contexts, the importance of not only tracking expected outcomes of a bio-physical and social nature, but also emergence in complex and radically open social-ecological contexts, is directing us to use multi-modality research such as the realist approach outlined by Andrew Sayer. For a selection of PowerPoint presentations mapping some of this work in the NRM context, please use the following links:
- ELRC Evaluation Research Selected Projects Rosenberg
- PMERL in Pictures Rosenberg
- Realist Evaluation Conference Presentation Rosenberg
- Taming Monsters M&E in NRM Rosenberg
- Innovation in M&E RESDEV2017 Rosenberg Burt Kotschy
Recently we started to focus on evaluation in the context of the national post-school sector education and training authorities (SETAs), as Rhodes was awarded a Research Chair for Monitoring and Evaluation in a SETA Environment (Rosenberg, Ward, Nodada et al.). This brings into play the need to capture, grasp and reflect a range of processes and impacts, from performativity to high level outcomes, in systems and processes that are practical, open to standardization and aggregation, but also generative of new insights into how best to achieve more, more relevant and more cost-effective skills outcomes in the South African post-school system.
We invite interested researchers to join us in studies that address the broad knowledge interest outlined here, and in particular to help the field to develop interesting, feasible evaluation methodologies including the use of new affordances provided by mathematical modeling, social media and smart phones, and strong learning theory.
Last Modified: Tue, 26 Sep 2023 18:36:20 SAST