For government and its agencies, local government, the European Commission, the Lottery, and charitable Trusts, impact evaluation has become a cornerstone in the use, accountability and effectiveness of new policies, programmes and initiatives. Different theories, options and alternative methods have proliferated, and those setting up and running programmes across the public and voluntary sector, and outside, find themselves confronted with difficult choices and rising expectations among policy makers and funding bodies for measuring programme effects, effectiveness and impacts. Pressure on the public purse has intensified those demands - with new calls among policy makers and others for sophisticated evaluation strategies to show impacts, ‘what works’, to demonstrate added-value and cost-effectiveness. This online course provides a more flexible opportunity for an introduction to impact evaluation. It shares much of the same content as the ‘face to face’ SRA course and is led by the same tutor.
By the end of the workshop, participants will:
Be aware of a range of options and methodological choices which can be applied to different impact evaluation circumstances and needs.
- Recognise the different method of impact evaluation, and their uses, including for piloting, scaling-up and roll-out of programmes and initiatives.
- Understand how to manage for realistic expectations of impact evaluation addressing both explicit and implicit needs of users
- Understand also key design principals including the use of proportionality in the design of multi-mode impact evaluation.
- Appreciate the how ‘counterfactual’ evidence can be built into each of experimental, quasi experimental, non-experimental and alternative impact evaluation methods
* SRA also offers an introductory course on Foundations of Evaluation and also to Research and Evaluation Project Management and also an advanced-level course in Theory-based Evaluation.
The course will cover:
- The nature of impact evaluation, how it relates to ‘experimental’ and ‘programme’ theories and its practical use?
- An introduction to different types of impact evaluation – and the principals for fitting proportionate designs to different circumstances.
- The use of sophisticated evaluation designs to measure, estimate or assess counterfactual evidence in different ‘causal’ contexts.
- A review of how experimental, quasi experimental, and non-experimental methods can each help with understanding attribution and ‘new’ alternative ways of looking at qualitatively assessing causality; as well as the pros and cons of these approaches and the main methodological options in each.
- The governments Magenta-Combined and other practical guidance – and it usefulness for meeting evaluation needs of publicly-funded programmes or initiatives.
Who will benefit?
This course is run in one day as a real-time online tutored course. It combines ‘tutor-led’ sessions, illustrations and case studies and practical (breakout group) exercises, and series of online supporting resources. It is aimed at those with some experience of evaluation who may be working in a wide range of situations procuring and specifying programme evaluations, or designing and conducting complex evaluations.
By the end of the course, participants with a basic understanding of evaluation will have been introduced to the contrasts between experimental and ‘theory-based’ impact evaluation and will understand a range of practical options available to them for both formative and summative impact evaluation. They will better understand national and other guidance to make informed decisions about how to effectively specify, combine and use sophisticated quantitative and qualitative impact evaluation methods. They will gain a practical insight into the use of ‘empirical’, ‘non-empirical’ and alternative approaches to impact evaluation, as well as practical pitfalls, pro’s and con’s, and how multi-goal evaluation can be made more cost-effective.
Professor David Parsons is a long-standing evaluation specialist, advisor and author of Demystifying Evaluation (Policy press, 2017) and a number of programme guidebooks on evaluation strategies. In academic, contract research and in practice within government and agencies, he has led nearly 50 independent evaluations of policy and programmes across seven government departments, the UKs research Councils, various agencies, the devolved administrations, European Commission and others. He has specialised in proportionate methods of impact evaluation, for both public policy and evidence-based practice, and in this role continues to advise a number of public bodies, regulators, charities and others on evaluation strategy and methods.
This course contributes 6 hours to the MRS CPD programme