A Better Start
A Better Start (ABS) is a National Lottery Community Fund (The Fund) 10-year, £215 million programme, supporting families to improve the diet and nutrition, language and communication skills, socio-emotional development and life chances of babies and very young children.
ABS is place-based and aims to create approaches, relationships and services that better support people and communities to thrive.
A key element of ABS is to bring about systems change; that is to change, for the better, the way that local health, public services and the voluntary and community sector work together with parents and communities to improve outcomes for children.
ABS is a ‘test and learn’ programme that amends programme delivery as it progresses, drawing upon both national and local evidence and the experiences of programme staff. The Fund invests in test and learn programmes because of the recognised benefits, particularly in complex environments, where delivery can be supported by being agile and responsive.
Since 2015 ABS has been rolled out in five areas , with ongstanding concentrations of poverty and deprivation (Blackpool, Bradford, Lambeth, Nottingham and Southend-on Sea). Over 45,500 pregnant women, parents, babies, and young children have already been reached by the programme.
ABS includes more than 120 interventions. These include national programmes such as HENRY, which provides a range of support to families to enable children to have the best start in life, and Family Nurse Partnership, which supports improving child development and school readiness. Each of the five partnerships also invest in specific services to meet local needs. For example, Better Start Bradford has its Bradford Doulas service, supporting expectant families through latter pregnancy, birth and beyond.
ABS is supported by a £3.9 million national evaluation, commissioned by The Fund in 2021 which aims to: 1) provide evidence to support ABS partnerships to improve delivery outcomes throughout the lifetime of the programme; 2) work with local evaluation teams to avoid duplication of evidence and enable collation of evidence from local evaluations; and 3) enable The Fund to confidently present evidence to inform policy and practice initiatives addressing early childhood development. The ABS national evaluation team are a consortium led by Natcen and include RSM UK, Research in Practice, National Children’s Bureau and The University of Sussex.
ABS and complexity
The ABS programme contains many features of complexity. These include, as identified in HM Treasury’s supplementary guidance to the Magenta Book (2020):
- The problem being addressed has multiple causes and covers more than one policy domain.
- The environment in which the policy is being introduced is, itself, in a state of flux, or there are already several other initiatives taking place.
- There are large numbers of actors (organisations or individuals) who need to be engaged in delivery of the intervention, increasing the likelihood of conflicts of interest and the presence of different perspectives on the intervention, its outcomes and value.
As noted by CECAN (Centre for the Evaluation of Complexity Across the Nexus) in a
2020 Evaluation and Practice Policy Note, systems are seen as complex when: they have many diverse interacting components; there are non-linear and non-proportional interactions between the components; and there is adaptation or learning by the components in response to change. ABS also includes these elements of complexity, which poses some challenges for evaluation. For example, the behaviour of complex systems is difficult to predict as they may be in a state of continual change, and evolve and interact with other systems in other ways (
CECAN 2020). To respond to this unpredictability, it is important to define the scope of the evaluation and allow for flexibility within the evaluation design.
Approach taken to the ABS national evaluation
1. Development of evaluation design and evaluation objectives
Prior to publication of the Invitation to Tender (ITT),The Fund’s Evaluation Team developed an evaluation design and draft objectives that: stemmed from a clear evaluation purpose; recognised complexity; incorporated understanding of the ABS programme; and ensured a feasible evaluation design. The four draft evaluation objectives presented in the ITT were as follows:
1) to identify the contribution made by ABS to the life chances of children,
2)to identify the factors that contribute to improving children’s diet and nutrition, social and emotional skills and language and communication skills,
3) to evidence the experiences of families through ABS systems, and
4) to evidence the contribution made by ABS to reducing costs to the public purse relating to primary school aged children.
The ITT specifically requested a two-stage approach to the ABS national evaluation. The first phase required scoping activity to:
- test the feasibility of the draft evaluation objectives developed by The Fund’s Evaluation Team;
- test assumptions about the availability of evidence and data to support the draft evaluation objectives; and
- finalise the approach to the national evaluation.
The second phase of the contract was delivery of the national evaluation (which began in November 2021).
2. Ensuring complexity aware approaches to the evaluation
Careful consideration was given to ensuring complexity aware approaches to the evaluation. To develop evidence about the extent to which ABS has contributed to intended outcomes and to the life chances of children who have received ABS interventions, the evaluations draws upon Contribution Analysis. This takes a step-by-step process to explore how the intervention came about, using a broad range of evidence (HM Treasury 2020).
Contribution Analysis refers to methods which are used to understand the likelihood the intervention has contributed to an outcome observed, or not: known as a contribution claim (for example the process tracing method). It does this through a step-by-step process which explores how the contribution would have come about and uses a broad range of evidence to test this. Contribution Analysis can make use of a broad range of evidence types and can be used for all types of interventions no matter how complex the theory of change is.
3. Evaluation undertaken by complexity aware evaluators
The ITT specified that a complexity aware evaluation team was required with experience of successfully designing and delivering large, complex, multi-part evaluations.
4. Ensuring the evaluation contract included flexibility
The evaluation contract was procured with built in flex to ensure the evaluation remains fit for purpose, aligns with the ABS programme and produces robust evidence during the lifetime of the programme.
5. Working with a Theory of Change (ToC)
The ABS national evaluation works with a
ToC, drawing upon The Fund’s ABS programme ToC and each of the five ABS partnerships’ ToCs. This is because: Contribution Analysis relies upon a clearly- articulated ToC (
HM Treasury 2020); and a ToC is a useful means of articulating and measuring impact in programmes which are both complicated (lots of parts) and complex (uncertain and emergent) (
Rogers 2008). ToC can also be a way of capturing the broader context around an intervention, either intrinsic to the programme itself or its external environment:
‘Every programme is packed with beliefs, assumptions and hypotheses about how change happens – about the way humans work, or organisations, or political systems, or ecosystems. Theory of change is about articulating these many underlying assumptions about how change will happen in a programme.’ (Rogers, 2008)
6. Working together to identify change and/or risk
The approach to delivering the national evaluation draws upon the strengths and experience of ABS partnerships, the national evaluation team and The Fund that goes beyond technical knowledge and experience, as required when undertaking a complex evaluation. Regular discussions take place to identify any developments that may affect delivery of the evaluation. These relate to, for example, the wider context within which ABS operates, developments within each of the five partnerships, and any resulting change required to the evaluation approach and timescales. The evaluation team and The Fund also work closely together to identify, and mitigate, risks.
Final comments
The ABS national evaluation is ambitious and complex. Explicit recognition of the complexity within which both the programme and evaluation operate helps to address challenges as they arise, as does close working relationships between ABS partnerships, the national evaluation team and The Fund. Clear and open communication, attention to detail, and prompt identification and response to change is integral to evaluating in complexity.
Author Bio: Emilie has been undertaking social research and evaluation for more than twenty years. She currently works as an Evaluation Manager with The National Lottery Community Fund and supports the A Better Start national evaluation.