The ‘Festival of Hope’
In March, Young Producers at five Hub museum services across the North West of England were poised to launch the Festival of Hope – a celebration and showcase of youth-led creative work. The festival is part of a five year project called Hope Streets that is aiming to embed best practice in youth engagement in the heritage sector (one of the National Heritage Lottery Fund ‘Kick the Dust’ projects). My role as external evaluator is to document the journey towards organisational change at each Hub - the achievements, the challenges and the learning to share. My brief turned to assessing the impact of COVID on the Festival of Hope, and the role of digital engagement (i.e. the use of digital channels to engage with Young Producers, their audiences and the people they work with).
1. Context Setting
Understanding the impact of the crisis on the project’s stakeholders as well as the project itself was an important step and will help the interpretation of evaluation findings. Context setting can confirm or challenge assumptions (e.g. about the needs and priorities of young people as the recovery continues, and their access to technology) and identify external factors that may affect project activity (e.g. the uncertain future funding landscape in the heritage and youth sectors).
2. Pause, Pivot, Pilot & Proceed
Describing project activities since the start of the crisis is helping to build the narrative of COVID impact. We are categorising what activities have paused (due to local circumstances, or to take time to reflect and plan), pivoted (adapted quickly to new forms of delivery), been piloted (using opportunities to develop and test new approaches) or proceeded (continued to work towards objectives) and why. Each of these crisis responses is valid and depends on the particular context of each Hub. Key for the project staff has been respecting Young Producers wishes to either pivot to an online festival or pause and hold onto their ideas for a ‘physical’ festival in the future.
3. Digital Baseline & Change
Monitoring change in digital capacity from baseline, through the crisis, to post-festival is illustrating how the COVID crisis is influencing organisational change. ‘Digital’ is a broad umbrella term taking in all aspects of an organisation. It has been useful to list what digital roles, activities, infrastructure and policies were in place prior to the crisis that enabled (or otherwise) project activity during the crisis, what needed to change immediately during the adjustment to the crisis, and what developments may be needed to facilitate future digital engagement work.
For Hub museums, the work of the Young Producers in a long-term funded project became more visible and influential. Prior to re-opening, key staff had more time to work with the Young Producers, in an exchange of ideas and skills. A range of new digital platforms have been piloted (e.g. for remote working and for producing digital content). Project staff have developed new policies and procedures for safe online working with different age groups.
4. The Quality of Digital Engagement
The ‘quality’ of digital festival content is not the output of interest in terms of the evaluation of organisational change. Of interest is what factors facilitated (or otherwise) work towards the project goal of embedding youth engagement at the Hub museums. Examples of mechanisms of change across the spectrum of digital engagement have been apparent.
Early in the crisis the project focused on maintaining connection with Young Producers and the continuity of the project, before moving on to consulting with young people about their vision for an adapted festival. While there has been a slight drop in the number of young people participating, the quality of engagement is felt to be deeper and more participatory. The move to online working has given Young Producers more decision-making agency over the content and shape of the festival. There have been more opportunities for Young People to invite heritage professionals into their world, albeit remotely.
5. Digital Skills
Understanding context, describing activities, tracking digital capacity and considering the quality of engagement lay the groundwork for evaluating digital organisational change. Have project outcomes been carried through the crisis and what impact on organisations has been observed? The Hope Streets evaluation is looking for evidence of change in 1) organisational data and documents (e.g. a youth focus in forward plans, including digital strategies), and 2) in attitudes, skills and knowledge (e.g. training plans specifically around young people). This is where the evaluation will go next.
If Hub museums choose to dig even deeper, there are a range of sector support tools and resources now available to support self-evaluation. I referred to the One-by-One project when thinking about this evaluation. They have provided a useful framework for defining and assessing Digital Skills: digital competencies (being able to perform digital actions such as set up a social media account or collect analytics), digital capabilities (being able to achieve an outcome, such as building a new community or finding new ways to access collections) and digital literacy (e.g. a deeper understanding of digital audiences, or developed digital strategies).
6. Analytics & Metrics
Young Producers and Hubs will be interested in how much attention digital festival content has received on websites and social media. Analytics could provide a powerful illustration of the reach of young people’s work, the online engagement generated by digital activities (e.g. in comments, shares etc.) and in conversions to new and more diverse digital audiences in the longer term.
However, I have deliberately not referred to analytics and metrics so far. I took a snapshot of each organisation two years ago and before the festivals, including the range and number of digital channels and digital audience size, and will be able to see any changes in these. But this isn’t an activity or outcome we are directly interested in. Just collecting analytics is not enough and to answer the evaluation questions about digital change means a mixed methods approach.
From the start of the project we have used analysis of documents and organisational data, interviews, diary methods and attitude surveys. Two months after lockdown it was useful to revisit a facilitated visioning exercise we had done in January (which included a SWOT analysis), to elicit new hopes and concerns for the future and explore new opportunities and challenges arising from the crisis.
Enjoy The Festival
The festivals and the evaluation continue. Please do visit the festival website to see amazing content by young people in the North West, such as stories of hope collected from people in Bolton, the alternative and hidden heritage of Carlisle, creative work from Chester, responses to current crises on film in Sefton and explorations of community in Winsford.
AUTHOR BIO: Dr Cath Dillon is an independent research and evaluator, currently commissioned by Curious Minds (an Arts Council England bridging organisation) to evaluate the Hope Streets project (led by Kelly Allen, project managed by Saul Argent, with Dr Steph Hawke and support from Museum Development North West and Youth Focus North West). The Festival of Hope is being produced by Blaze Arts in collaboration with five museum services in the North West – West Cheshire, Lancashire, Bolton, Tullie House in Carlisle and The Atkinson in Sefton).