Evaluating arts and social inclusion projects

Screen Shot 2015-10-29 at 11.57.56 AMYesterday I gave a presentation at the Creative Victoria Expert Arts Panel session on evaluating arts impact. Along with me, Deakin Uni’s Hilary Glow and Anne Kershaw presented about a recent evaluation they did for Vichealth on arts and wellbeing, and Mark Hogan from Regional Development talked about the Clunes Booktown regional transformation story.

Once Creative Victoria upload the full session to the web I will upload a link. In the meantime, I have uploaded my powerpoint presentation here.

 

Thumbnail guide to evaluation

This is just a little something I thought I would include here. I have been working on it for a client and thought it might be useful to others too. Nothing top secret, so completely fine to share.

Summary of key types of evaluation

Type Description Resources
Formative Evaluation This is a tool to establish the needs and baseline of the community before a project is designed and commenced. It can include:

  • A needs assessment
  • Community consultations about needs and priorities
  • Benchmarking research, e.g. surveys of the community about characteristics which the project will aim to change (e.g. wellbeing, social inclusion, arts access)

Mission Australia Youth Survey

ABS Community Profiles

The Warwick-Edinburgh Mental Wellbeing Scale

Process Evaluation This looks at how the project was implemented. It can include:

  • program fidelity – the extent to which the process was implemented as intended
  • feasibility – problems and solutions that were implemented along the way
  • reach, equity, intensity of involvement – the level of uptake and participation in the project

UNSW Centre for Social Impact guides

Impact Evaluation This looks at the impact the project has had over the short, medium and long-term. In this approach, parties agree on a ‘theory of change’ – what they think will happen as a result of the program. Then they look at whether this occurred, and what unintended outcomes may have arisen. They look at impacts for each major stakeholder group e.g. participants, partners, community groups, donors, and the arts organisation. Some evaluators use a ‘Program Logic’ approach, or a ‘Most Significant Change’ approach, where they try to identify the key change resulting from the project for each stakeholder group.

Methods of measuring impacts will depend on access to participants and what can be feasibly and reliably measured. Methods regularly include a review of participation numbers and data, surveys, interviews, ethnographic observations and focus groups.

UNSW Centre for Social Impact guides

Upcoming BYP Group presentations

We will be presenting at a number of events over the coming months. We will attempt to upload any powerpoint slides to this website, but just in case you think can come to any of the below, here are the details:

Jackie will be a panellist on an Expert Arts Panel on evaluation and the arts, Creative Victoria, Melbourne, 28 October 2015

Jackie will be delivering a Keynote Address – ‘Meaningful Measurements – How do we articulate the value of public libraries in a contemporary context?’ SWITCH 2015 – NSW Public Libraries Association, Sydney, 17-20 November 2015

Sarah will be delivering a free regional workshop for Creative Victoria – ‘Evaluating your Work – Measuring Success in Arts and Culture,’ in Kilmore, Victoria, Friday 27 November.

Jackie will be hosting the panel – ‘Evaluation and Research – 6 Minute Learnings’ at the Australia Council for the Arts Arts Learning Forum, Footscray, Melbourne 25-27 November 2015

 

Impacts of poetry in schools

Screen Shot 2015-09-22 at 2.53.53 pmKey findings from our evaluation of the Red Room Company’s schools poetry education program have now been published as part of the Red Room’s annual report. We interviewed students and teachers and conducted a survey of participants from the 2014 program. The company is using the findings in strategic planning and philanthropic work.

You can read the key findings and more about the Red Room Company here.

If you would like to talk to us about the evaluation or similar work, please contact Jackie jackie at bypgroup dot com or +61 428 576 372.