VALIDATED INSTRUMENTS FOR ARTS IMPACT RESEARCH
Please feel free to contact me about this list – it is not exhaustive and doubtless there are some in there which have more or less efficacy than others!
Instruments for analysing personal and group change in psycho-social interventions and training programs
This is a very helpful table outlining approx. 20 different scales which can be used for evaluating change resulting from an intervention like arts participation.
Instruments for analysing impact of arts participation on academic and non-academic outcomes
Andrew Martin et al, ‘The Role of Arts Participation in Students’ Academic and Non-Academic Outcomes: A Longitudinal Study of School, Home and Community Factors,’ Journal of Educational Psychology 2013 105(3): 709-727.
This study was conducted longitudinally amongst approx. 700 students, to account for prior variance. The survey included questions from the following instruments.
|Area of Enquiry||Instrument||More info||Rating scale||Reference||Availability|
|Active arts participation (out of school)||Adapted from Programme for International Student Assessment 2000 student survey (OECD 2000), consistent with US National Education Longitudinal Study (NELS 2012)||9 items about out of school participatione.g. During the past year, how often have you played a musical instrument? Performed in a play or live theatre?||Frequency ratings 1 to 4 (never or hardly ever to more than four times)||OECD Programme for International Student Assessment (PISA). (2000).||Read moreSample questions|
|Receptive arts participation (out of school)||Adapted from PISA 2000 Survey (OECD 2000)||11 items about out of school participatione.g. during the past year, how often have you gone to the gallery?||Frequency ratings 1 to 4 (never or hardly ever to more than four times)|
|Parent-child arts interaction||Adapted from PISA 2000 Survey (OECD 2000)||10 items about frequency of discussion and interaction in home about artse.g. In general, how often do your parents listen to or discuss popular music with you? / Film/cinema with you?||Frequency ratings 1 to 5 never or hardly ever – several times a week|
|Home arts-based resources||Adapted from PISA 2000 Survey (OECD 2000)||Access to objects e.g. film making equipment, books of poetrye.g. In your home, do you have a musical instrument?||Tally from no/yes responses|
|In-school tuition and external tuition||Adapted from PISA 2000 Survey (OECD 2000)||Frequency of participation during school time
e.g. On average, how much time do you spend each week during school time in these subject areas?External tuition – same item was asked e.g. On average, how much time do you spend each week outside of school in these subject areas?
|Frequency 1 to 4 no time – 3 hr or more a week|
|Arts engagement||General academic and domain-specific items have been administered and validated (Martin 2008, 2009).||Consistent with work on:Emotional engagement – valuing, interest and enjoyment
Cognitive engagement – self-efficacy
Behavioural engagement – persistence
Cognitive, affective and behavioural engagement items for each of the five arts curriculum areas.
|1 to 7 (Strongly disagree to strongly agree)||Martin, A. J. (2009). Motivation and engagement across the academic lifespan: A developmental construct validity study of elementary school, high school, and university/college students. Educational and Psychological Measurement, 69, 794–824.||Available for sampling and purchase|
|Motivation||Motivation and Engagement Scale (Martin 2009, 2010)||Measures student motivation on maladaptive and adaptive dimensions.Adaptive
Self-efficacy e.g. if I try hard, I believe I can do my school work well
Mastery orientation e.g. I feel pleased with myself when I do well at school by working hard
Valuing school e.g. learning at school is important
Persistence e.g. if I can’t understand my schoolwork at first, I keep going over it until I do
Planning e.g. I try to plan things out before I start working on my homework or assignmentsTask management e.g. when I study, I usually try to find a place where I can study well.Maladaptive
Self-handicapping e.g. I sometimes put assignments and study off until the last moment, so I have excuse if I don’t do so well
|1 to 7 (Strongly disagree to strongly agree)||Martin, A. J. (2010). Motivation and Engagement Scale. Summerhill, New South Wales, Australia: Lifelong Achievement Group.||As above|
|Educational resilience||Additional engagement measures – expanded academic engagement factors||Academic intentions/aspirations e.g. I intend to complete schoolAcademic buoyancy e.g. I don’t let study stress get on top of meSchool enjoyment e.g. I enjoy being a student at this schoolClass participation e.g. I participate when we discuss things in classHomework completion e.g. How often do you do and complete your homework/assignments?Sound factor structure, reliable dimensions that are approx. normally distributed and significantly associated with outcomes at school (Liem and Martin 2012)||1 to 7 (Strongly disagree to strongly agree); homework Q, 1 to 5 (never to always)||Liem, G. A. D., & Martin, A. J. (2012). ‘The motivation and engagement scale: Theoretical framework, psychometric properties, and applied yields.’ Australian Psychologist, 47, 3–13.||As above|
|Self-esteem||General Self-Esteem Scale of the Self-Description Questionnaire II (SDQ II) (Marsh 2007)||Overall evaluation of self-worth e.g. Overall, most things I do turn out wellGES has previously demonstrated high reliability (Marsh 2007)||1 to 7 (Strongly disagree to strongly agree)||Marsh, H. W. (2007). Self-concept theory, measurement and research into practice. Leicester, England: British Psychological Society.||Available for free|
|Sense of meaning and purpose||WHO Quality of Life instrument (WHOQOL 1998).||Sense of meaning and purpose in life e.g. My personal beliefs give meaning to my life.Sound reliability (WHOQOL Assessment Group 1998)||World Health Organization Quality of Life Assessment Group. (1998). The World Health Organization Quality of Life Assessment (WHOQOL): Development and general psychometric properties. Social Science & Medicine, 46, 1569–1585.||Contact WHO to request a copy|
|Satisfaction with life||Satisfaction with Life Scale (Diener et al, 1985)||Satisfaction with life in general e.g. In most ways my life is close to my idealGood reliability (Pavot and Diener, 1993)||Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The Satisfaction With Life Scale. Journal of Personality Assessment, 49,71–75.||Available for free|
|Socio-demographic data||Gender (0 female, 1 male)AgeLanguage spoken at home (English 0, non-English 1)Parent/caregiver highest level of education (1 Did not complete school, 2 completed school, 3 completed a college cert/diploma, 4 completed uni degree)|
|Prior achievement||NAPLAN||Student results in NAPLAN||Score for literacy and numeracy formed an achievement factor|
Winner, E., T. Goldstein and S. Vincent-Lancrin. (2013). Art for Art’s Sake? The Impact of Arts Education. Educational Research and Innovation, OECD Publishing http://dx.doi.org Accessed on 30 March 2016.
The following table includes a number of other instruments used in arts participation studies, drawn out of the OECD report and other sources.
|Area of Enquiry||Instrument||More info||Rating scale||Reference||Availability|
|Youth issues||Mission Australia Annual Youth Survey||Open survey of 15-19 y.o. in Australia||https://www.missionaustralia.com.au/what-we-do/research-evaluation/youth-survey|
|Warwick-Edinburgh Mental Well-Being Scale||14-item scale covers both hedonic and eudaimonic perspectivesA shorter 7-item validated version also exists||Available for free|
|Self-efficacy / Self-concept||Catterall and Peppler 2007||13-item global self-concept scalee.g. I am able to do things as well as most other people7-item self-efficacy scale
e.g. When I make plans, I think I can make them work
Every time I try to get ahead, someone stops me
I have control over my future2-item attribution scale
e.g. Good luck is more important than hard work
|Catterall, J.S. and K.A. Peppler (2007), “Learning in the visual arts and the worldviews of young children”, Cambridge Journal of Education, Vol. 37/4, pp. 543-560.|
|Self-esteem||Rosenberg Self-Esteem Scale||e.g. I feel that I do not have much to be proud of; On the whole I am satisfied with myself||Kennedy, J.R. (1998), The Effects of Musical Performance, Rational Emotive Therapy and Vicarious Experience on the Self-Efficacy and Self-Esteem of Juvenile Delinquents and Disadvantaged Children, Doctoral Dissertation, University of Kansas.||Available for free|
|Emotional regulation||Emotion Regulation Questionnaire (Gross and John 2003)||Assesses the emotion regulation strategies expressive suppression and cognitive reappraisalSuppression e.g. I keep my emotions to myselfCognitive reappraisal e.g. When I want to feel more positive emotion I change what I’m thinking about||7 point scale||Gross, J.J. and O.P. John (2003), “Individual difference in two emotion regulation processes: Implications for affect, relationships, and well-being”, Journal of Personality and Social Psychology, Vol. 85/2, pp. 348-362.||Available for free|
|Self-Concept||Piers-Harris Children’s Self-Concept Scale||Children (6-10 y.o.): self-rate happiness, adjustment, anxiety, feelings about school etc||Warger, C.L. and D. Kleman (1986), “Developing positive self-concepts ininstitutionalized children with severe behavior disorders”, Child Welfare, Vol. 65/2, pp. 165-176||Available for purchase|
|Theory of mind / empathy / perspective taking||Reading the Mind in the Eyes test (Goldstein, Wu and Winner 2009-10)||Participants are shown a picture of eyes and have to describe what the person is feeling (readers of fiction, depressed people, adolescents involved in theatre score higher than others)||Goldstein, T., K. Wu and E. Winner (2009-2010), “Actors are experts in theory of mind but not empathy”, Imagination, Cognition, and Personality, Vol. 29, pp. 115-133.||Available for free|
|Theory of mind / empathy / perspective taking||Empathic Accuracy Paradigm (Ickes, 2001)||View a film which stops at various points and participants identify mental state of filmed character – tests ability to infer mental states from moment to moment – dynamic cues – highly ecologically valid, naturalistic measure of theory of mind||Ickes, W. (2001), “Measuring empathic accuracy” in J.A. Hall and F.J. Bernieri (eds.) Interpersonal Sensitivity: Theory and Measurement, Erlbaum, Mahwah, NJ, pp. 219-241.|
|Creativity||Based on Torrance test of creativity (1968), adapted for primary school kids||Originality – able to produce many unique ideasFluency – produce a great many ideasFlexibility – produce multiple types of ideasElaboration – can take their ideas and expand upon them||Catterall, J.S. and K.A. Peppler (2007), “Learning in the visual arts and the worldviews of young children”, Cambridge Journal of Education, Vol. 37/4, pp. 543-560.||Torrance test available for free|
For more information, please contact:
Jackie Bailey, Principal, BYP Group
E: jackie at bypgroup dot com
P: +61 428 576 372
BYP Group does not take responsibility for the accuracy of the information contained in external links.