Creating a Theory of Change for Annual Quality Reporting
TSEP brought together staff and students from over twenty different institutions and unions to discuss the future for approaching Annual Quality Reports. This continues the development of Annual Quality Reports through funded projects funded by the QAA and delivered by NUS from 2012-14 and The Student Engagement Partnership in 2014-15. The majority of students’ unions that have received support to develop Annual Quality Reports for the first time provided positive feedback about the support they received and have continued to produce reports. However, in order to provide effective and sustainable guidance on developing AQR’s at a national level, it is important to understand in greater depth the effectiveness of AQR’s.
A Creative Approach to Developing the Framework
Throughout the sessions we have worked to identify gaps in knowledge and provision, and will to try and break the mould of what is expected of AQRs, coming up with imaginative solutions for change. The creative sessions encouraged participants to think outside of the box, challenge assumptions and innovate. These session aim to challenge our natural predisposition to generate then evaluate, generate then evaluation which can close down creative options, by focussing on separating the idea generation process aside from the evaluation process. The groups focused on generating multiple ideas before evaluation, encouraging new ideas to be explored through combining different ideas, using visual stimuli and a combination of verbal and written techniques, aiming to tackle existing problems and look at ideas from new perspectives.
The attached documents from each of our sessions show the feedback from these sessions, but also include the resources and instructions to these sessions so you can run them in your own unions and institutions, to help evaluate and troubleshoot issues around Annual Quality Reporting.
Session 1 – Perspectives and Gaps (SU Staff/SU Officers/Institutional Staff)
Notes
Brain Pool Expectations Game
Session 2 – The Ethics of AQR (Research Ethics, Marginalised Groups/Interests, Power Dynamics, Resources)
Notes
Card Match – Ethics
Ethics Scenarios
Session 3 – Partnership Working (Reputation, Resources, Reaction, What stages of the process)
Notes
Partnership Quotes
Session 4 – Student Engagement (before, during and after, what groups of students, reaching the hard to reach)
Notes
Just the ticket tickets
Session 5 – Process (Storyboards)
Notes
Session 6 – Lessons Learnt and Unlearnt (Problem Solving)
Notes
Session 7 – Future (New Quality Processes and the embedding of annual quality reporting for students)
Notes
Future AQR matrix
Headlines
Session 8 – Other Frameworks (Best of Lists)
Notes
Assessment and feedback benchmarking tool
Course Reps
Learning resources benchmark
Digital experience benchmarking tool
The Principles for a Theory of Change for AQRs
From the feedback we are constructing a framework, alongside training and guidance, which will support unions to develop their Annual Quality Reporting processes.
We will be focussing on the following areas as the framework principles:
- There is potential for student leadership at all levels, and all stages of the AQR process that should be explored, and where possible implemented. – This includes planning, research, prioritisation and analysis, opportunities for different levels of student engagement fully supported, reps, officers and students involved
- The process will be most effective when based on a critical partnership with the university. – This involved developing Trust, Critical Friendship, Balance, Honesty, recognition of power dynamics, data sharing agreements, multi-tiered across institution (linking with teaching staff as well as senior management).
- It must be underpinned by an ethical approach. – Access and Inclusion, Authenticity, Data Protection, Representing and Reaching all Student Groups, Recognising and Sharing Workloads
- It requires appropriate training and support processes to be in place. – Supported by TSEP, NUS, QAA, HEFCE, Institution and Union, local networks, prioritised time and resources within union, sharing of research expertise from the university.
- It is essential to develop a credible research methodology. – Thorough research planning processes, understanding your approach and limitations, realistic timescales and scope, where possible approach recognised/approved as robust by University
- There must be a comprehensive approach to data collection. – Varied Evidence Base, use of technology, outreach and incentives, mapping existing sources, finding/filling gaps not collecting everything from scratch.
- There must be a robust approach to research analysis. – Properly Resourced, Skilled Analysts, Tailored Approach to variety of Datasets, clarity of write up.
- There must be an effective year round communication strategy that focuses on making the language accessible. – Accessible Language, Varied Approaches Throughout Year, Feedback Loops, mixed media, different student groups/networks, face to face and use of technology.
- The report should lead to enhancement. – Action and Impact, planning, reviewing and accountability, strategic approaches, all levels, long term and short term vision. A focus on sharing best practice as well as critical feedback.
- The whole process should be transparent and flexible. – Futureproofing the Process, allowing for feedback from stakeholders on the process as well as the outcomes, space for critical reflection and development.
For further information on this project please contact: