Access and participation: Building an evidence base of ‘what works’

In this blog, Jo Astley and Dan West respond to the Office for Students’ (OfS) challenge to universities to prioritise generating, sharing and learning from evidence about what is working to make higher education more equitable, and explain how the University of Derby’s Policy and Evaluation Unit (PERU) is responding by establishing an innovative cross-institutional APP Evaluation Framework. The requirement for universities to produce Access and Participation Plans (APP) was introduced in 2018 as a condition of registration with the newly formed Office for Students (OfS). The raison d’etre of APPs is to secure greater fairness and improve social mobility for students described by the OfS as ‘disadvantaged and underrepresented’. The University of Derby’s APP demonstrates a strong commitment to eradicating awarding and graduate employability gaps, sets out the strategic measures (e.g. our new Attainment Policy) we will employ to remove barriers to student success, and presents the challenging targets we have set over 2020-25. Our reputation as a leading institution for social mobility in growing with three national award victories in 2020.

Evidence, evidence and more evidence…

The OfS is funding a new independent centre to help universities meet their APP targets to eliminate equality gaps in higher education within 20 years – the Centre for Transforming Access and Student Outcomes in Higher Education (TASO). Access and participation activity should be underpinned by a compelling evidence base, and we recognise the need to evaluate our own activities robustly to judge their continued effectiveness and contribute to the growing national evidence base.

The University’s Policy and Evaluation Unit (PERU) responded to this challenge by creating a new institutional APP Evaluation Framework, featuring evaluation principles (the why?), guidance and toolkits to support the development and application of evaluation techniques (the how?), and a “APP Knowledge Hub” to create opportunities for collaboration and a place where staff can share evaluation outcomes (the where?).

In doing so, the APP Evaluation Framework will help develop an evidence-informed culture at the University and contribute to “what works” in access and participation. It is underpinned by the OfS dimensions of evaluation model and evaluation self-assessment tools, and higher education sector evaluation networks e.g. NERUPI and TASO.

It builds on good practice – examples of embedded evidence-based approaches across our Centre for Student Life (CSL), Centre of Excellence in Learning and Teaching (CELT), College academic and research leads, and the Union of Students.

PERU is applying for ethical approval to conduct research into the efficacy of the APP Evaluation Framework in developing and sharing an effective evidence base, and its impact on the professional practice of our access and participation practitioners.

At the heart of our new approach is Theory of Change (ToC) – a process of reflection and planning that can be used to achieve our strategic aims. In developing the APP, the first stage was to create an institutional-level ToC pyramid model to present our overall access and participation goal, immediate outcomes, and the activities (strategic measures) that would help us to achieve our goal.

All the work we do at Inclusive Futures, with staff and students has a coaching philosophy at it’s core. As coaches, we do not give our story, we do not share our opinions, and we don’t give advice on how we or others have overcome obstacles or achieved outcomes. We wholeheartedly believe that our coachees are the experts on themselves and their situations, not us. They are not broken, and they do not need us to fix them or their situations.

The individuals we work with are resourceful and already have all the answers they need. Our job is to facilitate them to find the answers that feel right, drawing on the knowledge, strength and experience that they already have.

Research reviewed by TASO, along with wider studies have shown that coaching can enhance students’ problem-solving skills, coping skills, resilience, wellbeing, study skills and learning goals achievement[ii] which contributes to higher rates of engagement with higher education.

Coaching within the workplace can also have a huge impact on employees. A survey by the International Coaching Federation found that 80% of people who receive coaching report increased self-confidence, and over 70% benefit from improved work performance, relationships, and more effective communication skills[iii].

ToC model – development of access and participation strategic measures and activities

We chose a simple linear project ToC model and supplied a template to help staff to identify a situation or problem, their overall goals, and a set of measurable short- and medium-term outcomes needed to achieve the goals. Staff are asked to describe and justify activities (reflection on personal practice and independent published research) to achieve the outcomes, and identify who needs to participate in the activities for them to be successful, along with other the resource requirements.

The result is a comprehensive description of how and why a desired change is expected to happen in a specific context, informed by literature and stakeholders’ knowledge and experience. The ToC model allows us to monitor our progress, and to quickly share our evaluation findings through the APP Knowledge Hub. In applying this model, the University’s access and participation practice is being subjected to the levels of challenge and scrutiny expected by the OfS.

“It ain’t what you do, it’s the way that you do it!”

The University’s Widening Access team used the ToC model to develop their strategic plan for 2020/21, outreach delivery plans, and the identification of participant target groups and outcomes. It is helping the team to assess their progress and evaluate their provision. Through providing evidence for resource allocation/decision-making, the team has advanced their scholarship within their field through a resource that can be shared across the institution and the sector.

ToC allows us to focus on what really matters – what can be (reasonably) achieved, and on what basis those expectations can be justified. We can use it to evaluate our short- and medium-term outcomes and assess whether we are on the right track. If we are not, we return to the activities and evaluation to try to understand why. We adjust our practice and go again… and keep evaluating and reflecting on our practice until we get there!

Blog by Jo Astley - Evidence and Evaluation Manager, and Dan West is Policy and Research Lead for Social Mobility, in the Policy Research and Evaluation Unit (PERU) at the University of Derby.