Going for gold standard

Dr Jen Shearman, Director for Evaluation and Impact at the NCETM, discusses the importance and implications of the planned EEF-funded evaluations of two national projects in 2025/26.

I am delighted to finally confirm that the second decade of the Maths Hubs Network will begin with an Education Endowment Foundation (EEF) funded independent evaluation of two Network Collaborative Projects (NCPs). In the 2025/26 academic year there will be an evaluation of the Specialist Knowledge for Teaching Mathematics (Secondary Non-specialist Teachers) Programme, and, subject to final EEF Grants Committee confirmation, the Mastering Number at Reception and KS1 Programme. These programmes were chosen as their structures align both with evaluation methodologies preferred by the EEF, and national priority areas of Early Years and recruitment and retention of secondary maths teachers.

An important feature of these evaluations will be a Randomised Controlled Trial (RCT) which the EEF recommends as the optimal method for estimating whether something works, if done well (Nevill, 2019). An RCT works by comparing the outcomes of a group of trial participants to the outcomes of an identical group who were not involved. In the case of our projects, schools apply to the trial and evaluators randomly assign them to either an ‘intervention’ or ‘control’ group. Schools in the intervention group take part in the project, whilst schools in the control group do not. At the end of the academic year, pupils in both groups are assessed: if the pupils in the intervention group achieve higher assessment scores than the control group, then the programme is said to have a positive impact (Figure 1).

If the impact is large enough, the evaluators and the EEF will express its magnitude as ‘x months of additional progress’. Regardless of the outcome, the trial outcomes will be reported on the EEF website and will add to the growing evidence base of educational interventions.

Figure 1: The RCT process

Independent evaluations are necessary and desirable. I, along with my colleagues across the NCETM and the Maths Hubs Network, share the desire of the education community to understand the impact of publicly-funded projects on students. We of course will continue our own rigorous approaches to internal evaluation, and I am looking forward to working with evaluation experts to further investigate the impact of our activity.

I am not, however, assuming that administering these trials will be easy, and acknowledge three main challenges for our schools and the Maths Hubs.

1. Integral to an RCT is the ‘control group’. Half the schools that apply for a programme will not be able to receive it this academic year. The school will receive an incentive payment and compensation for the time spent undergoing assessments (and will, subject to DfE funding, have a guaranteed place on the programme in 2026/27), but we recognise no school applies for a trial hoping to be in the control group.
2. The NCETM has assigned each Maths Hub to be involved in either the SKTM Secondary Non-specialist Teachers or the Mastering Number at Reception and KS1 programme trial. Hubs will not be able to offer the choice of a ‘trial’ or ‘non-trial’ option to schools in their assigned programme. This is to ensure we can recruit sufficient schools. Since our projects are fully-funded (and thus ‘free’ to schools) a school would be very unlikely to sign up for a trial if they had the option to sign up for the regular programme. The Maths Hubs Network has built its success on developing positive and impactful relationships with schools in their community, and reducing choices for an academic year may make some relationships more complex to manage.
3. There will be administration and delivery challenges involved in running a trial, including additional paperwork and adjustments to the timing of some workshops so the project activity dovetails with evaluation activities. Two specific examples of adjustments include cancelling summer-start cohorts for the SKTM Secondary Non-specialist Teachers Programme in 2025/26, and a slightly-delayed launch of the Mastering Number at Reception and KS1 Programme for trial hubs.

I, along with the Evaluation and Impact strand and project leadership, am committed to working with evaluators, the EEF and colleagues across the NCETM and the Maths Hubs Network to mitigate these challenges.

With a nod to the medical origins of RCTs I will not forget that ‘being a student is not a disease, just as teaching is not a cure.’ (Biesta, 2007, p.8). We will work with the expert evaluators to minimise any disruptions to the programmes so our ‘business as usual’, rather than a ‘variation of’, is evaluated. The Maths Hub Lead Links for both projects have already been generous with their time and wisdom, and are advising how we can successfully balance recruitment with maintaining positive relationships with schools. More than half of schools in England have already participated in an EEF trial and we will continue to acknowledge the expertise, dedication and enthusiasm running throughout the network.

Despite the additional work I feel energised about the trials. The time is right to both silence ‘alarm bells’ related to a lack of evaluation (Schoolsweek, 2024) and to understand how best our activity helps all students, especially those from disadvantaged backgrounds, achieve a deep understanding of school mathematics.

References

Biesta, G. (2007), ‘WHY “WHAT WORKS” WON’T WORK: EVIDENCE-BASED PRACTICE AND THE DEMOCRATIC DEFICIT IN EDUCATIONAL RESEARCH’. Educational Theory, 57: 1-22. https://doi.org/10.1111/j.1741-5446.2006.00241.x (Accessed: February 2025)

Booth, S. (2024), ‘£700m spent on DfE’s hubs, but to what effect?’ Schoolsweek. Available at: https://schoolsweek.co.uk/700m-spent-on-dfes-hubs-but-to-what-effect/ (Accessed: February 2025)

Ly, B. (2020) ‘EEF: Using evidence to support school improvement’ Prospect. Available at: https://members.prospect.org.uk/news/id/2020/April/27/EEF-Using-evidence-support-school-improvement?_ts=16449#:~:text=One%20in%20two%20schools%20in%20England%20have%20now,which%20is%20where%20the%20research%20schools%20come%20in (Accessed: February 2025).

Neville, C. (2019), ‘Randomised controlled trials – 3 good things, 3 bad things, and 5 top tips.’ EEF Blog. Available at: https://educationendowmentfoundation.org.uk/news/eef-blog-randomised-controlled-trials-or-how-to-train-your-dragon (Accessed: February 2025)