Catalyst Awards Blog Series: High Impact Modules Project Review

The Catalyst Awards program recognizes innovation and excellence in our global community of practice. In this blog series, a selection of 2022 award-winning institutions from across the globe share their success stories and best practices.

This post was guest authored by Juan-Claude Lemmens, higher education researcher in the Department for Education Innovation at the University of Pretoria, South Africa.

University of Pretoria is a 2022 Catalyst Award winner in the "Student Success” category.

The High Impact Modules Project team at the University of Pretoria implemented effective course evaluation practices that resulted in improved student outcomes and improvement in the quality of teaching and learning. The purpose of this project was to provide a holistic review of a selection of courses with targeted interventions to increase the pass rates of the course. Blackboard® Learn data and qualitative course reviews were used to understand the specific issues that impact the performance of a course, namely in terms of curriculum, assessment, policies and practices, support services, communication, students, and lecturers.

The University of Pretoria is a large research-intensive public university with over 30,000 undergraduate students and about 13,000 postgraduate students. The university has nine faculties and four campuses. Academic programs include arts & humanities, education, science, engineering & technology (SET), business & economics, clinical & health, life & physical sciences, and social sciences. The university offers over 1,100 study programs, mostly bachelor’s degrees and postgraduate qualifications at master’s and doctoral levels. The delivery mode of undergraduate programs is contact mode, using a blended teaching method. The university has over 1,200 permanent teaching staff and almost 3,700 support staff.

High Impact Modules Reviews

The High Impact Modules (HIMs) Project is an initiative of the Tshebi Data Analytics Committee to improve the success rate of a selected number of gateway modules. The HIMs are courses with a pass rate below 75% and an enrollment of more than 500 students. In preparation for each module review, an analysis is done with Blackboard Learn data to understand the specific issues that impact the performance of the specific module, namely in terms of curriculum, assessment, policies and practices, support services, communication, students, and lecturers.

The process of High Impact Modules Reviews

• A task team was selected for the module reviews, consisting of faculty, heads of departments, course coordinators, and staff from the center for teaching and learning. A total of 20 modules were reviewed in 2019, 9 in the first semester and 11 in the second semester. Due to the impact of Covid-19, 24 modules were reviewed in 2020, all of them presented in the second semester. The process kicked off with several meetings with the various task teams. Identified during those meetings were the need for additional data analyses, the best time to administer the online course evaluation to the students, which Blackboard Learn data and reports were needed, as well as the necessity for dedicated support from learning technologists and learning designers.
• The approach in 2020 was to divide all undergraduate modules into three categories and to base the intervention(s) on a risk category of these courses. Based on the 2019 module success rates, 53 modules were identified as HIMs for review in 2020. However, as a result of Covid-19, an adapted approach was followed with a selection of second-semester modules only, and the team meeting task was moved online due to all staff working remotely.
• Blackboard Learn was used as the source of the dashboards displayed in Pyramid Analytics. Module review task team members had access to these dashboards as well as the analytics that were available in the Blackboard Learn Grade Center. In some cases, the faculty made use of the Blackboard Learn Retention Center to identify students that are at risk. Going forward, Blackboard Learn Retention Center will be piloted in more depth in some of the HIMs to determine its impact as an intervention.

The results

Module reviews play an important role in improving the quality of teaching and learning. This allows the faculty to address problems or gaps identified while students still have the opportunity to derive maximum benefit from the improvements made in the course. From a learning analytics perspective specifically, students’ interaction with Blackboard Learn can be analyzed and reveal student learning behavior, which can be used to adjust educational content and academic coursework accordingly. This transformation in pedagogical approaches can help in improving student learning and performance.

The overall improvement in module success rates of the 20 courses evaluated in 2019 was, on average, 13.3%. The overall improvement in module success rates of the 24 modules under review in 2020 was 12.6%. The overall improvement in module success rates for the institution was 1% when comparing the 2019 undergraduate module examination statistics to the 2018 statistics (from 82.5% in 2018 to 83.5% in 2019) and 4% when comparing the 2020 undergraduate module examination statistics to the 2019 statistics (from 83.8% in 2019 to 87.8% in 2020), showing that the reviewed modules made a meaningful contribution to the module success rates of the institution.

Lessons learned and recommendations

• UP found that for the HIMs project to be successful, it is important to get institutional buy-in from all stakeholders involved. It is also advantageous to make the project part of the strategic plan for T&L and student success.
• UP also found it beneficial to have a dedicated forum, such as the Tshebi Data-Analytics Committee, where the results can be shared and celebrated on a regular basis. UP could also leverage the committee to provide resources for change, especially where the institutional policies and practices impact the success rate of the module.
• It is also important to provide data that is actionable to faculty and that is directly related to student success in a module.
• During the task team meetings, it is important to create a caring atmosphere and safe space for module reviews — one could position the reviews as a part of the Scholarship of Teaching and Learning nested with a continual quality improvement cycle.
• It may also be of value to use a framework to conceptualize module reviews and to use the framework to develop a theory of change so that improvements can be transferred and duplicated across terms and modules in similar disciplines.

For more information on the program, visit Catalyst Awards. Learn more about our 2022 Catalyst Award winners and their stories.