Catalyst Awards Blog Series: Winning learning outcomes assessment

The Catalyst Awards program recognizes innovation and excellence in our global community of practice. In this blog series, a selection of 2022 award-winning institutions from across the globe share their success stories and best practices.

This post was guest authored by Norma Zavala Hernández, institutional assessment leader at the Institutional Effectiveness Office and professor at the Language Center, UPAEP, Mexico.

Universidad Popular Autónoma del Estado de Puebla (UPAEP) is a 2022 Catalyst Award winner in the “Assessment & Institutional Effectiveness” category.

We are pleased to have won the Catalyst award for using Anthology solutions to assess learning outcomes and would like to share what we have learned and gained through our experience.

Universidad Popular Autónoma del Estado de Puebla (UPAEP) is a private, nonprofit university located in Puebla, Mexico. UPAEP was founded in 1973 and was legally incorporated in 1976.  The university has two campuses: Puebla and Tehuacán, and has an academic offering of 161 programs, for undergraduate and graduate levels, in three educational modalities: grounded, semi-grounded, and online education.

• Puebla: 13,629 students
• Tehuacán: 452 students
• Total: 1,361 faculty members

Assessing student learning

Before 2018, UPAEP had no systematic approach to assessing student learning; the faculty used traditional methodologies to do this. In 2018, we started implementing a learning outcomes assessment approach to correct this situation. Given our enrollment numbers, this posed a great challenge. We faced the necessity to standardize and systematize our assessment process.

Implementing Anthology solutions

The initiative's goal was to create a structure and process of institutional scope to monitor the students' academic trajectory progressively. UPAEP wanted to have updated and readily available data to demonstrate the achievement of its institutional mission through fulfilling the institutional and program graduation profiles. However, while evolving, the project also incorporated two additional areas of assessment: General Education Learning Outcomes and Co-Curricular Learning Outcomes. Thus, the expected primary outcomes from the initiative were related to a meaningful, manageable, and sustainable learning outcomes assessment methodology.

Blackboard Learn was paramount in providing an architecture to host rubrics and connect with complementary software (like EAC). It was also essential for promoting the adoption of the institutional methodology for assessing student success since it was already our educational platform. Using a platform already in place and familiar to faculty and students eased our process. The accessibility of information and the location of rubrics were beneficial for faculty to start implementing the assessment system. Blackboard Learn also provided a solution by allowing the alignment of each rubric's criteria with the institutional or program goals. This was later replicated in the general education and co-curricular learning outcomes. This alignment was essential in providing relevant information for institutional effectiveness.

Key participants

Due to the initiative's scope, many areas of UPAEP were incorporated and coordinated by the Institutional Effectiveness Office to implement the assessment system:

• Faculty at all levels and educational modalities
• The general education task group
• The four co-curricular areas (the Culture and Leadership Office, Internship Office, Entrepreneurship Program, and International Affairs Office)
• The Writing Center
• The vice president for Academic Affairs
• The General Secretary
• The Information Technology department
• The learning outcomes committee

What we have achieved

Since 2018, the number of assessments conducted with Blackboard Learn has substantially increased, gradually incorporating the different educational modalities, campuses, and academic levels. Additionally, the periodicity of the assessment has increased. We are conducting assessments all year long, thus increasing the number of students and courses impacted by this innovative practice university-wide. We have assessed 17,386 students through 317 assessments of academic programs and 819 assessments of courses. For this to be possible, we have created and uploaded 310 different types of rubrics to the Blackboard Learn platform.

The number of faculty members involved in assessing and analyzing the assessment results continually rises. Full-time and part-time faculty alike are taking an active role in this practice, which is an outstanding achievement for institutional efficiency.

In addition, we are now able to assess the institutional and program graduation profiles and generate evidence and reliable data. Through the assessment of the Institutional Learning Outcome, the university can now track the students' trajectory, even at the individual level, to reflect, analyze, and adjust institutional practices to achieve the institutional mission.

Furthermore, the faculty of each academic program can now have reliable data for decision-making related to the student's professional graduation profile compared to the institutional goal. The Institutional Development Plan expresses that 80% of responses must be at the standard of each performance level in all rubrics.

Program assessment numbers:

• 73% responses at the standard
• 115 rubrics used
• 854 students assessed university-wide

Takeaways

Institutional learning has been a collateral benefit of the assessment initiative. The Institutional Effectiveness Office creates analytic reports for deeper reflection, new scenarios, and innovation. Statistical information is available through traditional means and also through dashboards, thanks to the compatibility between Blackboard Learn and UPAEP's business intelligence software. These dashboards graphically present the information and allow the tracking of institutional and educational objectives.

The practices that have contributed to our success are:

• Constant training
• An ongoing dialogue with the faculty
• A flexible structure that allows us to incorporate our findings into the process and adapt it

We have been able to foster a culture of evidence, assessment, and continuous improvement. Our path hasn't been easy, but it has been gratifying, and five years later, we are ready for the next step.

For more information on the program, visit Catalyst Awards. Learn more about our 2022 Catalyst Award winners and their stories.