Why is Quality Matters Important?

By now, you hopefully understand the purpose, history, and context surrounding Quality Matters rubrics and peer review process. Still, you may be wondering, how effective is the rubric in actual use cases? Luckily, the practical application of the QM Rubric has been observed by a growing group of researchers over the past decade, and the results are promising! Below is a summary of some of the most common findings regarding the practical application of the Rubric.

Finding 1: the QM rubric directly supports student success.

The QM Rubric directly supports a variety of online course features, such as alignment, engagement, learner support, transparency, and accessibility, that help students achieve success and meaningful learning experiences. 

Alignment

Quality Matters defines course alignment as the way that “critical course elements work together to ensure learners achieve the desired learning outcomes” (Quality Matters, 2018). Five of the eight Standards focus on the alignment of course materials and student learning outcomes, which is critical to student success (Joosten & Cusatis, 2019; Kauffman, 2015; Ni, Diomede, & Rutland, 2013). Stone (2021, p. 176) claims that “Students report feeling most engaged and connected with their teacher, other students and the learning content when their online course provides activities directly related to learning outcomes.” When a course is well-structured from a QM perspective, all roads lead to outcomes achievement.

Engagement and Active Learning

Many researchers have also found that learner engagement and active learning, addressed in Standards 5 and 6 of the QM Rubric, are essential qualities of an online course. Shelton, Hung, and Lowenthal (2017) found that more engaged online learners were at a lower risk of failing their courses when courses met these Standards, and Roddy, et al. (2017, p.4) emphasized that active engagement is a “core component of successful learning for students.” The Quality Matters Rubric attends to three different forms of engagement: learner-to-content engagement, learner-to-instructor engagement, and learner-to-learner engagement.

Learner Support

Standard 7 is dedicated entirely to ensuring that there is adequate support for students in an online learning environment. As Joosten and Cusatis (2019, p. 374) note, it is vital that students be provided with “easy access to technology, [academic] support, and accessibility options,” and that this support is offered within the course site. Roddy, et al. (2017, p.5) similarly state that provision of student support services is “paramount in intensive online environments, where disruptions to technology or lack of support services can pose a significant barrier to student engagement in learning.”

Transparency and Visibility

Another area of online course design which clearly separates the expert from the novice instructor is the provision of transparent course policies in prominent areas of the course (Kumar, et al., 2019). QM Rubric Standards 1, 2, 3, 5, and 7 encourage course design practices which make important course elements clear and visible to students. Standard 3, for instance, ensures that an online course is developed with transparent assessment practices, such as clear grading policies and specific, descriptive grading criteria. These practices are especially helpful in managing students’ expectations and supporting their learning (Joosten & Cusatis, 2019; Stone, 2021). Kumar, et al. (2019), who interviewed eight award-winning online faculty members from across the United States, discovered that these faculty prioritized the explanation of the purpose of their course activities and use of technology, which is covered in the Standards 2, 4, and 5 of the Rubric. 

Accessibility and Inclusivity

Research has also shown that courses which implement QM’s Standard 8 (Accessibility and Inclusivity) result in increased student success for all students, especially those who need accommodations or are generally underserved (Martin, Ndoye & Wilkins, 2016; Stone, 2021). Standard 8 guides course developers to integrate these accessible and inclusive practices into online course design.

Finding 2: students perceive courses developed with the QM Rubric more positively.

Students also care about online course quality! There is a growing body of research regarding student perceptions about their own academic success in online courses that have been developed according to QM principles (e.g., Alkramiti & Alsharidah, 2022;  Alizadeh, et al., 2019; Kumar, et al., 2019). For instance, Martin, Ndoye, and Wilkins (2016) found that Standard 5 of the Rubric, Learning Activities and Learner Interaction, was the most impactful standard for the students they surveyed. These students also reported that clear and measurable learning objectives (Standard 2) helped them engage with the course and “led to improved learning” overall (p. 228). The provision of course technology support (Standard 7) and well-developed instructional content (Standard 5) were also highly ranked. 

Barczyk, et al. (2017) found that the highest rated standards among the students they surveyed were Standard 3, (course assessment) and Standard 6 (course technology). Regarding Standard 6, students valued tools and media that supported learning outcomes, enhanced classmate interaction, provided clear instructions, and were easily downloadable. Furthermore, all students valued accessibility of instructional content (Standard 8), clearly stated learning outcomes (Standard 2), easy-to-understand grading policies (Standard 3), and assessment strategies that include provision of teacher feedback (Standard 3). Finally, Sadaf et al. (2019) found that most students rated each QM standard as influential for their ability to learn the required content in a course. However, the two QM standards rated as most important were Standard 5 (Course Activities and Learner Interaction) and Standard 2 (Learning Outcomes).

Finding 3: QM teacher trainings lead to improved academic achievement.

Last but certainly not least, instructional design support is an important determiner in the successful implementation of the QM standards. Instructors need proper training and support to achieve their potential as online educators, and students take notice when this potential is reached. Brown, Lewis, David, and Toussaint (2018), found that students perceived higher levels of course quality when faculty worked with instructional designers to develop online courses using a QM template. Similarly, Martin, Ndoye, and Wilkins (2016), who examined student perceptions of their own learning and engagement in QM-certified courses, found that “courses designed with the QM rubric resulted in improv[ed] student–content interaction” (p. 228).

QM professional development not only increases students’ positive perceptions of a course; it also promotes learner achievement. Hallowell et al. (2017), for example, created a learning community of ten science and mathematics faculty who participated in a QM professional development workshop series to aid in the redesign of their online courses. Their study describes the impact of QM training on faculty course design and the effect of QM-based course revisions on student outcomes. They found that QM review scores were significantly positively correlated with the final exam performance and overall course averages of their students. Harkness (2015), who tracked QM pre- and post-course design interventions at a Historically Black College University (HBCU), found similar results. For the courses that received QM interventions, there was a 19.7% increase in passing course grades, a 66.6% reduction in failing course grades, and a 23.5% reduction in course withdrawal.

Hopefully, these empirical studies will help answer the question of why we do Quality Matters here at UArizona. Online course quality is inherently linked to faculty support and student success, and Quality Matters provide a framework for both.

References

Alkramiti, A. M., & Alsharidah, M. A. (2022). Evaluating the Design of Mathematics Courses Available on the Blackboard Platform at Prince Sattam bin Abdulaziz University According to Quality Matters Standards. Eurasia Journal of Mathematics, Science and Technology Education, 18(4), 1-20. https://doi.org/10.29333/ejmste/11924  

Alizadeh, Mehran, P., Koguchi, I., & Takemura, H. (2019). Evaluating a blended course for Japanese learners of English: why Quality Matters. International Journal of Educational Technology in Higher Education, 16(1), 1–21. https://doi.org/10.1186/s41239-019-0137-2

Barczyk, Casimir C, Hixon, Emily, Buckenmeyer, Janet, & Ralston-Berg, Penny. (2017). The effect of age and employment on students' perceptions of online course quality. The American Journal of Distance Education, 31(3), 173–184. https://doi.org/10.1080/08923647.2017.1316151

Brown, Victoria S., Lewis, David, & Toussaint, Mario. (2018). Students’ perceptions of quality across four course development models. Online Learning (Newburyport, Mass.), 22(2), 173. https://doi.org/10.24059/olj.v22i2.1213

Harkness, S. (2015). How a historically black college university (HBCU) established a sustainable online learning program in partnership with Quality Matters. The American Journal of Distance Education, 29(3), 198-209. https://doi.org/10.1080/08923647.2015.1057440

Hollowell, G. P., Brooks, R. M., & Anderson, Y. B. (2017) Course design, Quality Matters training, and student outcomes. American Journal of Distance Education, 31(3), 207-216. https://doi.org/10.1080/08923647.2017.1301144

Joosten, T., & Cusatis, R. (2019). A cross-institutional study of instructional characteristics and student outcomes: Are quality indicators of online courses able to predict student success? Online Learning, 23(4), 354-378. https://doi:10.24059/olj.v23i4.1432  

Kauffman, H. (2015). A review of predictive factors of student success in and satisfaction with online learning. Research in Learning Technology, 23, 1-13. https://doi.org/10.3402/rlt.v23.26507

Kumar, Martin, F., Budhrani, K., & Ritzhaupt, A. (2019). Award-winning faculty online teaching practices: Elements of award-winning courses. Online Learning (Newburyport, Mass.), 23(4), 160-180. https://doi.org/10.24059/olj.v23i4.2077

Martin, F., Ndoye, A., & Wilkins, P. (2016). Using learning analytics to enhance student learning in online courses based on Quality Matters Standards. Journal of Educational Technology Systems, 45(2), 165–187. https://doi.org/10.1177/0047239516656369

Ni, X., Diomede, S., & Rutland, S. R. (2013). Effects of using the Quality Matters programme as an intervention for online education. International Journal of Social Media and Interactive Learning Environments, 1(1), 93–105. https://doi.org/10.1504/ijsmile.2013.051657  

Quality Matters (2018). The Quality Matters™ Higher Education Rubric Sixth Edition. MarylandOnline, Inc.

Roddy, C., Amiet, D. L., Chung, J., Holt, C., Shaw, L., McKenzie, S., ... & Mundy, M. E. (2017, November). Applying best practice online learning, teaching, and support to intensive online environments: an integrative review. Frontiers in Education (2). 1-10. https://doi.org/10.3389/feduc.2017.00059

Sadaf, Ayesha, Martin, Florence, & Ahlgrim-Delzell, Lynn. (2019). Student perceptions of the impact of “Quality Matters” certified online courses on their learning and engagement. Online Learning (Newburyport, Mass.), 23(4), 214-233. https://doi.org/10.24059/olj.v23i4.2009  

Stone, C. (2021). Improving student engagement, retention and success in online learning. In Shah, Kift, S., & Thomas, L. (Eds.) Student retention and success in higher education (pp.167-189). Springer International Publishing AG. https://doi-org/10.1007/978-3-030-80045-1_9

Shelton, B. E., Hung, J.-L., & Lowenthal, P. R. (2017). Predicting student success by modeling student interaction in asynchronous online courses. Distance Education, 38(1), 59–69. https://doi.org/10.1080/01587919.2017.1299562