What is Scale Data and Why do we use it?

This article is a collaboration between the program-level assessment team and the student course survey team. It is the first of a two-part series on scale data. This month, we will explain what scales are, why we use them, and how. Next month, we will discuss how to analyze rating scale data.

Measurement of physical phenomena (e.g., the number of times a specific question on a test is answered correctly) is relatively straightforward compared to the measurement of cognitive phenomena. These phenomena, which are of particular interest to us in education, are either not directly observable, e.g., a graduate's confidence in their ability to verbally communicate effectively, or observable but not directly quantifiable, e.g., an instructor's "Excellent "rating of a student's demonstration of the verbal communication. Due to these characteristics, collecting quantitative measures of these cognitive phenomena requires the development of rating scales and rubrics  (Harpe, 2015).

The Likert Scale is most likely a familiar data collection method. Likert scales are one of a group of aggregated measures, meaning that the underlying phenomena can be measured by aggregating an individual's responses to a set of at least three items related to the attribute or skill. In this type of scale, response categories are assigned a numerical value. 

An example of a Likert Scale is this excerpted version of Bray et al.'s 2020 instrument to measure students' confidence with critical skills, see Example 1. In their study, critical skills were collaboration, communication, creativity, self-direction, critical thinking, and technology. Participants were asked to rate their confidence relative to three items related to each of these six skills or attributes. Each item is scored using a (1 - 5) scale: Strongly disagree, disagree, don't know, agree, strongly agree.

Example 1

Likert Scale for Working with Others - Collaboration (Bray et al., 2020). 

Another common data collection method is Likert-type items. A Likert-type item is an individual item using any one of the various types of Likert response scales (Alkharusi, 2022; Harpe, 2015):

• Numerical: On a scale of 1 to 10, how helpful do you think video feedback was to your learning?

  [Not helpful at all-1] - 2 - 3 - 4 - [5-extremely helpful]

• Numeric and fully anchored: On a scale of 1 to 4, how helpful do you think video feedback was to your learning?

• Adjectival: How helpful do you think video feedback was to your learning?

  [Not helpful at all] - [A little helpful] - [Moderately helpful] - [Extremely helpful]

Now that you know a bit about the different types of rating scales, you might be wondering when and how to use them. Rating scales are used in surveys and rubrics. However, which sort of scale is best? Well, the answer depends on the question you are trying to answer with your data. If you are trying to gather data to measure an attribute or skill with numerous items, use a Likert Scale; however, if you want to gather data to measure a single item, use a Likert-type item. Each is analyzed differently, which should also be considered when deciding on the type of scale to use. For now, what is essential is that you choose the appropriate rating scale for your inquiry and that you ensure the validity and reliability of the measure through pilot testing and, as applicable, statistical testing - a topic that will be covered in next month's article on the analysis of scale data.

References:

Alkharusi, H. (2022). A descriptive analysis and interpretation of data from Likert scales in educational and psychological research. Indian Journal of Psychology and Education, 12(2), 13-16.

Bray, A., Byrne, P., & O'Kelly, M. (2020). A short instrument for measuring students' confidence with 'key skills' (sicks): Development, validation, and initial results. Thinking Skills and Creativity, 37, 100700. https://www.sciencedirect.com/science/article/pii/S1871187120301747

Harpe, S. E. (2015). How to analyze Likert and other rating scale data. Currents in pharmacy teaching and learning, 7(6), 836-850.

Vincent, J. (2022). Beyond measure: the hidden history of measurement. Faber & Faber.