Evaluating comprehension of written pharmacy materials: Preliminary experience with the Cloze procedure

Abstract

Objective: The objective of this research was to assess reading comprehension of a pharmacy-related educational pamphlet using the cloze procedure in subjects expected to exhibit a wide range of health literacy proficiency. Methods: The cloze procedure, a technique for assessing reading comprehension, was applied to an educational pamphlet describing safe medication practices written at the seventh grade level. Sixty subjects were recruited from a university community and affiliated adult literacy center. Consenting subjects were asked to complete a background interview, the Very Short Test of Functional Health Literacy in Adults (VS-TOFHLA) and the newly developed, pharmacy-relevant comprehension assessment. Subject performance on the VS-TOFHLA and newly developed pharmacy-relevant comprehension assessment were described and compared to assess reliability and validity. Results: A final sample of 52 subjects was analyzed. Mean age of the sample was 46.3 years (sd=12.9 years). Respondents were predominantly white (88%), female (83%) and college graduates (73%). The mean score on the VS-TOFHLA was 95.4% with all but one (2%) of the subjects demonstrating adequate functional health literacy. The mean score on the pharmacy-relevant comprehension assessment was 57.9%. Internal consistencies of the VS-TOFHLA and the pharmacy-relevant comprehension assessment were 0.95 and 0.91 respectively. Scores on the pharmacy-relevant comprehension assessment and the VS-TOFHLA were highly correlated (r=0.72, p<0.0001). Performance on the pharmacy-relevant comprehension assessment indicated approximately 57% of respondents with some college or less would need supplemental teaching about the pharmacy-relevant passage or would find it unsuitable for adequate understanding. Approximately 24% of the college graduates would need supplemental teaching. Conclusions: There is a lack of tools that can be used to assess patient comprehension of pharmacy-relevant information. This study indicates promise for using the cloze procedure for assessing patient comprehension of pharmacy-relevant educational materials.