a literature review validity

• UNC Chapel Hill

Critical Analysis of Reliability and Validity in Literature Reviews

Chetwynd, E.J.

Introduction

Literature reviews can take many forms depending on the field of specialty and the specific purpose of the review. The evidence base for lactation integrates research that cuts across multiple specialties (Dodgson, 2019) but the most common literature reviews accepted in the Journal of Human Lactation include scoping reviews, systematic reviews, and meta-analyses. Scoping reviews map out the literature in a particular topic area or answer a question about a particular concept or characteristic of the literature about a particular topic. They are broad, detailed, often focused on emerging evidence, and can be used to determine whether a more rigorous systematic review would be useful (Munn et al., 2018). To this end, a scoping review can draw from various sources of evidence, including expert opinion and policy documents, sometimes referred to as “grey literature” (Tricco, et al., 2018). A systematic review has a different purpose to a scoping review. According to the Cochrane Library (www. cochranelibrary.com), under the the section heading “What is a Systemic Review?” the following is stated: it will “identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a specific research question” https://www.cochranelibrary.com/about/ about-cochrane-reviews.). Meta-analysis takes the process of systematic review one step further by pooling the data collected and presenting aggregated summary results (Ahn & Kang, 2018). Each type of analysis or review requires a critical analysis of the methodologies used in the reviewed articles (Dodgson, 2021). In a scoping review, the results of the critical analysis are integrated and reported descriptively since they are designed to broadly encapsulate all of the research in a topic area and identify the current state of the science, rather than including only research that meets specific established quality guidelines (Munn et al., 2018). Systematic reviews and meta-analyses use critical analysis differently. In these types of reviews and analyses, the quality of research methods and study instruments becomes an inclusion criterion for deciding which studies to include for analysis so that their authors can ensure rigor in their processes (Page et al., 2021). Reliability and validity are research specific terms that may be applied throughout scientific literature to assess many elements of research methods, designs, and outcomes; however, here we are focusing specifically on their use for assessing measurement in quantitative research methodology. Specifically, we will be examining how they are used within literature review analyses to describe the nature of the instruments applied to measure study variables. Within this framework, reliability refers to the reproducibility of the study results, should the same measurement instruments be applied in different situations (Revelle & Condon, 2019). Validity tests the interpretation of study instruments and refers to whether they measure what they have been reported to be measuring, as supported by evidence and theory in the topic area of investigation (Clark & Watson, 2019). Reliability and validity can exist separately in a study; however, robust studies are both reliable and valid (Urbina & Monks, 2021). In order to establish a benchmark for determining the quality and rigor across all methodologies and reporting guidelines (Dodgson, 2019), the Journal of Human Lactation requires that the authors of any type of literature review include two summary tables. The first table illustrates the study design broadly, asking for study aims, a description of the sample, and the research design for each of the reviewed articles. The second required table is focused on measurement. It guides authors to list each study’s variables, the instruments used to measure each variable, and the reliability and validity of each of these study instrument (https://journals .sagepub.com/author-instructions/jhl#LiteratureReview; Simera et. al., 2010). The techniques used to describe the measurement reliability and validity are sometimes described explicitly using either statistical testing or other recognized forms of testing (Duckett, 2021). However, there are times when the methods for evaluating the measurement used have not been explicitly stated. This situation requires the authors of the review to have a clear understanding of reliability and validity in measurement to extrapolate the methods researchers may have used. Lactation is a topic area that incorporates many fields of specialty; therefore, this article will not be an exhaustive exploration of all types of tests for measurement of reliability and validity. The aim, instead, is to provide readers with enough information to feel confident about finding and assessing implicit types of measurement reliability and validity within published manuscripts. Additionally, readers will be better able to evaluate the usefulness of reviews and the instruments included in those reviews. To that end, this article will: (1) describe types of reliability and validity used in measurement; (2) demonstrate how realiability and validity might be implemented; and (3) discuss how to critically review reliability and validity in literature reviews.

Chetwynd EM, Wasser HM, Poole C. Breastfeeding Support Interventions by International Board Certified Lactation Consultants: A Systemic Review and Meta-Analysis. J Hum Lact. 2019 Aug;35(3):424-440. doi: 10.1177/0890334419851482. Epub 2019 Jun 17. PMID: 31206317.

Publication Link

Methodological Approaches to Literature Review

• Living reference work entry
• First Online: 09 May 2023
• Cite this living reference work entry

• Dennis Thomas 2 ,
• Elida Zairina 3 &
• Johnson George 4  

768 Accesses

1 Citations

The literature review can serve various functions in the contexts of education and research. It aids in identifying knowledge gaps, informing research methodology, and developing a theoretical framework during the planning stages of a research study or project, as well as reporting of review findings in the context of the existing literature. This chapter discusses the methodological approaches to conducting a literature review and offers an overview of different types of reviews. There are various types of reviews, including narrative reviews, scoping reviews, and systematic reviews with reporting strategies such as meta-analysis and meta-synthesis. Review authors should consider the scope of the literature review when selecting a type and method. Being focused is essential for a successful review; however, this must be balanced against the relevance of the review to a broad audience.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Similar content being viewed by others

Reviewing Literature for and as Research

Discussion and Conclusion

Systematic Reviews in Educational Research: Methodology, Perspectives and Application

Akobeng AK. Principles of evidence based medicine. Arch Dis Child. 2005;90(8):837–40.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Alharbi A, Stevenson M. Refining Boolean queries to identify relevant studies for systematic review updates. J Am Med Inform Assoc. 2020;27(11):1658–66.

Article   PubMed   PubMed Central   Google Scholar  

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Article   Google Scholar  

Aromataris E MZE. JBI manual for evidence synthesis. 2020.

Google Scholar  

Aromataris E, Pearson A. The systematic review: an overview. Am J Nurs. 2014;114(3):53–8.

Article   PubMed   Google Scholar  

Aromataris E, Riitano D. Constructing a search strategy and searching for evidence. A guide to the literature search for a systematic review. Am J Nurs. 2014;114(5):49–56.

Babineau J. Product review: covidence (systematic review software). J Canad Health Libr Assoc Canada. 2014;35(2):68–71.

Baker JD. The purpose, process, and methods of writing a literature review. AORN J. 2016;103(3):265–9.

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326.

Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017;6(1):1–12.

Brown D. A review of the PubMed PICO tool: using evidence-based practice in health education. Health Promot Pract. 2020;21(4):496–8.

Cargo M, Harris J, Pantoja T, et al. Cochrane qualitative and implementation methods group guidance series – paper 4: methods for assessing evidence on intervention implementation. J Clin Epidemiol. 2018;97:59–69.

Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126(5):376–80.

Article   CAS   PubMed   Google Scholar  

Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med. 1997;127(5):380–7.

Cummings SR, Browner WS, Hulley SB. Conceiving the research question and developing the study plan. In: Cummings SR, Browner WS, Hulley SB, editors. Designing Clinical Research: An Epidemiological Approach. 4th ed. Philadelphia (PA): P Lippincott Williams & Wilkins; 2007. p. 14–22.

Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. JMLA. 2018;106(4):420.

Ferrari R. Writing narrative style literature reviews. Medical Writing. 2015;24(4):230–5.

Flemming K, Booth A, Hannes K, Cargo M, Noyes J. Cochrane qualitative and implementation methods group guidance series – paper 6: reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses. J Clin Epidemiol. 2018;97:79–85.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108.

Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropr Med. 2006;5(3):101–17.

Gregory AT, Denniss AR. An introduction to writing narrative and systematic reviews; tasks, tips and traps for aspiring authors. Heart Lung Circ. 2018;27(7):893–8.

Harden A, Thomas J, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol. 2018;97:70–8.

Harris JL, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 2: methods for question formulation, searching, and protocol development for qualitative evidence synthesis. J Clin Epidemiol. 2018;97:39–48.

Higgins J, Thomas J. In: Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3, updated February 2022). Available from www.training.cochrane.org/handbook.: Cochrane; 2022.

International prospective register of systematic reviews (PROSPERO). Available from https://www.crd.york.ac.uk/prospero/ .

Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118–21.

Landhuis E. Scientific literature: information overload. Nature. 2016;535(7612):457–8.

Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI; 2020. Available from https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-03 .

Chapter   Google Scholar  

Lorenzetti DL, Topfer L-A, Dennett L, Clement F. Value of databases other than medline for rapid health technology assessments. Int J Technol Assess Health Care. 2014;30(2):173–8.

Moher D, Liberati A, Tetzlaff J, Altman DG, the PRISMA Group. Preferred reporting items for (SR) and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;6:264–9.

Mulrow CD. Systematic reviews: rationale for systematic reviews. BMJ. 1994;309(6954):597–9.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Munthe-Kaas HM, Glenton C, Booth A, Noyes J, Lewin S. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol. 2019;19(1):1–13.

Murphy CM. Writing an effective review article. J Med Toxicol. 2012;8(2):89–90.

NHMRC. Guidelines for guidelines: assessing risk of bias. Available at https://nhmrc.gov.au/guidelinesforguidelines/develop/assessing-risk-bias . Last published 29 August 2019. Accessed 29 Aug 2022.

Noyes J, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 1: introduction. J Clin Epidemiol. 2018b;97:35–8.

Noyes J, Booth A, Flemming K, et al. Cochrane qualitative and implementation methods group guidance series – paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol. 2018a;97:49–58.

Noyes J, Booth A, Moore G, Flemming K, Tunçalp Ö, Shakibazadeh E. Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods. BMJ Glob Health. 2019;4(Suppl 1):e000893.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Healthcare. 2015;13(3):141–6.

Polanin JR, Pigott TD, Espelage DL, Grotpeter JK. Best practice guidelines for abstract screening large-evidence systematic reviews and meta-analyses. Res Synth Methods. 2019;10(3):330–42.

Article   PubMed Central   Google Scholar  

Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7(1):1–7.

Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Brit Med J. 2017;358

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Br Med J. 2016;355

Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283(15):2008–12.

Tawfik GM, Dila KAS, Mohamed MYF, et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health. 2019;47(1):1–9.

The Critical Appraisal Program. Critical appraisal skills program. Available at https://casp-uk.net/ . 2022. Accessed 29 Aug 2022.

The University of Melbourne. Writing a literature review in Research Techniques 2022. Available at https://students.unimelb.edu.au/academic-skills/explore-our-resources/research-techniques/reviewing-the-literature . Accessed 29 Aug 2022.

The Writing Center University of Winconsin-Madison. Learn how to write a literature review in The Writer’s Handbook – Academic Professional Writing. 2022. Available at https://writing.wisc.edu/handbook/assignments/reviewofliterature/ . Accessed 29 Aug 2022.

Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999;18(20):2693–708.

Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16(1):15.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Yoneoka D, Henmi M. Clinical heterogeneity in random-effect meta-analysis: between-study boundary estimate problem. Stat Med. 2019;38(21):4131–45.

Yuan Y, Hunt RH. Systematic reviews: the good, the bad, and the ugly. Am J Gastroenterol. 2009;104(5):1086–92.

Download references

Author information

Authors and affiliations.

Centre of Excellence in Treatable Traits, College of Health, Medicine and Wellbeing, University of Newcastle, Hunter Medical Research Institute Asthma and Breathing Programme, Newcastle, NSW, Australia

Dennis Thomas

Department of Pharmacy Practice, Faculty of Pharmacy, Universitas Airlangga, Surabaya, Indonesia

Elida Zairina

Centre for Medicine Use and Safety, Monash Institute of Pharmaceutical Sciences, Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Parkville, VIC, Australia

Johnson George

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Johnson George .

Section Editor information

College of Pharmacy, Qatar University, Doha, Qatar

Derek Charles Stewart

Department of Pharmacy, University of Huddersfield, Huddersfield, United Kingdom

Zaheer-Ud-Din Babar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this entry

Cite this entry.

Thomas, D., Zairina, E., George, J. (2023). Methodological Approaches to Literature Review. In: Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy. Springer, Cham. https://doi.org/10.1007/978-3-030-50247-8_57-1

Download citation

DOI : https://doi.org/10.1007/978-3-030-50247-8_57-1

Received : 22 February 2023

Accepted : 22 February 2023

Published : 09 May 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-50247-8

Online ISBN : 978-3-030-50247-8

eBook Packages : Springer Reference Biomedicine and Life Sciences Reference Module Biomedical and Life Sciences

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Literature Review - what is a Literature Review, why it is important and how it is done

• Strategies to Find Sources

Evaluating Literature Reviews and Sources

Reading critically, tips to evaluate sources.

• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings
• Useful Resources

A good literature review evaluates a wide variety of sources (academic articles, scholarly books, government/NGO reports). It also evaluates literature reviews that study similar topics. This page offers you a list of resources and tips on how to evaluate the sources that you may use to write your review.

• A Closer Look at Evaluating Literature Reviews Excerpt from the book chapter, “Evaluating Introductions and Literature Reviews” in Fred Pyrczak’s Evaluating Research in Academic Journals: A Practical Guide to Realistic Evaluation , (Chapter 4 and 5). This PDF discusses and offers great advice on how to evaluate "Introductions" and "Literature Reviews" by listing questions and tips. First part focus on Introductions and in page 10 in the PDF, 37 in the text, it focus on "literature reviews".
• Tips for Evaluating Sources (Print vs. Internet Sources) Excellent page that will guide you on what to ask to determine if your source is a reliable one. Check the other topics in the guide: Evaluating Bibliographic Citations and Evaluation During Reading on the left side menu.

To be able to write a good Literature Review, you need to be able to read critically. Below are some tips that will help you evaluate the sources for your paper.

Reading critically (summary from How to Read Academic Texts Critically)

• Who is the author? What is his/her standing in the field.
• What is the author’s purpose? To offer advice, make practical suggestions, solve a specific problem, to critique or clarify?
• Note the experts in the field: are there specific names/labs that are frequently cited?
• Pay attention to methodology: is it sound? what testing procedures, subjects, materials were used?
• Note conflicting theories, methodologies and results. Are there any assumptions being made by most/some researchers?
• Theories: have they evolved overtime?
• Evaluate and synthesize the findings and conclusions. How does this study contribute to your project?

Useful links:

• How to Read a Paper (University of Waterloo, Canada) This is an excellent paper that teach you how to read an academic paper, how to determine if it is something to set aside, or something to read deeply. Good advice to organize your literature for the Literature Review or just reading for classes.

Criteria to evaluate sources:

• Authority : Who is the author? what is his/her credentials--what university he/she is affliliated? Is his/her area of expertise?
• Usefulness : How this source related to your topic? How current or relevant it is to your topic?
• Reliability : Does the information comes from a reliable, trusted source such as an academic journal?

Useful site - Critically Analyzing Information Sources (Cornell University Library)

• << Previous: Strategies to Find Sources
• Next: Tips for Writing Literature Reviews >>
• Last Updated: Jul 3, 2024 10:56 AM
• URL: https://lit.libguides.com/Literature-Review

The Library, Technological University of the Shannon: Midwest

Literature Review: Measures for Validity

According to Brown (2006) there are five criteria for the evaluation of the validity of literature review: purpose, scope, authority, audience and format. Accordingly, each of these criteria have been taken into account and appropriately addressed during the whole process of literature review.

McNabb (2008), on the other hand, formulates three fundamental purposes of literature review that are described below:

First, literature review shows the audience of the study that the author is familiar with the major contributions that have already been done to the research area by other authors. Second, literature helps to identify the key issues in the research area and obvious gaps in the current literature.

Third, the literature review assists the readers of the research in term of comprehending the principles and theories that have been used by the author in different parts of the study.

• Brown RB, 2006, Doing Your Dissertation in Business and Management: The Reality of Research and Writing, Sage Publications
• McNabb, DE, 2008, Research Methods in Public Administration and Non-Profit Management: Qualitative and Quantitative Approaches, 2 nd edition, ME Sharpe
• Wysocki, DK, 2007, Readings in Social Research Methods, Cengage Learning

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Critical Analysis of Reliability and Validity in Literature Reviews

Affiliation.

• 1 Department of Family Medicine, School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
• PMID: 35796304
• DOI: 10.1177/08903344221100201

Keywords: breastfeeding; lactation research; literature review; measurement; quantitative research; reliability; research methodology; validity.

PubMed Disclaimer

• Critical Analysis: The Often-Missing Step in Conducting Literature Review Research. Dodgson JE. Dodgson JE. J Hum Lact. 2021 Feb;37(1):27-32. doi: 10.1177/0890334420977815. J Hum Lact. 2021. PMID: 33630706 No abstract available.

Similar articles

• Quantitative Research Excellence: Study Design and Reliable and Valid Measurement of Variables. Duckett LJ. Duckett LJ. J Hum Lact. 2021 Aug;37(3):456-463. doi: 10.1177/08903344211019285. Epub 2021 Jun 9. J Hum Lact. 2021. PMID: 34107788 No abstract available.
• Beyond the BMI: Validity and Practicality of Postpartum Body Composition Assessment Methods during Lactation: A Scoping Review. Biddulph C, Holmes M, Kuballa A, Carter RJ, Maher J. Biddulph C, et al. Nutrients. 2022 May 25;14(11):2197. doi: 10.3390/nu14112197. Nutrients. 2022. PMID: 35683995 Free PMC article.
• Reliability and validity of a breastfeeding attitudes instrument for nursing staff. Dávila Torres RR, Parrilla AM, Gorrín Peralta J. Dávila Torres RR, et al. P R Health Sci J. 2000 Sep;19(3):263-7. P R Health Sci J. 2000. PMID: 11076373
• The validity and reliability of maternal recall of breastfeeding practice. Li R, Scanlon KS, Serdula MK. Li R, et al. Nutr Rev. 2005 Apr;63(4):103-10. doi: 10.1111/j.1753-4887.2005.tb00128.x. Nutr Rev. 2005. PMID: 15869124 Review.
• Breastfeeding studies: culture, biomedicine, and methodology. Beasley A. Beasley A. J Hum Lact. 1991 Mar;7(1):7-14. doi: 10.1177/089033449100700113. J Hum Lact. 1991. PMID: 2006977 Review.
• Healthy City Community Space-Oriented Structural Planning and Management Optimization under COVID-19. Yang Y, Jiang Z, Hou Y, Wang H, Wang Z. Yang Y, et al. Int J Environ Res Public Health. 2023 Feb 21;20(5):3863. doi: 10.3390/ijerph20053863. Int J Environ Res Public Health. 2023. PMID: 36900874 Free PMC article. Review.

Publication types

• Search in MeSH

LinkOut - more resources

Full text sources.

• Ovid Technologies, Inc.
• MedlinePlus Consumer Health Information
• MedlinePlus Health Information

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE: Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Sep 4, 2024 9:40 AM
• URL: https://libguides.usc.edu/writingguide

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

What is the purpose of literature review , a. habitat loss and species extinction: , b. range shifts and phenological changes: , c. ocean acidification and coral reefs: , d. adaptive strategies and conservation efforts: .

• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• How to write a literature review faster with Paperpal? 

Frequently asked questions 

What is a literature review .

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

1. Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 

2. Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field.

Find academic papers related to your research topic faster. Try Research on Paperpal

3. Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 

4. Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 

5. Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 

6. Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example 

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:  

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

Strengthen your literature review with factual insights. Try Research on Paperpal for free!

How to write a good literature review 

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Write and Cite as yo u go with Paperpal Research. Start now for free!

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review 

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:  

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:  

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:  

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:  

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:  

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:  

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

Whether you’re exploring a new research field or finding new angles to develop an existing topic, sifting through hundreds of papers can take more time than you have to spare. But what if you could find science-backed insights with verified citations in seconds? That’s the power of Paperpal’s new Research feature!  

How to write a literature review faster with Paperpal?  

Paperpal, an AI writing assistant, integrates powerful academic search capabilities within its writing platform. With the Research | Cite feature, you get 100% factual insights, with citations backed by 250M+ verified research articles, directly within your writing interface. It also allows you auto-cite references in 10,000+ styles and save relevant references in your Citation Library. By eliminating the need to switch tabs to find answers to all your research questions, Paperpal saves time and helps you stay focused on your writing.   

Here’s how to use the Research feature:  

• Ask a question: Get started with a new document on paperpal.com. Click on the “Research | Cite” feature and type your question in plain English. Paperpal will scour over 250 million research articles, including conference papers and preprints, to provide you with accurate insights and citations. 

• Review and Save: Paperpal summarizes the information, while citing sources and listing relevant reads. You can quickly scan the results to identify relevant references and save these directly to your built-in citations library for later access. 
• Cite with Confidence: Paperpal makes it easy to incorporate relevant citations and references in 10,000+ styles into your writing, ensuring your arguments are well-supported by credible sources. This translates to a polished, well-researched literature review. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a good literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. By combining effortless research with an easy citation process, Paperpal Research streamlines the literature review process and empowers you to write faster and with more confidence. Try Paperpal Research now and see for yourself.  

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is a comprehensive AI writing toolkit that helps students and researchers achieve 2x the writing in half the time. It leverages 22+ years of STM experience and insights from millions of research articles to provide in-depth academic writing, language editing, and submission readiness support to help you write better, faster.  

Get accurate academic translations, rewriting support, grammar checks, vocabulary suggestions, and generative AI assistance that delivers human precision at machine speed. Try for free or upgrade to Paperpal Prime starting at US$19 a month to access premium features, including consistency, plagiarism, and 30+ submission readiness checks to help you succeed.  

Experience the future of academic writing – Sign up to Paperpal and start writing for free!  

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• How Long Should a Chapter Be?
• How to Use Paperpal to Generate Emails & Cover Letters?

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, academic integrity vs academic dishonesty: types & examples, dissertation printing and binding | types & comparison , what is a dissertation preface definition and examples , the ai revolution: authors’ role in upholding academic..., the future of academia: how ai tools are..., how to write a research proposal: (with examples..., how to write your research paper in apa..., how to choose a dissertation topic, how to write a phd research proposal, how to write an academic paragraph (step-by-step guide).

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

Reliability vs. Validity in Research | Difference, Types and Examples

Published on July 3, 2019 by Fiona Middleton . Revised on June 22, 2023.

Reliability and validity are concepts used to evaluate the quality of research. They indicate how well a method , technique. or test measures something. Reliability is about the consistency of a measure, and validity is about the accuracy of a measure.opt

It’s important to consider reliability and validity when you are creating your research design , planning your methods, and writing up your results, especially in quantitative research . Failing to do so can lead to several types of research bias and seriously affect your work.

Table of contents

Understanding reliability vs validity, how are reliability and validity assessed, how to ensure validity and reliability in your research, where to write about reliability and validity in a thesis, other interesting articles.

Reliability and validity are closely related, but they mean different things. A measurement can be reliable without being valid. However, if a measurement is valid, it is usually also reliable.

What is reliability?

Reliability refers to how consistently a method measures something. If the same result can be consistently achieved by using the same methods under the same circumstances, the measurement is considered reliable.

What is validity?

Validity refers to how accurately a method measures what it is intended to measure. If research has high validity, that means it produces results that correspond to real properties, characteristics, and variations in the physical or social world.

High reliability is one indicator that a measurement is valid. If a method is not reliable, it probably isn’t valid.

If the thermometer shows different temperatures each time, even though you have carefully controlled conditions to ensure the sample’s temperature stays the same, the thermometer is probably malfunctioning, and therefore its measurements are not valid.

However, reliability on its own is not enough to ensure validity. Even if a test is reliable, it may not accurately reflect the real situation.

Validity is harder to assess than reliability, but it is even more important. To obtain useful results, the methods you use to collect data must be valid: the research must be measuring what it claims to measure. This ensures that your discussion of the data and the conclusions you draw are also valid.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Reliability can be estimated by comparing different versions of the same measurement. Validity is harder to assess, but it can be estimated by comparing the results to other relevant data or theory. Methods of estimating reliability and validity are usually split up into different types.

Types of reliability

Different types of reliability can be estimated through various statistical methods.

Types of validity

The validity of a measurement can be estimated based on three main types of evidence. Each type can be evaluated through expert judgement or statistical methods.

To assess the validity of a cause-and-effect relationship, you also need to consider internal validity (the design of the experiment ) and external validity (the generalizability of the results).

The reliability and validity of your results depends on creating a strong research design , choosing appropriate methods and samples, and conducting the research carefully and consistently.

Ensuring validity

If you use scores or ratings to measure variations in something (such as psychological traits, levels of ability or physical properties), it’s important that your results reflect the real variations as accurately as possible. Validity should be considered in the very earliest stages of your research, when you decide how you will collect your data.

• Choose appropriate methods of measurement

Ensure that your method and measurement technique are high quality and targeted to measure exactly what you want to know. They should be thoroughly researched and based on existing knowledge.

For example, to collect data on a personality trait, you could use a standardized questionnaire that is considered reliable and valid. If you develop your own questionnaire, it should be based on established theory or findings of previous studies, and the questions should be carefully and precisely worded.

• Use appropriate sampling methods to select your subjects

To produce valid and generalizable results, clearly define the population you are researching (e.g., people from a specific age range, geographical location, or profession).  Ensure that you have enough participants and that they are representative of the population. Failing to do so can lead to sampling bias and selection bias .

Ensuring reliability

Reliability should be considered throughout the data collection process. When you use a tool or technique to collect data, it’s important that the results are precise, stable, and reproducible .

• Apply your methods consistently

Plan your method carefully to make sure you carry out the same steps in the same way for each measurement. This is especially important if multiple researchers are involved.

For example, if you are conducting interviews or observations , clearly define how specific behaviors or responses will be counted, and make sure questions are phrased the same way each time. Failing to do so can lead to errors such as omitted variable bias or information bias .

• Standardize the conditions of your research

When you collect your data, keep the circumstances as consistent as possible to reduce the influence of external factors that might create variation in the results.

For example, in an experimental setup, make sure all participants are given the same information and tested under the same conditions, preferably in a properly randomized setting. Failing to do so can lead to a placebo effect , Hawthorne effect , or other demand characteristics . If participants can guess the aims or objectives of a study, they may attempt to act in more socially desirable ways.

It’s appropriate to discuss reliability and validity in various sections of your thesis or dissertation or research paper . Showing that you have taken them into account in planning your research and interpreting the results makes your work more credible and trustworthy.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Middleton, F. (2023, June 22). Reliability vs. Validity in Research | Difference, Types and Examples. Scribbr. Retrieved September 12, 2024, from https://www.scribbr.com/methodology/reliability-vs-validity/

Is this article helpful?

Fiona Middleton

Other students also liked, what is quantitative research | definition, uses & methods, data collection | definition, methods & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

A Literature Review of Indoor Wayfinding in Virtual Environment: Comparability to Real Environment and Ecological Validity

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, index terms.

General and reference

Document types

General literature

Recommendations

The effects of immersion and body-based rotation on learning multi-level indoor virtual environments.

The goal of this study was to investigate how the immersion level of virtual environments (HMD vs. desktop) and rotation method (physical vs. imagined) affects wayfinding performance in multi-story virtual buildings and the development of multi-level ...

FindMyWay: Toward Developing an Accessible Wayfinding Application for Visually Impaired in an Indoor Space

Due to the limitations of satellite-based navigation systems, such as GPS, indoor wayfinding has been very challenging for people with disabilities. Unfamiliarity and infrastructure complexity in an indoor space could put an additional burden on ...

Stepping off a ledge in an HMD-based immersive virtual environment

We explore whether a gender-matched, calibrated self-avatar affects the perception of the affordance of stepping off of a ledge, or visual cliff, in an immersive virtual environment. Visual cliffs form demonstrations in many immersive virtual ...

Information

Published in.

DFKI, Saarbrücken, Germany

Saarland University, Saarbrücken, Germany

UTS, Sydney, Australia

University of Trier, Trier, Germany

Université Paris-Saclay, Orsay, France

Dalhousie University, Halifax, Canada

• SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques
• SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Association for Computing Machinery

New York, NY, United States

Publication History

Check for updates, author tags.

• Ecological Validity
• Indoor Wayfinding
• Sensory Perception in Navigation
• Virtual Reality Wayfinding
• Refereed limited

Acceptance Rates

Upcoming conference, contributors, other metrics, bibliometrics, article metrics.

• 0 Total Citations
• 0 Total Downloads
• Downloads (Last 12 months) 0
• Downloads (Last 6 weeks) 0

View Options

Login options.

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

View options, share this publication link.

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

Service update: Some parts of the Library’s website will be down for maintenance on August 11.

Secondary menu

• Log in to your Library account
• Hours and Maps
• Connect from Off Campus
• UC Berkeley Home

Search form

Psychology 194: honors seminar: the literature review.

• Finding Articles
• The Literature Review
• Citations & Bibliographic Software (Zotero)
• Doing Original Research

Quick Links

• Google Scholar This link opens in a new window Search across many disciplines and sources including articles, theses, books, abstracts and court opinions, from academic publishers, professional societies, online repositories, universities and other web sites. more... less... Lists journal articles, books, preprints, and technical reports in many subject areas (though more specialized article databases may cover any given field more completely). Can be used with "Get it at UC" to access the full text of many articles.

• UC Library Search Start here to search for articles, books and more, in the collections of the University of California

What is a Literature Review?

A literature review is a survey of research on a given topic. It allows you see what has already been written on a topic so that you can draw on that research in your own study. By seeing what has already been written on a topic you will also know how to distinguish your research and engage in an original area of inquiry.

Why do a Literature Review?

A literature review helps you explore the research that has come before you, to see how your research question has (or has not) already been addressed.

You will identify:

• core research in the field
• experts in the subject area
• methodology you may want to use (or avoid)
• gaps in knowledge -- or where your research would fit in

Elements of a Successful Literature Review

According to Byrne's  What makes a successful literature review? you should follow these steps:

• Identify appropriate search terms.
• Search appropriate databases to identify articles on your topic.
• Identify key publications in your area.
• Search the web to identify relevant grey literature. (Grey literature is often found in the public sector and is not traditionally published like academic literature. It is often produced by research organizations.)
• Scan article abstracts and summaries before reading the piece in full.
• Read the relevant articles and take notes.
• Organize by theme.
• Write your review .

from Byrne, D. (2017). What makes a successful literature review?. Project Planner . 10.4135/9781526408518. (via SAGE Research Methods )

Research help

Email : Email your research questions to the Library.

Appointments : Schedule a 30-minute research meeting with a librarian. 

Find a subject librarian : Find a library expert in your specific field of study.

Research guides on your topic : Learn more about resources for your topic or subject.

• << Previous: Finding Articles
• Next: Citations & Bibliographic Software (Zotero) >>
• Last Updated: Sep 13, 2024 7:17 PM
• URL: https://guides.lib.berkeley.edu/Psych194-HonorsSeminar

• Open access
• Published: 14 September 2024

Mapping the extent of the literature and psychometric properties for the Physical Activity Scale for the Elderly (PASE) in community-dwelling older adults: a scoping review

• Cassandra D’Amore 1   na1 ,
• Lexie Lajambe 1   na1 ,
• Noah Bush 1 ,
• Sydney Hiltz 1 ,
• Justin Laforest 1 ,
• Isabella Viel 1 ,
• Qiukui Hao 1 &
• Marla Beauchamp 1  

BMC Geriatrics volume  24 , Article number:  761 ( 2024 ) Cite this article

Metrics details

Identifying valid and accessible tools for monitoring and improving physical activity levels is essential for promoting functional ability and healthy aging. The Physical Activity Scale for the Elderly (PASE) is a commonly used and recommended self-report measure of physical activity in older adults. The objective of this scoping review was to map the nature and extent to which the PASE has been used in the literature on community-dwelling older adults, including the evidence for its psychometric properties.

Seven electronic databases (MEDLINE (Ovid), Embase (Ovid), AMED (Ovid), Emcare (Ovid), CINAHL (EBSCO), Ageline (EBSCO)) were searched from inception to January 25, 2023. Studies were included if physical activity was part of the aim(s) and measured using the PASE, participants had a mean age of 60 years or older and lived in the community, and papers were peer-reviewed journal articles published in English. Pairs of independent reviewers screened abstracts, full-texts, and extracted data. Where possible, weighted mean PASE scores were calculated for different subgroups based on age, sex, and clinical population.

From 4,124 studies screened, 232 articles from 35 countries met the inclusion criteria. Most studies were cross-sectional (60.78%), completed in high-income countries (86.4%) and in North America (49.57%). A variety of clinical conditions were included ( n  = 21), with the most common populations being osteoarthritis ( n  = 13), Parkinson’s disease ( n  = 11), and cognitive impairment ( n  = 7). Psychometric properties of ten versions of the PASE were found. All versions demonstrated acceptable test-retest reliability. Evidence for construct validity showed moderate correlations with self-reported physical activity, fair to moderate with accelerometry derived activity and fair relationships with physical function and self-reported health. Pooled means were reported in graphs and forest plots for males, females, age groups, and several clinical populations.

The PASE was widely used in a variety of clinical populations and geographical locations. The PASE has been culturally adapted to several populations and evaluated for its reliability and convergent validity; however, further research is required to examine responsiveness and predictive validity. Researchers can use the weighted mean PASE scores presented in this study to help interpret PASE scores in similar populations.

OSF registration

osf.io/7bvhx

Peer Review reports

A pressing issue in the current healthcare system is the growing burden of chronic disease and multimorbidity associated with the world’s aging population [ 1 , 2 ]. There is an increasing number of older adults who require home care or housing options to support additional needs, including retirement homes, assisted living, or long-term care facilities [ 1 ]. Maintaining functional ability in later adulthood is a key public health priority and the promotion of physical activity (PA) is a central strategy for healthy aging initiatives [ 3 ]. Regular participation in PA has been shown to improve physical function, reduce impairments, promote independent living, and improve quality of life in older adults [ 4 ]. Physical activity can assist in maintaining cardiovascular, metabolic, and cognitive function; all of which reduce the risk of multimorbidity [ 5 , 6 , 7 ].

The World Health Organization (WHO) defines PA as “any bodily movement produced by skeletal muscles that requires energy expenditure” [ 8 ]. A growing body of evidence has demonstrated the importance of overall activity levels, including lighter intensity activities [ 9 ]. In addition to recommendations for moderate to vigorous activities, PA guidelines encourage changes in time allocation from sitting activities to light intensity activities, including standing [ 8 , 10 ]. Given the inclinations for lighter intensity activities in older ages (e.g., walking, gardening), clinicians and researchers must have tools to accurately assess and monitor the full spectrum of physical activities in this population.

Direct measures of PA (e.g., pedometers, accelerometers, and the gold standard of the doubly labelled water method) [ 11 ] can capture the full spectrum of activities. However, these measures can be more expensive, rely on equipment availability, and place a greater burden on participants [ 5 ]. Alternatively, self-report measures can be a low-cost, feasible tool for assessing and monitoring activity levels [ 12 ]. While not all questionnaires capture the same breadth of activities, the Physical Activity Scale for the Elderly (PASE) has been recommended for use in older adults for its inclusion of lighter intensity activities [ 5 ]. The PASE was designed to consider a greater number of activity domains more representative of the typical activities undertaken by older adults (e.g., gardening and household tasks) [ 13 ]. The questionnaire was developed for older adults (≥ 65), takes approximately 10 min to complete (10 questions), and asks participants to recall their activity over the last 7-days [ 13 , 14 ]. Activity types include sitting, walking, sport/recreation, exercise, occupational, and household [ 13 ]. A total score for PA can be calculated using these answers and the predetermined weights associated with each activity [ 13 ]. The PASE has been described as a suitable PA outcome measure for older adults who have multiple chronic conditions and is a recommended for measuring total PA in older adults based on evidence for its reliability and validity compared to other questionnaires [ 12 ].

To date, there has not been a comprehensive review of the populations and settings in which the PASE has been used. Rather, the literature on the PASE has focused on comparing the psychometric properties of multiple self-report measures of PA for specific populations. For example, Sattler et al. (2020) explored PA measures in healthy older adults and Garnett et al. (2019) in community-dwelling older adults with multiple chronic conditions. As part of their syntheses of all self-report PA measures both included a summary on the PASE, of ten and seven studies respectively [ 5 , 12 ]. As both these reviews recommend the use of the PASE, a more thorough exploration of the PASE with broader criteria is warranted. Further, the extent of the literature on its psychometric properties has not been thoroughly investigated. Therefore, the purpose of this scoping review was to map the nature and extent of the literature on the PASE in older populations (mean age 60) and to consolidate knowledge about the characteristics of studies using the PASE as an outcome measure, including available data on its psychometric properties. Our research questions were as follows:

To what extent has the PASE been used in older populations (e.g., number of studies, PASE administration, outcome operationalization from the PASE)?

What are the characteristics of studies that have used the PASE as an outcome measure (e.g., locations, sample characteristics, study designs)?

What is the nature and extent of the literature on the psychometric properties of the PASE in older populations (e.g., reliability, validity, cultural translation)?

The JBI guidelines for scoping reviews were followed in addition to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) guidelines (checklist available in Additional file 1 Table A1) [ 15 , 16 ]. This review protocol was registered with Open Science Framework ( https://doi.org/10.17605/OSF.IO/7BVHX ).

Search strategy

A broad search strategy was created with the assistance of a research librarian at the Health Sciences Library at McMaster University using the following key terms: “Physical Activity Scale for the Elderly”, “PASE”, “physical activity profile”, and “older”. Unique search strategies were developed for the following electronic databases: MEDLINE (Ovid), Embase (Ovid), Allied and Complementary Medicine Database (AMED; Ovid), Emcare (Ovid), CINAHL (EBSCO), Ageline (EBSCO). Databases were searched from inception to January 25 th , 2023. Backward citation searching was performed in Web of Science (Clarivate) for the original PASE article by Washburn and colleagues [ 13 ]. The complete search strategy for all databases is available in Additional file 1 Table A2. Reference lists of relevant systematic reviews, meta-analyses, and scoping reviews were screened and hand searched for additional articles.

Inclusion/exclusion criteria

To be included in this review studies must have populations consisting of older adults with a mean age greater than or equal to 60 years in line with the United Nations definition of older adults [ 17 ]. No restrictions were placed on sex, race or cultural background.

The overarching concept for this scoping review was the PASE; this included the original version and translated versions. Therefore, to be included studies must have incorporated PA in their aims and present results from the administration of the PASE. This criterion was further refined to specify that PASE must be included as a primary or secondary outcome (i.e., not just a covariate). The outcomes of interest to this review were the characteristics of the studies (e.g., cross-sectional vs prospective) and populations the PASE was used in (e.g., country, clinical populations, sex), mean total scores of the PASE, how the PASE was used (e.g., to look at relationships with PA, to determine intervention efficacy), as well as psychometric properties that have been evaluated.

Studies from any geographic location were included. After initial full-text screening the inclusion criteria was further refined to improve heterogeneity of included studies and ensure feasibility of the project due to the large number of results. The setting was restricted to designated community-dwelling populations which reflects the original context the PASE was designed in [ 13 ].

Studies were excluded if they were not written in English or if they were conference abstracts, presentations, systematic reviews, meta-analyses, scoping reviews, evidence maps, rapid reviews, literature reviews, narrative reviews, or critical reviews. Reviews were flagged and screened for additional citations.

Study selection

Results from the comprehensive literature search were organized in Endnote 20 (Clarivate, Philadelphia, USA) and uploaded to Covidence systematic review software (Veritas Health Innovation, Melbourne, Australia) for screening. Duplicated studies were removed using both programs prior to screening and any remaining were removed by hand. Prior to each phase of screening the reviewer team conducted pilot screening to improve agreement. For title and abstracts screening and full-text eligibility two independent reviewers (NB, LL, JL, IV, SH, and CD) confirmed the predetermined eligibility criteria. Due to the volume of full-text screening authors were not contacted for further details; where information for a given eligibility criteria was not reported or unclear the paper was excluded. Any disagreements during the abstract or the full-text review process were resolved by either consensus or arbitration by a third reviewer when necessary.

Data extraction and analysis

Data was extracted from the studies verbatim by two or more independent reviewers (NB, LL, JL, IV, SH, and CD). Modifications to the initial data extraction table made during the piloting process included the removal of details not necessary in a scoping review (e.g., funding sources, conflicts of interest) and the aims of this study (e.g., setting, recruitment methods). Additionally, separate columns were added to distinguish values calculated or extrapolated by reviewers versus authors (e.g., mean PASE scores, income classification). The following descriptive data was extracted: study details (geographical location, outcome measures, study design), population description (number of participants, mean age, sex, clinical population), PASE version and administration method, how the PASE was reported (e.g., mean vs categorical, subcategories vs full questionnaire), and psychometric properties reported.

Data was summarized in a descriptive manner through counts and percentages in tabular presentation. Weighted means and variances were calculated for total PASE scores across identified subgroups (sex, age, and clinical populations) where appropriate using the ‘metamean’ package in RStudio Team (R version 4.2.2, 2020, PBC, Boston, MA). In studies that reported only subgroup mean total PASE score or age, the authors combined the subgroup data using methods recommended in the Cochrane handbook [ 18 ]. Where possible, studies that provided median scores were converted to mean scores using the methodology developed by Wan et al. [ 19 ]. Studies that did not provide sufficient information for either transformation were omitted from some review syntheses. Studies were grouped by income based on the World Bank ratings from 2023 [ 20 ].

The database search produced 6,372 articles and hand searching citations produced another 24 articles for a total of 6,396. A total of 886 studies were assessed for full-text eligibility and 536 articles were found to use the PASE in older adults, 232 of which met all inclusion criteria (i.e., community-dwelling and the PASE was a primary/secondary outcome). An overview of the screening process can be found in PRISMA-ScR flow diagram (Fig.  1 ), and reasons for full-text study exclusions can be found in Additional file 2 Table A2.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram. Searches run on January 25 th , 2023

Summary of PASE use

The PASE was used for a variety of reasons with the most common being to explore the effect of PA on a health outcome(s) (e.g., an association of PA type with all-cause mortality) [ 21 ], and the relationship of a determinant with PA (e.g., the association between walkability and walking time) [ 22 ]. Almost all the studies used the PASE in its entirety (96.55%). The studies that used partial aspects of PASE often focused on leisure time PA (e.g., walking, sport/recreation, and exercise) [ 23 , 24 , 25 ], and two studies focused on walking exclusively [ 26 , 27 ]. Most authors (93.97%) used total PASE scores (i.e., used provided activity weights). Nineteen studies (8.19%) included a measure other than central tendency for total PASE score (e.g., dichotomous, tertiles, quartiles, quintiles). Eleven studies did not use the PASE score but instead operationalized PA using different pieces of the PASE (e.g., frequency, time). Details on the use of PASE are summarized in Table  1 .

The PASE was primarily delivered in person (69.40%) followed by mail (11.21%); 45 studies were either unclear or did not report how the PASE was administered to participants. A total of 15 different versions or languages were reported; the most common version used was English (63.79%). Six studies did not report which version or language the PASE was delivered in. In many cases, only the seminal paper on the English version by Washburn et al. was cited, with no further clarification of the version or modifications made, including several papers from countries where the primary language is not English ( n  = 29).

Study characteristics

A summary of the study characteristics can be found in Table  2 . The PASE was used throughout the world; however, nearly half of the studies were completed in North America (49.57%). In total, studies from 35 different countries were included in this review; the most common countries outside of North America included China ( n  = 20), Australia ( n  = 19), and Japan ( n  = 10). Most studies were conducted in high-income countries (86.64%). The mean age for studies ranged from 60.00 [ 28 ] to 84.40 [ 29 ] with the majority (43.10%) falling between 70–74 years old. Most studies included mixed sex samples ( n  = 184), with only 17 looking at females and 22 at males. Fifty-three studies looked specifically at 21 clinical conditions (e.g., musculoskeletal, cognitive impairment, and cardiorespiratory). The 232 studies of community-dwelling older adults included 171,206 participants, with individual study samples ranging from 8 [ 30 ] to 14,881 [ 31 ]. Studies were published between 1993 [ 13 ] and 2023 [ 32 , 33 , 34 , 35 , 36 ]. The PASE was used in a variety of study designs, including cross-sectional studies (60.78%), prospective studies (25.43%), and experimental (12.07%).

Where possible, weighted means for different subgroups were summarised based on age, sex, and clinical population. Studies with a mean age between 60–64 years had the highest mean PASE scores (159.53 (95% CI 146.58, 172.49)) and studies with a mean age over 80 years old had the lowest mean PASE scores (67.17 (95% CI 51.95, 82.39)) (Fig.  2 , Forest plots available in Additional file 1 Figure B1-B5). Figure  3 presents forest plots for the combined total mean PASE score for female only studies ( n  = 13) 123.99 (95% CI 108.09, 139.88) [ 26 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 ] and male only studies ( n  = 14) 136.27 (95% CI 122.46, 150.09) [ 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 ]. Based on data availability, pooled means were created for the following clinical populations: cancer ( n  = 2) [ 28 , 66 ], Chronic Obstructive Pulmonary Disease (COPD) ( n  = 2) [ 67 , 68 ], cognitive impairment ( n  = 6) [ 33 , 69 , 70 , 71 , 72 , 73 ], Diabetes ( n  = 3) [ 74 , 75 , 76 ], Osteoarthritis ( n  = 12) [ 46 , 77 , 78 , 79 , 80 , 81 , 82 , 83 , 84 , 85 , 86 , 87 ], and Parkinson’s disease (PD) ( n  = 10) [ 88 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 ]. Forest plots for clinical populations are available in Additional file 1 Figure B6.

Pooled Mean PASE scores by age groups

Pooled Mean PASE score forest plots for females(1) and males(2)

Psychometric properties of the PASE

Several papers evaluated the psychometric properties of the original PASE ( n  = 5) along with a number of validation studies ( n  = 14) for different translations and clinical populations (acute coronary event [ 98 ], COPD [ 68 ], Cancer [ 28 , 66 ], and Parkinson’s disease [ 89 ]). In total, ten different versions of the PASE were assessed for reliability and/or validity in community-dwelling older adults, including: English ( n  = 5) [ 13 , 14 , 66 , 98 , 99 ], Malay ( n  = 2) [ 100 , 101 ], Arabic ( n  = 1) [ 102 ], Chinese ( n  = 2) [ 68 , 103 ], Italian ( n  = 1) [ 104 ], Norwegian ( n  = 1) [ 105 ], Persian ( n  = 1) [ 106 ], Polish ( n  = 1) [ 107 ], Taiwanese ( n  = 2) [ 28 , 108 ], Turkish ( n  = 1) [ 109 ], and two studies did not report the version [ 65 , 89 ].

Sixteen studies reported on the test-retest reliability of the PASE, time frames ranging from 3 days [ 99 , 105 ] to 3–7 weeks [ 13 ] and sample sizes ranging from 18 [ 98 ] to 349 [ 100 ] (details available in Table  3 ). Across all versions of the PASE 12 studies reporting ICCs for the total score, only two fell below acceptable limits proposed in the COSMIN guidelines [ 110 ] (Malay version 0.49 (95% CI 0.37, 0.59) [ 100 ] and version NR 0.66 (95% CI 0.46–0.71) [ 89 ]). However, the majority of values were 0.90 and above ( n  = 8). Internal consistency was examined in seven versions and all Cronbach alpha’s fell within an acceptable range (0.70 (Arabic and Persian subcategory lowest) to 0.82 (Italian total score)). Only four studies examined measurement error. Alqarni et al. reported the minimal detectable change (MDC95) for PASE subcategories (9.0–23.6) [ 102 ] of the Arabic version and MDC95 for total scores were provided for the Chinese version (19.21) [ 68 ] and the Polish version (38.39) [ 107 ]. Two studies also included standard errors of measurement for the PASE total score (Chinese version 6.93 [ 68 ] and NR version 30.00 [ 89 ]).

Four studies stated they were exploring criterion validity; however, each used a different measurement tool as their gold standard for PA: pedometer (walking steps and energy expenditure) [ 68 ], Actigraph (activity counts/minutes) [ 28 ], International Physical Activity Questionnaire (IPAQ) [ 109 ], doubly labeled water (total energy expenditure, energy expenditure/resting metabolic rate) and VO2max [ 65 ]. The PASE was significantly correlated to all but the doubly labelled water outcomes and VO2max [ 65 ]. During the development of the PASE Washburn et al. assessed the three aspects content validity by asking participants ( n  = 36) about the appropriateness of the items, the completeness (i.e., comprehensiveness), and the comprehensibility; results were used to inform the final version of the PASE [ 13 ]. Three additional studies assessed and reported acceptable content validity for the PASE across three different clinical groups: acute coronary events (English) [ 98 ], COPD (Chinese) [ 68 ], and cancer survivors (Taiwanese) [ 28 ]. Only the English version had responsiveness and minimal important difference (MID) reported and this was in a sample of individuals with lung cancer [ 66 ].

Construct validity was the most commonly assessed form of validity, predominantly exploring convergent validity (details available in Table  4 ). Physical function performance measures and self-report questionnaires were commonly cited, and relationships ranged from fair to moderate, including the Timed Up and Go ( r  = -0.45 to r  = -0.69) [ 102 , 106 , 107 ], Berg Balance ( r  = 0.20 to r  = 0.82) [ 14 , 104 , 107 ], and the physical function section of the Short Form-36 ( r  = 0.53 to r  = 0.58) [ 68 , 103 , 109 ]. Muscle strength was another common construct with poor to fair correlations; specifically, grip strength ( r  = 0.29 to r  = 0.43) [ 13 , 68 , 100 , 102 , 103 ], and lower limb strength ( r  = 0.18 to r  = 0.37) [ 13 , 66 , 103 ]. There were also several self-report measures examining general health ( r  = -0.12 to r  = 0.44) [ 13 , 68 , 98 , 100 , 103 ] and activities of daily living ( r  = 0.10 to r  = 0.78) [ 100 , 106 ]. The PASE demonstrated moderate correlations with the IPAQ ( r  = 0.65 to r  = 0.74) [ 68 , 107 , 109 ]. Five studies compared the PASE to a direct measure of PA (e.g., accelerometers and pedometers), including outcomes such as steps per day ( r  = 0.39 to r  = 0.61) [ 66 , 68 , 101 ] and activity counts ( r  = 0.43 to r  = 0.64) with fair to moderate correlations [ 28 , 99 , 101 ]. Only Bonnefoy et al. used the gold standard doubly labelled water, and they found no significant correlations [ 65 ].

To the authors’ knowledge, this is the first review to provide a comprehensive summary of the use of the PASE in community-dwelling older adults. The PASE has been used extensively to measure PA in older adults (536 primary papers before restricting to community-dwelling settings); however, it was mainly used in high-income countries with cross-sectional research designs. While strong evidence was summarized supporting test-retest reliability and construct validity, there was a paucity of evidence examining the PASE’s responsiveness, important change thresholds, and predictive validity. In addition, we have presented pooled means for different age groups and clinical populations to provide preliminary reference values to improve interpretations of total scores.

The PASE has been used extensively in community-dwelling older adults; 171,206 participants from 35 countries were included in this review. The PASE was developed in the United States, which is reflected in the greater uptake in North America and high-income countries [ 13 ]. However, the PASE has been used across five continents and in some middle-income countries ( n  = 8). Importantly, we have seen the validation of several translated versions including Arabic, Chinese, Malay, Persian, and Turkish. Furthermore, the application of the PASE to clinical and disease-specific populations has also occurred, and the high content validity in these populations is promising. The use of the PASE in persons with chronic conditions has been supported previously based on feasibility and psychometric properties [ 5 ]. While the literature summarized is extensive, more is available outside of community-dwelling populations not captured in this review, including further translations and validations (e.g., Nigerian translation) [ 111 ]. Our results show the PASE is a commonly used measure of worldwide but has been used sparingly in countries outside of North America and in lower-income countries. Decreasing the heterogeneity in how PA is measured is imperative for meaningful comparisons and data harmonization. Large numbers of self-report PA measures already exist, and previous work has recommended using these rather than creating more [ 12 , 112 ]. This review shows the large uptake of the PASE, presenting a suitable choice for research on older adults. However, it is important that psychometric measures are assessed for the population of interest.

Psychometric properties are essential for outcome measures to ensure their validity, reliability, and interpretability. Of the 232 studies included, 19 studies aimed to examine the psychometric properties of the PASE in community-dwelling older adults. According to COSMIN, most studies (12/15) found acceptable test-retest reliability for the PASE total score. However, there was variability between studies that was more pronounced between subcategories of activity types (e.g., ICC subcategory values 0.56–0.94 [ 99 ], 0.76–0.93 [ 106 ], 0.78–0.99 [ 107 ]), which may suggest more variation week to week in single activity types and less for overall activity. There was a paucity of evidence on measurement error, including MDC and standard error of measurement. Of the four studies reporting in this area, one only provided values for activity subcategories, not total score [ 102 ], and two were for clinical populations (COPD and Parkinson’s disease). The varying populations may explain the large difference in values (e.g., MDC95 = 38.4 (general) vs MDC95 = 19.2(COPD); and SEM = 30 (PD) vs SEM = 6.9 (COPD)). Establishing the minimal detectable change values is essential for ensuring differences are real and not from measurement error. In addition, none of the included studies reported minimal clinically important differences (MCID), another important parameter for interpreting change in score. This paucity of evidence must be addressed across versions in community-dwelling older adults to support further use and interpretability of the PASE.

The PASE was validated in community-dwelling older adults in ten different languages. Content validity is regarded as the most important psychometric measurement property [ 113 ]; however, other than the sentinel paper, only three included studies reported on the relevance, comprehensiveness, and comprehensibility [ 28 , 68 , 98 ]. As presented in these papers, PA appears to be influenced by cultural/societal norms, highlighting the importance and continued need to verify the content validity of PA questionnaires when validating in new populations [ 28 ]. Fair to moderate relationships between the PASE and performance-based measures of physical function and mobility, strength, and health outcomes were regularly reported for construct validity. Four studies stated they examined criterion validity, which compares the PASE score to the gold standard of the same construct. However, only one study used the commonly regarded gold standard of PA doubly labelled water and did not find a significant relationship [ 65 ]. The remaining three studies found moderate correlations (> 0.60) using more accessible measures of PA: a pedometer [ 68 ], accelerometer [ 28 ], and a questionnaire [ 109 ]. The PASE-Polish [ 107 ] demonstrated the highest correlation at 0.74 with the IPAQ, which has been validated in 12 different countries, including low-income countries and rural samples [ 114 ]. The IPAQ was the only PA questionnaire reported, and only two other studies compared direct measures of PA (i.e., accelerometers). The correlations with the IPAQ ranged from 0.65–0.74, whereas correlations with direct measures tended to be lower and more variable (e.g., activity counts 0.43–64, walking steps 0.39–0.61). Several PASE versions did not contain a measure of PA in their validity analysis ( n  = 3). Further studies investigating these metrics using a wider variety of measures of PA (e.g., different questionnaires and more direct measures) are needed to clarify these relationships.

No studies reported on longitudinal validity, demonstrating a great need for studies to evaluate the PASE’s predictive validity for important health outcomes in community-dwelling populations across the globe. Despite almost 20 studies using the PASE to measure change in PA, responsiveness, which is critical for ensuring the PASE can accurately reflect change over time, has not been reported in any of the included studies. Therefore, research is needed to explore the predictive validity and responsiveness of the PASE to inform whether the PASE can be used to predict important health outcomes (e.g., future falls, hospitalization) and change in PA (e.g., over time or through intervention) for community-dwelling older adults.

A noteworthy finding of this review was the reporting of pooled means by age, sex, and clinical population. Pooled PASE scores decreased with increasing age groups from < 65 (159.53 (95% CI 146.58, 172.49)) to the 80 years and older group (67.17 (95% CI 51.95, 82.39)). In general, this is consistent with the literature where levels of PA progressively decrease with age for both men and women [ 115 , 116 ]. Some clinical populations appeared to have greater decreases in PA than others (e.g., cognitive impairment 91.11 (95% CI 72.77, 109.40) vs osteoarthritis 129.53 (95% CI 110.40, 148.65)). Clinical groups also appear to be important in addition to age for PA level; for example, the studies in the cognitive impairment group were mostly younger age groups (5/6 less than 80 years old), but the mean PASE score was closer to the two oldest age groups. The provided reference data for age, sex, and clinical population can be used to improve the interpretability of PASE scores among similar populations of community-dwelling older adults. However, future research creating normative values for the PASE could further improve interpretability and uptake of this questionnaire.

There are several limitations of this scoping review that should be acknowledged. First, several eligibility criteria were placed on this review, resulting in papers related to the PASE being excluded. Specifically, studies were restricted to the English language, age of 60 years or older, and community-dwelling settings. These decisions were made for feasibility and to reflect the original PASE; however, they have limited our understanding of how far the PASE has been applied in different populations. With the robust search strategy reviewed by a health research librarian, we are confident that the summarized evidence accurately reflects the current literature for community-dwelling older adults. A second limitation is that only published studies were included, and grey literature was not considered, which opens the possibility that new and emerging research regarding the PASE was missed. Finally, several studies used data from the same databases/studies, resulting in the same or overlapping samples; we did not extract the information necessary to tease this apart. Therefore, pooled means will be biased toward samples included more than once. In addition, pooled mean PASE scores in clinical populations with only two studies should be interpreted cautiously due to limited sample sizes.

This review has identified areas for future consideration, including further expanding the validation of the PASE to middle- and low-income countries. A systematic review focused on the psychometric properties of the PASE with no setting restrictions may provide a valuable resource for researchers. Future investigations are needed on psychometric properties of the PASE, including thresholds of important change, responsiveness, and predictive validity for all versions of the PASE, as well as data on psychometric properties in specific clinical populations.

This review found that the PASE is a widely used PA measure among community-dwelling older adults, with evidence supporting its test-retest reliability and construct validity. The widespread use of a questionnaire increases the ability for data harmonization across studies and improves the ability to compare between studies. Further research is warranted to investigate the PASE’s ability to detect meaningful change (i.e., MDC, MCID) along with predictive validity and responsiveness. Pooled mean total PASE scores reported in this review can provide preliminary reference values for different age groups and clinical populations to help improve the interpretability of PASE scores until normative values are established.

Availability of data and materials

All data generated or analyzed during this study are included in this published article [and its supplementary information files].

Abbreviations

Chronic Obstructive Pulmonary Disease

International Physical Activity Questionnaire - Short Form

• Physical activity

Physical Activity Scale for the Elderly

Parkinson’s Disease

Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews

Minimal clinical important difference

Minimal detectable change

Intraclass correlation coefficient

Interquartile range

Standard deviation

Standard error of measurement

Timed Up and Go

World Health Organization

Not reported

Confidence Interval

Statistics Canada. Census in Brief: A portrait of Canada’s growing population aged 85 and older from the 2021 Census. Catalogue no. 98-200-X(2021004). 2022. Available from: https://www12.statcan.gc.ca/census-recensement/2021/as-sa/98-200-X/2021004/98-200-X2021004-eng.cfm#moreinfo .

Kirkland SA, Griffith LE, Menec V, Wister A, Payette H, Wolfson C, et al. Mining a unique Canadian resource: the Canadian longitudinal study on aging. Can J Aging. 2015;34(3):366–77.

Article   PubMed   Google Scholar  

Decade of healthy aging: baseline report. Geneva: World Health Organization; 2020. Licence: CC BY-NC-SA 3.0 IGO.

Cunningham C, O’Sullivan R. Why physical activity matters for older adults in a time of pandemic. Eur Rev Aging Phys Act. 2020;17(1):16.

Article   PubMed   PubMed Central   Google Scholar  

Garnett A, Ploeg J, Markle-Reid M, Strachan PH. Self-report tools for assessing physical activity in community-living older adults with multiple chronic conditions: a systematic review of psychometric properties and feasibility. Can J Aging. 2020;39(1):12–30.

Mcphee JS, French DP, Jackson D, Nazroo J, Pendleton N, Degens H. Physical activity in older age: perspectives for healthy ageing and frailty. Biogerontology. 2016;17(3):567–80.

Steeves JA, Shiroma EJ, Conger SA, Van Domelen D, Harris TB. Physical activity patterns and multimorbidity burden of older adults with different levels of functional status: NHANES 2003–2006. Disabil Health J. 2019;12(3):495–502.

WHO guidelines on physical activity and sedentary behaviour. Geneva: Worl Health Organization; 2020.

Mayo A, Sénéchal M, Boudreau J, Bélanger M, Bouchard DR. Potential functional benefits of a comprehensive evaluation of physical activity for aging adults: a CLSA cross-sectional analysis. Aging Clin Exp Res. 2021;33(2):285–9.

Ross R, Chaput JP, Giangregorio LM, Janssen I, Saunders TJ, Kho ME, et al. Canadian 24-hour movement guidelines for adults aged 18–64 years and adults aged 65 years or older: an integration of physical activity, sedentary behaviour, and sleep. Appl Physiol Nutr Metab. 2020;45(10 (Suppl. 2)):S57–102.

Rao AK. Wearable Sensor Technology to Measure Physical Activity (PA) in the Elderly. Current Geriatrics Reports. 2019;8(1):55–66.

Sattler MC, Jaunig J, Tosch C, Watson ED, Mokkink LB, Dietz P, et al. Current evidence of measurement properties of physical activity questionnaires for older adults: an updated systematic review. Sports Med (Auckland, NZ). 2020;50(7):1271–315.

Article   Google Scholar  

Washburn RA, Smith KW, Jette AM, Janney CA. The Physical Activity Scale for the Elderly (PASE): development and evaluation. J Clin Epidemiol. 1993;46(2):153–62.

Washburn RA, McAuley E, Katula J, Mihalko SL, Boileau RA. The physical activity scale for the elderly (PASE): evidence for validity. J Clin Epidemiol. 1999;52(7):643–51.

Aromataris E, Munn Z. JBI manual for evidence synthesis. 2020.

Google Scholar  

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

United Nations, Department of Economic and Social Affairs PD. World Population Ageing 2017. 2017;ST/ESA/SER.A/408. Available from: https://www.un.org/development/desa/pd/sites/www.un.org.development.desa.pd/files/files/documents/2020/May/un_2017_worldpopulationageing_report.pdf .

Higgins J, Li T, Deeks JE. Chapter 6: Choosing effect measures and computing estimates of effect. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane handbook for systematic reviews of interventions version 62 (updated February 2021). Cochrane; 2021. Available from www.training.cochrane.org/handbook .

Wan X, Wang W, Liu J, Tong T. Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Med Res Methodol. 2014;14(1):135.

The World Bank. World Bank Country and Lending Groups - World Bank Data Help Desk. The World Bank Group. Available from: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups .Cited 2021 July.

Yu R, Leung J, Woo J. Housework reduces all-cause and cancer mortality in Chinese men. PLoS One. 2013;8(5):e61529.

Yu R, Cheung O, Lau K, Woo J. Associations between perceived neighborhood walkability and walking time, wellbeing, and loneliness in community-dwelling older Chinese people in Hong Kong. Int J Environ Res Public Health. 2017;14(10):1199.

Liang Y, Lai FTT, Kwan JLY, Chan W, Yeoh E-K. Sport and recreational physical activities attenuate the predictive association of multimorbidity with increased geriatric depressive symptoms: a 14-year follow-up study of community-dwelling older adults. J Aging Phys Act. 2022;30(2):252–60.

Kitano N, Tsunoda K, Tsuji T, Osuka Y, Jindo T, Tanaka K, et al. Association between difficulty initiating sleep in older adults and the combination of leisure-time physical activity and consumption of milk and milk products: a cross-sectional study. BMC Geriatr. 2014;14:118.

Wang YF, Li NY, Zhu JJ, Deng Q, Hu JLL, Xu J, et al. Association between socio-ecological factors and leisure time physical activity (LTPA) among older adults in Sichuan, China: a structural equation modeling analysis. BMC Geriatr. 2022;22(1):60.

Kenny AM, Dawson L, Kleppinger A, Iannuzzi-Sucich M, Judge JO. Prevalence of sarcopenia and predictors of skeletal muscle mass in nonobese women who are long-term users of estrogen-replacement therapy. J Gerontol A Biol Sci Med Sci. 2003;58(5):M436–40.

Julien D, Gauvin L, Richard L, Kestens Y, Payette H. Longitudinal associations between walking frequency and depressive symptoms in older adults: results from the VoisiNuAge study. J Am Geriatr Soc. 2013;61(12):2072–8.

Su C-C, Lee K-D, Yeh C-H, Kao C-C, Lin C-C. Measurement of physical activity in cancer survivors: a validity study. J Cancer Surviv. 2014;8(2):205–12.

Moored KD, Qiao Y, Boudreau RM, Roe LS, Cawthon PM, Cauley JA, et al. Prospective associations between physical activity and perceived fatigability in older men: differences by activity type and baseline marital status. J Gerontol A Biol Sci Med Sci. 2022;77(12):2498–506.

Li JX, Hodgson N, Lyons MM, Chen KC, Yu F, Gooneratne NS. A personalized behavioral intervention implementing mHealth technologies for older adults: a pilot feasibility study. Geriatr Nurs. 2020;41(3):313–9.

Chang VC, Do MT. Risk factors for falls among seniors: implications of gender. Am J Epidemiol. 2015;181(7):521–31.

Bernstein JP, Noland MD, Dorociak KE, Leese MI, Lee SY, Hughes A. Executive functioning predicts discrepancies between objective and self-reported physical activity in older adults: a pilot study. Aging Neuropsychol Cogn. 2023;30(1):124–34.

Boa Sorte Silva NC, Dao E, Hsu CL, Tam RC, Lam K, Alkeridy W, et al. Myelin and Physical Activity in Older Adults with Cerebral Small Vessel Disease and Mild Cognitive Impairment. The journals of gerontology Series A, Biological sciences and medical sciences. 2022;78(3):545–3.

Escher C, Asken BM, VandeBunte A, Fonseca C, You M, Kramer JH, et al. Roles of physical activity and diet in cognitive aging: is more better? The Clinical neuropsychologist. 2022;37(2):1–18.

Haggard AV, Tennant JE, Shaikh FD, Hamel R, Kline PW, Zukowski LA. Including cognitive assessments with functional testing predicts capabilities relevant to everyday walking in older adults. Gait Posture. 2023;100:75–81.

Ibrahim AM, Singh DKA, Mat S, Mat Ludin AF, Shahar S. Incidence and Predictors of Physical Inactivity Among Malaysian Community-Dwelling Older Persons. Journal of aging and physical activity. 2022;31(1):105–6.

Gregorio L, Brindisi J, Kleppinger A, Sullivan R, Mangano KM, Bihuniak JD, et al. Adequate dietary protein is associated with better physical performance among post-menopausal women 60–90 years. J Nutr Health Aging. 2014;18(2):155–60.

Nemmers TM, Miller JW. Factors influencing balance in healthy community-dwelling women age 60 and older. J Geriatr Phys Ther (2001). 2008;31(3):93–100.

McAuley E, Hall KS, Motl RW, White SM, Wojcicki TR, Hu L, et al. Trajectory of declines in physical activity in community-dwelling older women: social cognitive influences. J Gerontol B Psychol Sci Soc Sci. 2009;64(5):543–50.

McAuley E, Morris KS, Doerksen SE, Motl RW, Hu L, White SM, et al. Effects of change in physical activity on physical function limitations in older women: mediating roles of physical function performance and self-efficacy. J Am Geriatr Soc. 2007;55(12):1967–73.

Martin FC, Hart D, Spector T, Doyle DV, Harari D. Fear of falling limiting activity in young-old women is associated with reduced functional mobility rather than psychological factors. Age Ageing. 2005;34(3):281–7.

Morris KS, McAuley E, Motl RW. Neighborhood satisfaction, functional limitations, and self-efficacy influences on physical activity in older women. Int J Behav Nutr Phys Act. 2008;5:13.

Legg HS, Arnold CM, Trask C, Lanovaz JL. Does functional performance and upper body strength predict upper extremity reaction and movement time in older women? Hum Mov Sci. 2021;77:102796.

Leis KS, Reeder BA, Chad KE, Spink KS, Fisher KL, Bruner BG. The relationship of chronic disease and demographic variables to physical activity in a sample of women aged 65 to 79 years. Women Health. 2010;50(5):459–74.

Liu-Ambrose T, Khan KM, Eng JJ, Lord SR, McKay HA. Balance confidence improves with resistance or agility training - Increase is not correlated with objective changes in fall risk and physical abilities. Gerontology. 2004;50(6):373–82.

Lange AK, Singh MAF, Smith RM, Foroughi N, Baker MK, Shnier R, et al. Degenerative meniscus tears and mobility impairment in women with knee osteoarthritis. Osteoarthritis Cartilage. 2007;15(6):701–8.

Greenspan SL, Resnick NM, Parker RA. The effect of hormone replacement on physical performance in community-dwelling elderly women. Am J Med. 2005;118(11):1232–9.

Newton RA, Cromwell RL, Rogers HL. The relationship between physical performance and obesity in elderly African-American women. Phys Occup Ther Geriatr. 2009;27(6):423–40.

Talley KMC, Wyman JF, Gross CR, Lindquist RA, Gaugler JE. Change in balance confidence and its associations with increasing disability in older community-dwelling women at risk for falling. J Aging Health. 2014;26(4):616–36.

Kenny AM, Kleppinger A, Wang Y, Prestwood KM. Effects of ultra-low-dose estrogen therapy on muscle and physical function in older women. J Am Geriatr Soc. 2005;53(11):1973–7.

Bellantonio S, Fortinsky R, Prestwood K. How well are community-living women treated for osteoporosis after hip fracture? J Am Geriatr Soc. 2001;49(9):1197–204.

Klima DW. Physical performance and balance confidence among community-dwelling older adult men. 2010. (Ph.D.). p. 159.

Eendebak RJAH, Ahern T, Swiecicka A, Pye SR, O’Neill TW, Bartfai G, et al. Elevated luteinizing hormone despite normal testosterone levels in older men-natural history, risk factors and clinical features. Clin Endocrinol. 2018;88(3):479–90.

Laddu D, Parimi N, Cauley JA, Cawthon PM, Ensrud KE, Orwoll E, et al. The association between trajectories of physical activity and all-cause and cause-specific mortality. J Gerontol A Biol Sci Med Sci. 2018;73(12):1708–13.

Lange-Maia BS, Cauley JA, Newman AB, Boudreau RM, Jakicic JM, Glynn NW, et al. Sensorimotor peripheral nerve function and physical activity in older men. J Aging Phys Act. 2016;24(4):559–66.

Mackey DC, Hubbard AE, Cawthon PM, Cauley JA, Cummings SR, Tager IB. Usual physical activity and hip fracture in older men: an application of semiparametric methods to observational data. Am J Epidemiol. 2011;173(5):578–86.

Cousins JM, Petit MA, Paudel ML, Taylor BC, Hughes JM, Cauley JA, et al. Muscle power and physical activity are associated with bone strength in older men: the osteoporotic fractures in men study. Bone. 2010;47(2):205–11.

Mesinovic J, Scott D, Seibel MJ, Cumming RG, Naganathan V, Blyth FM, et al. Risk factors for incident falls and fractures in older men with and without type 2 diabetes mellitus: the Concord Health and Ageing in Men Project. J Gerontol A Biol Sci Med Sci. 2021;76(6):1090–100.

Hsu B, Merom D, Blyth FM, Naganathan V, Hirani V, Le Couteur DG, et al. Total physical activity, exercise intensity, and walking speed as predictors of all-cause and cause-specific mortality over 7 years in older men: the Concord Health and Aging in Men Project. J Am Med Dir Assoc. 2018;19(3):216–22.

Ng CA, Scott D, Seibel MJ, Cumming RG, Naganathan V, Blyth FM, et al. Higher Impact Physical Activity is Associated with Maintenance of Bone Mineral Density but Not Reduced Incident Falls or Fractures in Older Men: The Concord Health and Ageing in Men Project. Journal of bone and mineral research : the official journal of the American Society for Bone and Mineral Research. 2020;36(4):662–72.

Cronholm F, Rosengren BE, Nilsson J, Ohlsson C, Mellstrom D, Ribom E, et al. The fracture predictive ability of a musculoskeletal composite score in old men - data from the MrOs Sweden study. BMC Geriatr. 2019;19(1):90.

Kenny AM, Biskup B, Robbins B, Marcella G, Burleson JA. Effects of vitamin D supplementation on strength, physical function, and health perception in older, community-dwelling men. J Am Geriatr Soc. 2003;51(12):1762–7.

Kenny AM, Prestwood KM, Kenny AM, Prestwood KM, Marcello KM, Raisz LG. Determinants of bone density in healthy older men with low testosterone levels. J Gerontol A Biol Sci Med Sci. 2000;55(9):M492–7.

Liu PY, Wishart SM, Handelsman DJ. A double-blind, placebo-controlled, randomized clinical trial of recombinant human chorionic gonadotropin on muscle strength and physical function and activity in older men with partial age-related androgen deficiency. J Clin Endocrinol Metab. 2002;87(7):3125–35.

Bonnefoy M, Normand S, Pachiaudi C, Lacour JR, Laville M, Kostka T. Simultaneous validation of ten physical activity questionnaires in older men: a doubly labeled water study. J Am Geriatr Soc. 2001;49(1):28–35.

Granger C, Parry S, Denehy L, Granger CL, Parry SM. The self-reported Physical Activity Scale for the Elderly (PASE) is a valid and clinically applicable measure in lung cancer. Support Care Cancer. 2015;23(11):3211–8.

Oliveira CC, McGinley J, Lee AL, Irving LB, Denehy L. Fear of falling in people with chronic obstructive pulmonary disease. Respir Med. 2015;109(4):483–9.

Tao Y-X, Wang L, Dong X-Y, Zheng H, Zheng Y-S, Tang X-Y, et al. Psychometric properties of the Physical Activity Scale for the Elderly in Chinese patients with COPD. Int J Chron Obstruct Pulmon Dis. 2017;12:105–14.

Burns JM, Mayo MS, Anderson HS, Smith HJ, Donnelly JE. Cardiorespiratory fitness in early-stage Alzheimer disease. Alzheimer Dis Assoc Disord. 2008;22(1):39–46.

Farina N, Tabet N, Rusted J. Habitual physical activity (HPA) as a factor in sustained executive function in Alzheimer-type dementia: a cohort study. Arch Gerontol Geriatr. 2014;59(1):91–7.

Lin YP, Yang YH, Hsiao SF. Physical activity, muscle strength, and functional fitness: comparing older adults with and without Alzheimer dementia. Top Geriatr Rehabil. 2019;35(4):280–8.

Talamonti D, Gagnon C, Vincent T, Nigam A, Lesage F, Bherer L, et al. Exploring cognitive and brain oxygenation changes over a 1-year period in physically active individuals with mild cognitive impairment: a longitudinal fNIRS pilot study. BMC Geriatr. 2022;22(1):648.

van Santen J, Droes RM, Twisk JWR, Henkemans OAB, van Straten A, Meiland FJM. Effects of exergaming on cognitive and social functioning of people with dementia: a randomized controlled trial. J Am Med Dir Assoc. 2020;21(12):1958-#x0002B;

El Rahi B, Shatenstein B, Morais JA. The joint effects of diet quality and physical activity on functional decline among diabetic older adults from the NuAge cohort. J Am Geriatr Soc. 2013;61(SUPPL. 1):S209.

Rahi B, Morais JA, Gaudreau P, Payette H, Shatenstein B. Decline in functional capacity is unaffected by diet quality alone or in combination with physical activity among generally healthy older adults with T2D from the NuAge cohort. Diabetes Res Clin Pract. 2014;105(3):399–407.

Sazlina S-G, Browning CJ, Yasin S. Effectiveness of personalized feedback alone or combined with peer support to improve physical activity in sedentary older Malays with type 2 diabetes: a randomized controlled trial. Front Public Health. 2015;3:178.

Barker KL, Room J, Knight R, Dutton S, Toye F, Leal J, et al. Home-based rehabilitation programme compared with traditional physiotherapy for patients at risk of poor outcome after knee arthroplasty: the CORKA randomised controlled trial. BMJ Open. 2021;11(8):e052598.

Batsis JA, Zbehlik AJ, Barre LK, Bynum JPW, Pidgeon D, Bartels SJ. Impact of obesity on disability, function, and physical activity: data from the Osteoarthritis Initiative. Scand J Rheumatol. 2015;44(6):495–502.

Bennell KL, Ahamed Y, Jull G, Bryant C, Hunt MA, Forbes AB, et al. Physical therapist-delivered pain coping skills training and exercise for knee osteoarthritis: randomized controlled trial. Arthritis Care Res. 2016;68(5):590–602.

Bieler T, Anderson T, Beyer N, Rosthoj S. The impact of self-efficacy on activity limitations in patients with hip osteoarthritis: results from a cross-sectional study. ACR Open Rheumatol. 2020;2(12):741–9.

Chmelo E, Nicklas B, Davis C, Legault C, Miller GD, Messier S. Physical activity and physical function in older adults with knee osteoarthritis. J Phys Act Health. 2013;10(6):777–83.

Dunlop DD, Semanik P, Song J, Sharma L, Nevitt M, Jackson R, et al. Moving to maintain function in knee osteoarthritis: evidence from the Osteoarthritis Initiative. Arch Phys Med Rehabil. 2010;91(5):714–21.

Hinman RS, Wrigley TV, Metcalf BR, Campbell PK, Paterson KL, Hunter DJ, et al. Unloading shoes for self-management of knee osteoarthritis: a randomized trial. Ann Intern Med. 2016;165(6):381–9.

McDonald DD, Shellman JM, Graham L, Harrison L. The relationship between reminiscence functions, optimism, depressive symptoms, physical activity, and pain in older adults. Res Gerontol Nurs. 2016;9(5):223–31.

Sharma L, Cahue S, Song J, Hayes K, Pai Y, Dunlop D. Physical functioning over three years in knee osteoarthritis: role of psychosocial, local mechanical, and neuromuscular factors. Arthritis Rheum. 2003;48(12):3359–70.

Skou ST, Wise BL, Lewis CE, Felson D, Nevitt M, Segal NA. Muscle strength, physical performance and physical activity as predictors of future knee replacement: a prospective cohort study. Osteoarthritis Cartilage. 2016;24(8):1350–6.

Smith TO, Mansfield M, Dainty J, Hilton G, Mann CJV, Sackley CM. Does physical activity change following hip and knee replacement? Matched case-control study evaluating Physical Activity Scale for the Elderly data from the Osteoarthritis Initiative. Physiotherapy. 2018;104(1):80–90.

Amara AW, Chahine L, Seedorff N, Caspell-Garcia CJ, Coffey C, Simuni T, et al. Self-reported physical activity levels and clinical progression in early Parkinson’s disease. Parkinsonism Relat Disord. 2019;61:118–25.

Ånfors S, Kammerlind A-S, Nilsson MH. Test-retest reliability of physical activity questionnaires in Parkinson’s disease. BMC Neurol. 2021;21(1):1–13.

Ashburn A, Pickering R, McIntosh E, Hulbert S, Rochester L, Roberts HC, et al. Exercise- and strategy-based physiotherapy-delivered intervention for preventing repeat falls in people with Parkinson’s: the PDSAFE RCT. Health Technol Assess (Winchester, England). 2019;23(36):1–150.

Balci B, Aktar B, Buran S, Tas M, Colakoglu BD. Impact of the COVID-19 pandemic on physical activity, anxiety, and depression in patients with Parkinson’s disease. Int J Rehabil Res. 2021;44(2):173–6.

Bryant MS, Hou JG, Collins RL, Protas EJ. Contribution of axial motor impairment to physical inactivity in Parkinson disease. Am J Phys Med Rehabil. 2016;95(5):348–54.

Bryant MS, Kang GE, Protas EJ. Relation of chair rising ability to activities of daily living and physical activity in Parkinson’s disease. Arch Physiother. 2020;10(1):22.

Bryant MS, Rintala DH, Hou J-G, Protas EJ. Relationship of falls and fear of falling to activity limitations and physical inactivity in Parkinson’s disease. J Aging Phys Act. 2015;23(2):187–93.

Ellis T, Cavanaugh JT, Earhart GM, Ford MP, Foreman KB, Fredman L, et al. Factors associated with exercise behavior in people with Parkinson disease. Phys Ther. 2011;91(12):1838–48.

Mantri S, Wood S, Duda JE, Morley JF. Understanding physical activity in Veterans with Parkinson disease: a mixed-methods approach. Parkinsonism Relat Disord. 2019;61:156–60.

Mantri S, Wood S, Duda JE, Morley JF. Comparing self-reported and objective monitoring of physical activity in Parkinson disease. Parkinsonism Relat Disord. 2019;67:56–9.

Allison MJ, Keller C, Hutchinson PL. Selection of an instrument to measure the physical activity of elderly people in rural areas. Rehabil Nurs. 1998;23(6):309–14.

Dinger MK, Oman F, Taylor EL, Vesely SK, Able J. Stability and convergent validity of the Physical Activity Scale for the Elderly (PASE). J Sports Med Phys Fitness. 2004;44(2):186–92.

PubMed   Google Scholar  

Ismail N, Hairi F, Choo WY, Hairi NN, Peramalah D, Bulgiba A. The Physical Activity Scale for the Elderly (PASE): validity and reliability among community-dwelling older adults in Malaysia. Asia Pac J Public Health. 2015;27(8 Suppl):62S-72S.

Singh DKA, Rahman NNAA, Rajaratnam BS, Yi TC, Shahar S. Validity and reliability of physical activity scale for elderly in Malay language (PASE-M). Malays J Public Health Med. 2018;2018(Specialissue1):116–23.

Alqarni AM, Vennu V, Alshammari SA, Bindawas SM. Cross-cultural adaptation and validation of the Arabic version of the Physical Activity Scale for the Elderly among community-dwelling older adults in Saudi Arabia. Clin Interv Aging. 2018;13:419–27.

Ngai SP, Cheung RT, Lam PL, Chiu JK, Fung EY. Validation and reliability of the Physical Activity Scale for the Elderly in Chinese population. J Rehabil Med. 2012;44(5):462–5.

Covotta A, Gagliardi M, Berardi A, Maggi G, Pierelli F, Mollica R, et al. Physical Activity Scale for the Elderly: Translation, Cultural Adaptation, and Validation of the Italian Version. Current Gerontology & Geratrics Research. 2018;8294568:1–7.

Loland NW. Reliability of the physical activity scale for the elderly (PASE). Eur J Sport Sci. 2002;2(5):1–12.

Keikavoosi-Arani L, Salehi L. Cultural adaptation and psychometric adequacy of the Persian version of the physical activity scale for the elderly (P-PASE). BMC Res Notes. 2019;12(1):555.

Wisniowska-Szurlej A, Cwirlej-Sozanska A, Woloszyn N, Sozanski B, Wilmowska-Pietruszynska A, Washburn R. Cultural adaptation and validation of the Polish version of the physical activity scale for older people living in a community: a cross-sectional study. Eur Rev Aging Phys Act. 2020;17(1):19.

Wu C-Y, Su T-P, Fang C-L, Yeh CM. Sleep quality among community-dwelling elderly people and its demographic, mental, and physical correlates. J Chin Med Assoc. 2012;75(2):75–80.

Ayvat E, Kilinc M, Kirdi N. The Turkish version of the Physical Activity Scale for the Elderly (PASE): its cultural adaptation, validation, and reliability. Turk J Med Sci. 2017;47(3):908–15.

Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27(5):1147–57.

Okoye EC, Akosile CO, Maruf FA, Onwuakagba IU, Chukwuma VC. Cross-cultural adaptation and validation of Nigerian (Igbo) version of the physical activity scale for the elderly. J Aging Phys Act. 2021;29(4):553–61.

D’Amore C, Saunders S, Bhatnagar N, Griffith LE, Richardson J, Beauchamp MK. Determinants of physical activity in community-dwelling older adults: an umbrella review. Int J Behav Nutr Phys Act. 2023;20(1):135.

Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res. 2018;27(5):1159–70.

Craig CL, Marshall AL, Sjöström M, Bauman AE, Booth ML, Ainsworth BE, et al. International physical activity questionnaire: 12-country reliability and validity. Med Sci Sports Exerc. 2003;35(8):1381–95.

Sun F, Norman IJ, While AE. Physical activity in older people: a systematic review. BMC Public Health. 2013;13(1):449.

Aleksovska K, Puggina A, Giraldi L, Buck C, Burns C, Cardon G, et al. Biological determinants of physical activity across the life course: a “Determinants of Diet and Physical Activity” (DEDIPAC) umbrella systematic literature review. Sports Med Open. 2019;5(1):2.

Download references

Acknowledgements

Ms. Neera Bhatnagar, a librarian at Health Sciences Library at McMaster University, for guiding the authors in the development of the search strategy.

Not applicable.

Author information

Cassandra D’Amore and Lexie Lajambe are co-first authors.

Authors and Affiliations

School of Rehabilitation Science, Faculty of Health Sciences, Institute of Applied Health Sciences, McMaster University, 1400 Main St. West Hamilton, Room 403, Hamilton, ON, L8S 1C7, Canada

Cassandra D’Amore, Lexie Lajambe, Noah Bush, Sydney Hiltz, Justin Laforest, Isabella Viel, Qiukui Hao & Marla Beauchamp

You can also search for this author in PubMed   Google Scholar

Contributions

MB and CD conceptualized the research question; LL, NB, SH, JL, IV in consultation with Ms Bhatnagar and CD, QH, and MB created protocol and search strategies. NB, LL, SH, JL, IV and CD carried out screening and extracting papers. JL, CD and QH carried out analyses and all authors contributed to the final manuscript.

Corresponding author

Correspondence to Marla Beauchamp .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

Author MB is supported by a Tier 2 Canada Research Chair in Mobility, Aging and Chronic Disease.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. additional methods and results details., additional file 2. full list of included and excluded studies., additional file 3. data extraction sheet., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

D’Amore, C., Lajambe, L., Bush, N. et al. Mapping the extent of the literature and psychometric properties for the Physical Activity Scale for the Elderly (PASE) in community-dwelling older adults: a scoping review. BMC Geriatr 24 , 761 (2024). https://doi.org/10.1186/s12877-024-05332-3

Download citation

Received : 11 April 2024

Accepted : 26 August 2024

Published : 14 September 2024

DOI : https://doi.org/10.1186/s12877-024-05332-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Psychometrics
• Questionnaire

BMC Geriatrics

ISSN: 1471-2318

• General enquiries: [email protected]

• Open access
• Published: 14 September 2024

Development of an infertility perception scale for women (IPS-W)

• Miok Kim 1 &
• Minkyung Ban 2  

BMC Women's Health volume  24 , Article number:  513 ( 2024 ) Cite this article

Metrics details

The purpose of this study was to develop an Infertility Perception Scale for Women (IPS-W).

Initial items were based on an extensive literature review and in-depth interviews with five infertile women and fifteen women not diagnosed with infertility. Forty-one items were derived from a pilot survey. Data were collected from 203 women who had experienced intrauterine insemination (IUI) and in-vitro fertilization (IVF) more than once. The data were analyzed to verify the reliability and validity of the scale.

Four factors containing 21 items were extracted from the exploratory factor analysis (EFA) to verify the construct validity. The four factors of infertility perception scale were perceived feelings, personal stigma, social stigma, and acceptance. These factors explained 59.3% of the total variance. The confirmatory factor analysis (CFA) confirmed a four-factor structure of the 21-item IPS-W. All fit indices were satisfactory (χ 2 /df ≤ 3, RMSEA < 0.08). These items were verified through convergent, discriminant, known group validity, concurrent validity testing. The internal consistency reliability was acceptable (Cronbach’s α = 0.90).

The scale reflects the perception of infertility within the cultural context of Korea. The findings can help nurses provide support that is appropriate for individual circumstances by examining how women experiencing infertility perceive infertility.

Peer Review reports

Introduction

Infertility, defined as the failure to achieve pregnancy within one year of regular unprotected sexual intercourse for couples of reproductive age, is perceived in the socio-cultural context of Korea as an impediment to an essential social rite of passage and a health problem that threatens family continuity [ 1 ]. In fact, Koreans struggling with infertility believe that giving birth to a child is a prerequisite for a stable marital relationship and that having children allows the development of family relationships [ 2 ].

Infertility is perceived negatively due to the uncertainty, loss of control, and stressful situations [ 3 ] it causes, having a negative impact on quality of life [ 4 ]. Accordingly, decisions about treatment procedures should be made by each individual and as a couple after exploring various options and with an appropriate social support system. However, individuals with infertility often decide on infertility treatment without an appropriate decision-making process, due to the fear of experiencing social prejudice and negative perceptions when revealing infertility problems [ 5 ].

The psychological difficulties faced by individuals with infertility are partly caused by negative social perceptions [ 5 , 6 ]. Approximately half of all infertile couples tend to hide their infertility problem out of fear of social stigma [ 7 ]. Such stigmatizing views of infertility create a vicious cycle leading infertile individuals to have a negative perception of their problem, avoid or prematurely discontinue infertility treatment, and feel isolated from society [ 5 ].

Harzif, Santawi, and Wijaya [ 8 ] compared differences in perceptions towards infertility treatment between urban and rural areas by examining the level of knowledge on the risk factors for infertility, attitudes towards infertility, the social impact of infertility, and other options for infertile couples. However, as yet, the perceptions of the individuals receiving infertility treatment have not been specifically measured [ 9 ]. Despite the limited tools available for measuring infertility perceptions, some studies have assessed the stigmatizing characteristics of infertility; however, in most cases, the term “infertility” does not appear in the generic tools for measuring stigma [ 7 , 10 , 11 ]. Taebe et al. [ 5 ] developed the female infertility stigma instrument (ISI-F) based on the premise that infertility represents one of the biggest challenges in female reproductive and sexual health in most societies. However, the development of ISI-F relied on females attempting natural pregnancy, including ovulation induction. Consequently, while it shares some of the stigmatizing characteristics of infertile individuals, this tool may not adequately reflect the psycho-emotional difficulties of individuals receiving infertility treatments. The Infertility Stigma Scale (ISS) developed by Fu et al. [ 12 ] is designed to measure the perceived self-stigmas that a female receiving infertility treatment places on herself. However, the scale only encompasses the stigmatizing characteristics of infertility, therefore it does not allow for the assessement of the general perception that females undergoing infertility treatment have about their infertility, especially in a Korean context. The concept of stigma refers to a psychological attitude linked to a series of negative outcomes [ 12 ]. In contrast, perception refers to the process of recognizing and interpreting the nature and meaning of all types of stimuli and may vary depending on how individuals interpret the situation and the society they live in. Perception can be used to explore problems and make decisions on what should be changed and the strategy ahead [ 13 ]. Self-perceived stigma acts as a stress factor that leads to negative social stigma about infertility and interferes with life adaptation [ 14 ]; hence, it is all the more important to improve the perceptions of infertility.

Therefore, this study aims to develop a sensitive tool to measure how women who experience infertility treatment perceive their infertility, positioning them as active agents in coping with infertility. The goal is to assess their personal perceptions of infertility and facilitate a positive shift in those perceptions.

The development and validation of the infertility perception scale was performed in accordance with the method proposed by Devellis [ 15 ]. This study adheres to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines.

Development of the infertility perception scale

Development of preliminary items.

To identify the components of infertility perception, previous literature published in RISS, PUBMED, EMBASE, and NDSL were searched for relevant studies and existing tools in English or Korean. Only studies with full-text availability were included. The search keywords included “infertility,” “fertility,” “infertile women,” “infertility for women,” “infertility experiences,” “experiences of infertility,” “assisted reproductive technology,” “intrauterine insemination,” “in vitro fertilization,” “perception,” and “awareness.” Among articles from PubMed, EMBase, and RISS, a total 42 articles were selected and reviewed, excluding duplicate articles ( n  = 102), studies with no women subjects ( n  = 48), studies not relevant to perception ( n  = 205), systematic reviews ( n  = 6), and case studies ( n  = 21).

In-depth interviews were then conducted to confirm the initially identified items of infertility perception. The interviews were conducted separately for women who had not experienced infertility and women who were receiving infertility treatment to avoid the possible influence of social perception on the perception of infertility among the women under treatment [ 16 ]. Moreover, the emotional ups and downs experienced by individuals receiving infertility treatment may vary significantly depending on the number of follicles, the number and quality of the collected eggs, the need to undergo repetitive infertility treatments, and added negative emotions from previous failed treatments.

The women without experience of infertility were married women aged 20–69 years (three women per age group) selected by purposive sampling with consideration of their education level, economic status, and type of residence. The interviews, which lasted between 20 and 40 min, were conducted between April 7 and April 20, 2020.

Subsequently, interviews were conducted on six women undergoing IVF procedures to identify the central concept of infertility perception. Announcements were posted inside treatment centers to recruit potential participants. The interviews, which lasted between 50 and 70 min, were conducted between May 10 and May 18, 2020.

The central question in the interviews was “How do you feel about infertility?” The supplementary question for women not diagnosed with infertility was “What did you think when you found out someone who was diagnosed with infertility or has experienced receiving infertility treatment?” The supplementary question for women receiving infertility treatment was “What feelings or thoughts did you have as you were diagnosed with infertility and undergoing infertility treatment?” The in-depth interviews were recorded with the consent of the participants, transcribed immediately, and analyzed according to the content analysis procedure proposed by Krippendorff [ 17 ].

After defining the conceptual framework through content analysis based on literature review and in-depth interviews, a total of 103 items were derived. Subsequently, duplicate items and those with unclear content were deleted, revised, or supplemented through a meeting with an expert with infertility nursing and research experience. As a result, a total of 66 preliminary items were derived, including 29, 18, and 19 items in personal, relational, and social dimensions, respectively.

To avoid central bias, during the instrument development process, the 4-point scale proposed by Lynn [ 18 ] was used to grade each item based on the level of agreement (1: “Strongly disagree” to 4: “Strongly agree”). The items in sections 1, 2, and 3 of the scale are reverse-scored, so that a higher total score indicates a more positive perception of infertility.

Content validity testing

In this study, content validity was tested twice by calculating the content validity index (CVI) based on expert opinion. The appropriateness of each item was assessed using a 4-point Likert scale (4: “Highly relevant”; 3: “Quite relevant”; 2: “Somewhat relevant”; and 1: “Not relevant”). Items with an Item-Content Validity Index (I-CVI) ≥ 0.78 were selected. The first content validity testing was conducted in September 2020 by a 10-member expert panel [ 18 ].

The second content validity testing was conducted by five members from the first 10-member expert panel. Items with I-CVI ≥ 0.78 were selected, leading to a total of 41 preliminary items.

Pilot study

A pilot study was conducted with 20 women living in city B who had been diagnosed with infertility and received at least one round of assisted reproductive therapy to assess the level of understanding about the instrument and the time required to complete the questionnaire. The number of participants was based on the sample size of 20–40 participants for pilot studies proposed by Devellis [ 15 ]. Item appropriateness was assessed through questions such as “Are there any items that are difficult to understand?”, “Are there any items with ambiguous expressions?”, and “Are there any items that you believe lack relevance to the perception of infertility?”

Instrument evaluation

The instrument was assessed through item analysis, exploratory factor analysis (EFA), confirmatory factor analysis (CFA), convergent validity, discriminant validity, criterion validity, and reliability testing.

The subjects were married women aged 20 and above diagnosed with infertility; had received at least one round of IUI or IVF; currently receiving infertility treatment; understood the study’s purpose; and signed an informed consent. Those who had difficulties in understanding and responding to the self-reported questionnaire; had problems with cognitive comprehension; or had other physical disorders in addition to infertility were excluded. Based on the criteria that a sample size more than 200 or approximately five times the number of items would be appropriate for factor analysis [ 15 , 19 ] and considering a dropout rate of 10%, a total of 212 subjects were recruited.

Data was collected from eligible, voluntary participants from two hospitals between November 30, 2020 and February 5, 2021. In our study, a high valid response rate of 94.9% (203 valid responses out of 214 recruited participants) was achieved. This was primarily attributed to the structured approach in participant recruitment and data collection. Participants were initially approached by a researcher or trained assistant at the infertility treatment hospital, who provided a detailed explanation of the study’s purpose, the voluntary nature of participation, and assurances of confidentiality. Only those who expressed voluntary willingness to participate were provided with a QR code to access an online survey, which took approximately 15 to 20 min to complete. Conducting the survey during patients’ waiting times further facilitated participation. These factors collectively ensured that participants felt informed and comfortable in participating in the study.

For the concurrent validity, ISS developed by Fu et al. [ 12 ] was used. After obtaining permission, ISS was translated into Korean and was reviewed. Subsequently, the translated version was translated back into the original language and compared with the original items. The final version consisted of 27 items in Korean. The self-reported instrument consisted of four domains (self-devaluation, social withdrawal, public stigma, and family stigma) in 5-point scale (1: “Do not agree at all” to 5: “Strongly agree”). At the time of development, the reliability of the scale was indicated by Cronbach’s alpha = 0.94. Cronbach’s alpha was 0.86, 0.77, 0.92, and 0.84 for self-devaluation, social withdrawal, public stigma, and family stigma, respectively. In this study, the reliability of the scale was indicated by Cronbach’s alpha = 0.97. Cronbach’s alpha was 0.94, 0.86, 0.95, and 0.91 for self-devaluation, social withdrawal, public stigma, and family stigma, respectively.

Based on the evidence that a stigmatizing perception of infertility is associated with greater experience of negative emotions such as depression and anxiety [ 20 ], the Korean version of the Center for Epidemiologic Studies Depression Scale-Revised (K-CESD-R) [ 21 ] was used for known-group validity. The scale measures the severity of depressive symptoms according to four levels based on the frequency of the symptoms experienced during the past week. Each item receives 0 point for occurring never or rarely (during less than 1 day), 1 point for occurring some or a little of the time (1–2 days), 2 points for occurring occasionally or a moderate amount of time (3–4 days), and 3 points for occurring most or all of the time (5–7 days). The total scores for all the items of 0–15, 16–24, and 25–60 points were defined as normal, probable depression, and definite depression, respectively. According to Cho & Kim [ 22 ], the reliability of the scale was indicated by Cronbach’s alpha = 0.90 in the normal group ( N  = 540), 0.93 in the clinical patient group ( N  = 164), and 0.89 in the major depression group ( N  = 46). In this study, the reliability of the scale was indicated by Cronbach’s alpha = 0.94.

Collected data were analyzed using SPSS version 25.0 program (IBM Corp., Armonk, NY, USA). The general and infertility-related characteristics of the subjects were analyzed by descriptive statistics using the frequency, percentage, mean, and standard deviation (SD). Item analysis was performed using mean, SD, skewness, and kurtosis. Moreover, item-total correlation analysis was performed, and items with an item-total correlation coefficient inferior to 0.30 were reviewed and deleted [ 23 ]. The construct validity was tested by EFA and the appropriateness of factor analysis was identified using the Kaiser-Meyer-Olkin (KMO) test and Bartlett’s test of sphericity. Furthermore, principal component analysis was used for factor extraction, while varimax rotation was used for factor rotation. For determination of the number of factors, eigenvalue ≥ 1, factor loading ≥ 0.50, commonality ≥ 0.40, and scree plot were considered [ 24 ].

The concurrent validity was tested using the Pearson’s correlation between the developed instrument and ISS [ 12 ], while the known-groups validity was tested using K-CESD-R [ 26 ]. Meanwhile, the differences in the infertility perception scores among the normal, probable depression, and definite depression groups were analyzed using one-way ANOVA, while a post-hoc test was performed using Scheffe’s test. Cronbach’s alpha was calculated to test the reliability.

General characteristics

The subjects’ age averaged 37.03 ± 4.56. The most response to cause of infertility was “unexplained” (57.1%). Most subjects (83.7%) had no children. The current treatments were IVF (57.1%) and IUI (42.9%). The 31.5% considered the treatment as moderately affordable, 15.8% viewed it as hardly affordable. The 11.3% had experience counseling for infertility. The 68.5% answered that their spousal support to infertility treatment was passive (Table  1 ).

Item analysis

Analysis of the items in the scale showed that the mean value was 1.27-3.00 and the SD was 0.48–0.97. The absolute skew and kurtosis values were 0.03–1.93 and 0.07–4.32, respectively, satisfying the criteria of an absolute skew value ≤ 2.0 and absolute kurtosis value ≤ 7.0 [ 25 ]. Accordingly, since multivariate normality was confirmed, all items were used in the analysis. The item-total correlation coefficient was 0.04–0.77, with 12 out of 41 items (items #6, 8, 9, 15, 22, 25, 28, 33, 36, 37, 38, and 39) showing an item-total correlation coefficient ≤ 0.30. Since these items were assessed as offering little contribution to the scale, they were deleted, leaving remaining items in the scale.

Validity testing

Construct validity testing.

A KMO value of 0.91 for 29 items and a Bartlett’s test of sphericity χ² value of 3137.40 ( p  < .001) wear measured, confirming that the data were appropriate for factor analysis. After the first EFA, two items that were double-loaded on two factors (items #30 and 31) were deleted. After the second EFA on the remaining 27 items, three items that were double-loaded on two factors (items #12, 29, and 35) and three items with factor loading < 0.50 (items #1, 23, and 34) were deleted. Subsequently, the third EFA was performed using 21 items. The results showed a KMO value of 0.89 and a Bartlett’s test of sphericity χ² value of 1997.07 ( p  < .001). Moreover, the commonality was 0.40–0.81, factor loading was 0.51–0.89, and there were four factors with eigenvalue > 1, which had a cumulative explanatory power of 59.3% (Table  2 ).

Four factors extracted according to infertility perception explained 59.3% of the total variance. In social science, an explanation of 40–60% of the variance for multifactor patterns is considered to be sufficient [ 26 ]. Factor-1 was named “perceived feelings,” consisting of six items explaining 34.6% of the total variance. Factor-2 was named “personal stigma,” consisting of eight items explaining 12.0% of the total variance. Factor-3 was named “social stigma,” consisting of three items explaining 6.6% of the total variance. Factor-4 was named “acceptance,” consisting of four items explaining 6.1% of the total variance.

According to the results of the exploratory factor analysis, the 21-item IPS-W with 4 factors underwent confirmatory factor analysis. The model fit was evaluated against predefined cutoff values for each fit index (χ 2 /df ≤ 3, AGFI ≥ 0.90, GFI ≥ 0.90, CFI ≥ 0.90, RMSEA < 0.08) [ 27 ]. The model achieved the following fit indices: χ 2 /df = 2.09, AGFI = 0.81, GFI = 0.85, CFI = 0.90, RMSEA = 0.07 (Fig.  1 ).

Measurement model of an infertility perception scale for women (IPS-W). χ 2 /df = 2.09, AGFI = 0.81, GFI = 0.85, CFI = 0.90, RMSEA = 0.07 χ 2 /df = Chi-square divided by degrees of freedom; AGFI = Adjustedm Goodness of Fit Index; GFI = Goodness of Fit Index; CFI = Comparative Fit Index; RMSEA = Root Mean Square Error of Approximation

Multi-trait/multi-item matrix analysis was performed. Convergent validity was validated since the correlation between each item and the total score in the corresponding subscale was 0.59–0.89, which satisfied the cut-off value of 0.40. Moreover, discriminant validity was also validated since the value derived by subtracting twice the standard error from the correlation coefficient between the item and the corresponding subscale was larger than the correlation coefficient of other sub-components.

Criterion validity testing

Concurrent validity was first tested based on the correlation of the scale with ISS. The results showed a positive correlation with a correlation coefficient of 0.63 ( p  < .001), while also showing positive correlations between sub-factors with correlation coefficients ranging between 0.30 and 0.54. Accordingly, the concurrent validity of the scale was validated (Table  3 ).

Secondly, the known-groups validity was tested by dividing the subjects into groups by the level of depression based on CES-D cut-off points and analyzing the differences in infertility perception among the groups. Since the results showed significant differences in infertility perception according to the level of depression, the known-groups validity was validated (F = 18.84, p  < .001; Table  4 ).

Reliability testing

The Cronbach’s alpha value of 21 infertility perception items was 0.90, and for each sub-domain, the Cronbach’s alpha value was 0.91, 0.85, 0.77, and 0.60 for Factor-1, 2, 3, and 4, respectively. According to the rationale by Hair et al. [ 19 ], a reliability ≥ 0.70 is considered acceptable for any new instrument (Table  2 ).

Factor-1 (perceived feelings) represents overall feelings about infertility, having a total explanatory power of 34.6%. Infertility not only causes psychological distress, such as sadness and loss of hope for the future [ 4 ], but it is also defined as an infertility crisis accompanied by physical, economic, and social stress [ 28 ]. The 50% of women considered this process as the most stressful experience in their life [ 29 ], while 84.2% of Korean women receiving infertility treatment experienced depression [ 4 ]. In Factor-1, the overall feeling about infertility was reflected by negative emotions including a sense of loss, anxiety, despair, depression, sadness, and guilt. Therefore, efforts are needed to understand the negative emotions of people who experience infertility and to alleviate these emotions.

Factor-2 refers to “personal stigma.” Infertility can be seen as the fault of the individual, while individuals who experience infertility perceive it as a void and a problem that is difficult to reveal. Personal stigma refers to the extent to which a person believes that negative stereotypes associated with a group they belong to will also be applied to them, while experiencing more personal stigma results in increased self-stigma, which is the feeling that the stigma is applicable to themselves [ 30 ]. Factor-2 reflects previous reports that women who marry but have not given birth are stigmatized as not fulfilling the role of a married woman and denying their own femininity [ 31 ]. Moreover, infertility diagnosis and treatment endlessly give women existential angst and hoping for a child is perceived as the attempt of women to understand their own existence even if they realize that such hope is in vain [ 32 ].

Infertility is understood differently depending on the socio-cultural context. A significant number of women diagnosed with infertility in Korea experience embarrassment and despair from their unexpected difficulties with fertility. Originally, they thought that having a child would be a natural process after getting married, and thus experience confusion about their sense of identity as a woman when this does not happen [ 33 , 34 ]. The perceptions of the Korean society surrounding infertility has been emphasized as an important factor influencing the sense of identity and emotions of individuals with infertility [ 35 ]. In other words, individuals currently undergoing infertility treatment have always perceived infertility negatively, rather than positively, as a member of the society before being diagnosed with infertility themselves. Such perception becomes palpable in their own lives once they are diagnosed with infertility, which adds stigmatizing characteristics to their perception of infertility.

Factor-3 (“social stigma”) includes stigmatizing characteristics about infertility at the societal level, believing that childbirth are an essential social rite of passage. But as they face difficulties with fertility, the views of other people become uncomfortable to women, leading them to withdraw from relationships. This could be interpreted as a factor similar to the “social withdrawal” mentioned by Fu et al. [ 12 ]. This process is well explained by studies reporting that, under the ideology of the Korean society, which regards motherhood as the main definer of a woman’s identity and has favorable views on having children, having friends or relatives that perpetrate a negative stigma around infertility and take a cruel view of women regardless of whether they are directly responsible for the infertility [ 36 ] can cause women with infertility to withdraw from relationships [ 33 ]. In fact, over 50% of Korean women who received infertility treatment experienced prejudice due to infertility and 43% reported serious withdrawal from relationships for this reason [ 34 ].

As previously described, infertility is not well understood and not adequately perceived within many societies [ 37 , 38 ]. Negative social perception about infertility can cause women who experience infertility to internalize social stigma, withdraw from interpersonal relationships, lose self-esteem, and experience decline in quality of life [ 14 ]. All items included in Factors 2 and 3 are close to problems stemming from perceived stigma, meaning one’s own belief that the society views them as a member of a stigmatized group [ 39 ].

Factor-4 reflects “acceptance,” whereby infertility is perceived as an acceptable problem that can be overcome together as a couple, rather than a problem that causes a negative outcome in life. Having no child due to infertility and the surrounding stigma can place a serious burden on the couple’s relationship and the difficulties may destroy the marriage or strengthen their bond [ 40 ]. The perception of infertility is deeply associated with the socio-cultural context due to the longstanding family norms in Korea. Recognizing infertility as a problem that can be overcome and understanding that individuals and the society must work together to resolve the problem.

Meanwhile, Factor-4 had a somewhat low reliability score of 0.60. This could be due to large differences in how positively individuals receiving infertility treatment think about infertility within their socio-cultural and familial context and how significant are their psycho-emotional changes, which may vary according to treatment stage, frequency of treatment, and outcomes during treatment (number of follicles, quality of eggs, number of embryos, quality of embryos, etc.). In the future, it may be necessary to repeat the measurements after unifying the infertility treatment on IUI or IVF or using the same measurement point like the start of treatment rather than during treatment to reduce the factors possibly acting as variables.

Validity and reliability

Concurrent validity test of the developed scale and ISS suggested that a more negative perception of infertility led to a higher perceived stigma about infertility [ 16 , 41 ]. Moreover, known-groups validity for comparing the level of infertility perception between groups showed that the depression groups perceived infertility more negatively than the normal group, consistent with a previous study [ 20 ], which validated the known-groups validity of the scale developed in the present study. The reliability of our scale corresponded to a Cronbach’s alpha of 0.90, indicating that the reliability was at an acceptable level and that all items were easily understood by women receiving infertility treatment.

Up to now, while there have been efforts to measure infertility perception, such efforts have focused mostly on the individual’s knowledge about infertility diagnosis and treatment or the stigmatizing characteristics of infertility within various socio-cultural contexts. In Korea, in particular, infertility has usually been perceived as having stigmatizing characteristics such as insufficiency and difficulty, but it is important to view infertility as a problem that can be overcome by working together, rather than a negative life event that can destroy the sense of identity of women and cause relationship problems among couples. The newly developed infertility perception scale also includes a significant number of the stigmatizing characteristics of infertility. Hence, efforts are needed to solve the issues surrounding inadequate beliefs through the understanding of the stigmatizing characteristics of infertility in order to transition towards a positive perception of infertility. Infertility not only causes individuals to experience various negative emotions but also impacts their life cycle, potentially increasing the risk of conditions such as coronary heart disease (CHD) [ 42 ] and elevating the likelihood of early menopause [ 43 ]. Therefore, a positive perception of infertility by those affected directly influences the individual, playing a crucial role in maintaining and promoting overall health.

Strengths, limitations, and future research

This study convenience sampled women receiving infertility treatment at two Korean hospitals specializing in infertility treatment, limiting the generalization of the findings. In addition, although we selected two institutions with similar treatment processes and patient education, there is a limitation in that we were unable to systematically analyze the similarities and differences. Prior to data collection, the authors informed the study participants that the survey was anonymous and that their responses would be kept confidential. However, there is a limitation that, due to social stigma, participants might have underreported or misreported their true opinions. Moreover, Cultural differences across countries can significantly influence perceptions of infertility, where what may be considered stigma or acceptable responses in one culture may not be the same in another. Therefore, it is essential to assess the validity and reliability across diverse populations, including different ethnic groups, and adjust the questionnaire accordingly to ensure cultural relevance and applicability. In future research, it is recommended that the validity and reliability of the scale be tested only among women who are at the IVF stage, as IVF is in the final stage of assisted reproductive therapy. Additionally, follow-up studies should aim to identify the level of infertility perception and to develop and test the impact of nursing interventions.

The infertility perception scale developed in the present study could predict the psycho-emotional state of women receiving infertility treatment by examining their infertility perception. These findings could be used for the development of nursing interventions that could help women experiencing infertility to approach positively the problem of infertility.

Conclusions

We developed and tested an infertility perception scale consisting in a 4-point Likert scale with 21 items under four factors. The scale has a score range of 21–84 points, with higher scores indicating more negative perceptions of infertility. The scale can be used to measure infertility perception among women who are experiencing infertility and their spouse, family, and friends. The findings can help nurses provide support that is appropriate for individual circumstances by examining how women experiencing infertility perceive infertility.

Data availability

Datasets used/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Infertility Perception Scale for Women

Intrauterine insemination

in-Vitro Fertilization

Exploratory Factor Analysis

Confirmatory Factor Analysis

Female Infertility Stigma Instrument

Infertility Stigma Scale

Strengthening the Reporting of Observational Studies in Epidemiology

Content Validity Index

Item-Content Validity Index

Korean version of the Center for Epidemiologic Studies Depression Scale-Revised

Kaiser-Meyer-Olkin

Root-mean-square error of approximation

Adjusted Goodness-of-Fit Index

Goodness-of-Fit Index

Comparative Fit Index

Lee YH, Park JS. Factors affecting the infertility-related quality of life among the infertility women. J Korean Soc Maternal Child Health. 2019;23(3):191–201. https://doi.org/10.21896/jksmch.2019.23.3.191 .

Article   Google Scholar  

Podolska M, Bidzan M. Infertility as a psychological problem. Ginekologia Polska. 2011;82(1):44–9.

PubMed   Google Scholar  

Covington SN, Burns LH, editors. Infertility counseling: a comprehensive handbook for clinicians. Cambridge University Press; 2006.

Hwang NM. Factors related to the depression of infertile women. Health Social Welf Rev. 2013;33(3):161–87.

Taebi M, Kariman N, Montazeri A, et al. Development and psychometric evaluation of the female infertility stigma instrument (ISI-F): protocol for a mixed method study. Reproductive Health. 2020;17:70. https://doi.org/10.1186/s12978-020-0904-5 .

Article   PubMed   PubMed Central   Google Scholar  

Amiri M, Khosravi A, Chaman R, Sadeghi Z, Raei M, Jahanitiji MA, Mehrabian F. Social consequences of infertility on families in Iran. Global J Health Sci. 2016;8(5):89–94. https://doi.org/10.5539/gjhs.v8n5p89 .

Ergin RN, Polat A, Kars B, Öztekin D, Sofuoğlu K, Çalışkan E. Social stigma and familial attitudes related to infertility. Turkish J Obstet Gynecol. 2018;15(1):46–9. https://doi.org/10.4274/tjod.04307 .

Harzif AK, Santawi VPA, Wijaya S. Discrepancy in perception of infertility and attitude towards treatment options: Indonesian urban and rural area. Reproductive Health. 2019;16:126. https://doi.org/10.1186/s12978-019-0792-8 .

Meissner C, Schippert C, von Versen-Höynck F. Awareness, knowledge, and perceptions of infertility, fertility assessment, and assisted reproductive technologies in the era of oocyte freezing among female and male university students. J Assist Reprod Genet. 2016;33:719–29. https://doi.org/10.1007/s10815-016-0717-1 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Jansen NA, Saint Onge JM. An internet forum analysis of stigma power perceptions among women seeking fertility treatment in the United States. Soc Sci Med. 2015;147:184–9. https://doi.org/10.1016/j.socscimed.2015.11.002 .

Article   PubMed   Google Scholar  

Naab F, Brown R, Heidrich S. Psychosocial health of infertile Ghanaian women and their infertility beliefs. J Nurs Scholarsh. 2013;45(2):132–40. https://doi.org/10.1111/jnu.12013 .

Fu B, Qin N, Cheng L, Tang G, Cao Y, Yan C, Huang X, Yan P, Zhu S, Lei J. Development and validation of an infertility stigma scale for Chinese women. J Psychosom Res. 2015;79(1):69–75. https://doi.org/10.1016/j.jpsychores.2014.11.014 .

Introduction to perception. Retrieved from https://courses.lumenlearning.com/boundless-psychology/chapter/introduction-to-perception/

Gong JS. The effect of ex-prisoner’s perception of stigma on trust in international relationship. Korean Secur J. 2018;57:57–84.

Google Scholar  

DeVellis RF. Scale development: theory and applications. 4th ed. Sage; 2016. pp. 39–199.

Seifer DB, Petok WD, Agrawal A, Glenn TL, Bayer AH, Witt BR, Burgin BD, Lieman HJ. Psychological experience and coping strategies of patients in the Northeast US delaying care for infertility during the COVID-19 pandemic. Reproductive Biology Endocrinol. 2021;19(1):28. https://doi.org/10.1186/s12958-021-00721-4 .

Article   CAS   Google Scholar  

Krippendorff K. Content analysis: an introduction to its methodology. 2nd ed. Sage; 2004. pp. 1–413.

Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35(6):382–5.

Article   CAS   PubMed   Google Scholar  

Hair JF, Black WC, Babin BJ, Anderson RE. Multivariate data analysis: a global perspective. 7th ed. Pearson Education; 2010. pp. 111–3.

Davis M, Ventura JL, Wieners M, Covington SN, Vanderhoof VH, Ryan ME, et al. The psychosocial transition associated with spontaneous 46, XX primary ovarian insufficiency: illness uncertainty, stigma, goal flexibility, and purpose in life as factors in emotional health. Fertil Steril. 2010;93(7):2321–9. https://doi.org/10.1016/j.fertnstert.2008.12.122 .

Lee S, Oh ST, Ryu SY, Jun JY, Lee K, Lee E, et al. Validation of the Korean version of Center for epidemiologic studies Depression Scale-revised (K-CESD-R). Korean J Psychosom Med. 2016;24(1):83–93.

Cho MJ, Kim KH. Diagnostic validity of the CES-D (Korean version) in the assessment of DSM-III-R major depression. J Korean Neuropsychiatric Association. 1993;32(3):381–99.

Lee CY. (2016). Advanced nursing statistics (pp. 231–264). Soomunsa.

Noh GS. (2019). The proper methods of statistical analysis for dissertation: SPSS & AMOS 21 (pp. 141–193). Hanbit Academy.

Curran PJ, West SG, Finch JF. The robustness of test statistics to non-normality and specification error in confirmatory factor analysis. Psychol Methods. 1996;1(1):16–29. https://doi.org/10.1037/1082-989X.1.1.16 .

Sternke EA, Abrahamson K. Perceptions of women with infertility on stigma and disability. Sex Disabil. 2015;33:3–17. https://doi.org/10.1007/s11195-014-9348-6 .

Moon SB. Understanding and application of structural equation modeling. Seoul: Hakjisa; 2009.

Faramarzi M, Pasha H, Esmailzadeh S, Kheirkhah F, Heidary S, Afshar Z. The effect of cognitive behavioral therapy and pharmacotherapy on infertility stress: a randomized controlled trial. Int J Fertility Steril. 2013;7(3):199–206.

Herrmann D, Scherg H, Verres R, Von Hagens C, Strowitzki T, Wischmann T. Resilience in infertile couples acts as a protective factor against infertility-specific distress and impaired quality of life. J Assist Reprod Genet. 2011;28(11):1111–7. https://doi.org/10.1007/s10815-011-9637-2 .

Muñoz M, Finnegan K, Zeladita J, Caldas A, Sanchez E, Callacna M, et al. Community-based DOT-HAART accompaniment in an urban resource-poor setting. AIDS Behav. 2010;14(3):721–30. https://doi.org/10.1007/s10461-009-9559-5 .

Jeong KI. A study on hermeneutic grounded theory of women who experience infertility suffering and relief from pain. Korean J Social Welf Stud. 2014;45(4):5–35. https://doi.org/10.16999/KASWS.2014.45.4.5 .

Boz I, Okumus H. The everything about the existence experiences of Turkish women with infertility: solicited diaries in qualitative research. J Nurs Res. 2017;25(4):268–75. https://doi.org/10.1097/jnr.0000000000000166 .

Yang SN. The experience of infertility treatments among working women without children. Korean J Social Welf Educ. 2019;48:93–120. https://doi.org/10.1111/j.1467-9566.2009.01213.x .

Jeong YM, Kang SK. A phenomenological study on infertile experiences of women: Waiting for Meeting. Stud Life Cult. 2017;44:271–319.

Ryu SJ. A phenomenological study on infertility as the missing piece in socialization process: focusing on university students’ perception of infertility. Family Cult. 2021;33(2):38–65. https://doi.org/10.21478/family.33.2.202106.002 .

Ryu SJ. Infertility, the social context of deferred motherhood: a qualitative inquiry based on an online infertility community. J Asian Women. 2019;58(2):5–38.

Bunting L, Boivin J. Knowledge about infertility risk factors, fertility myths and illusory benefits of healthy habits in young people. Hum Reprod. 2008;23(8):1858–64. https://doi.org/10.1093/humrep/den168 .

Fido A. Emotional distress in infertile women in Kuwait. Int J Fertil Women’s Med. 2004;49(1):24–8.

LeBel TP. Perceptions of and responses to stigma. Sociol Compass. 2008;2(2):409–32. https://doi.org/10.1111/j.1751-9020.2007.00081.x .

Valsangkar S, Bodhare T, Bele S, Sai S. An evaluation of the effect of infertility on marital, sexual satisfaction indices and health-related quality of life in women. J Hum Reproductive Sci. 2011;4(2):80–5. https://doi.org/10.4103/0974-1208.86088 .

Sobral MP, Costa ME, Schmidt L, Martins MV. COMPI fertility problem stress scales is a brief, valid and reliable tool for assessing stress in patients seeking treatment. Hum Reprod. 2017;32(2):375–82. https://doi.org/10.1093/humrep/dew315 .

Farland LV, Wang YX, Gaskins AJ, Rich-Edwards JW, Wang S, Magnus MC, et al. Infertility and risk of cardiovascular disease: a prospective cohort study. J Am Heart Association. 2023;12(5):e027755. https://doi.org/10.1161/JAHA.122.027755 .

Liang C, Chung HF, Dobson AJ, Cade JE, Greenwood DC, Hayashi K, et al. Is there a link between infertility, miscarriage, stillbirth, and premature or early menopause? Results from pooled analyses of 9 cohort studies. Am J Obstet Gynecol. 2023;229(1):47e1. 47.e9.

Download references

Acknowledgements

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2019R1F1A106291912).

Author information

Authors and affiliations.

Department of Nursing, College of Nursing, Dankook University, Cheonan, South Korea

Department of Nursing, College of Nursing, Catholic University of Pusan, Busan, South Korea

Minkyung Ban

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization or/and Methodology: Kim Miok. Data curation or/and Analysis: Ban Minkyung. Funding acquisition: Kim MiokInvestigation: Ban Minkyung. Project administration or/and Supervision: Kim Miok. Resources or/and Software: Kim Miok, Ban Minkyung. Validation: Kim Miok. Visualization: Kim Miok, Ban Minkyung. Writing: original draft or/and review & editing: Kim Miok, Ban Minkyung.

Corresponding author

Correspondence to Minkyung Ban .

Ethics declarations

Ethics approval.

This study obtained an approval from the Institutional Review Board (IRB) of Dankook University (IRB approval no.: DKU 2021-01-022). Consent was obtained from the subjects after informing them about the purpose and procedures of the study and the protection of personal information. Only those who voluntarily consented to participate in the study were included. The subjects were given a small token of appreciation for their participation.

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Kim, M., Ban, M. Development of an infertility perception scale for women (IPS-W). BMC Women's Health 24 , 513 (2024). https://doi.org/10.1186/s12905-024-03336-0

Download citation

Received : 21 September 2023

Accepted : 27 August 2024

Published : 14 September 2024

DOI : https://doi.org/10.1186/s12905-024-03336-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Infertility, female
• Social stigma
• Factor analysis, statistical

BMC Women's Health

ISSN: 1472-6874

• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.9(10); 2019

Questionnaire validation practice: a protocol for a systematic descriptive literature review of health literacy assessments

Melanie hawkins.

1 School of Health and Social Development, Faculty of Health, Deakin University, Burwood, Victoria, Australia

Gerald R Elsworth

Richard h osborne.

2 Global Health and Equity, Faculty of Health, Arts and Design, Swinburne University of Technology, Hawthorn, Victoria, Australia

Associated Data

bmjopen-2019-030753supp001.pdf

Introduction

Contemporary validity testing theory holds that validity lies in the extent to which a proposed interpretation and use of test scores is justified, the evidence for which is dependent on both quantitative and qualitative research methods. Despite this, we hypothesise that development and validation studies for assessments in the field of health primarily report a limited range of statistical properties, and that a systematic theoretical framework for validity testing is rarely applied. Using health literacy assessments as an exemplar, this paper outlines a protocol for a systematic descriptive literature review about types of validity evidence being reported and if the evidence is reported within a theoretical framework.

Methods and analysis

A systematic descriptive literature review of qualitative and quantitative research will be used to investigate the scope of validation practice in the rapidly growing field of health literacy assessment. This review method employs a frequency analysis to reveal potentially interpretable patterns of phenomena in a research area; in this study, patterns in types of validity evidence reported, as assessed against the criteria of the 2014 Standards for Educational and Psychological Testing , and in the number of studies using a theoretical validity testing framework. The search process will be consistent with the Preferred Reporting Items for Systematic Reviews and Meta-analyses statement. Outcomes of the review will describe patterns in reported validity evidence, methods used to generate the evidence and theoretical frameworks underpinning validation practice and claims. This review will inform a theoretical basis for future development and validity testing of health assessments in general.

Ethics and dissemination

Ethics approval is not required for this systematic review because only published research will be examined. Dissemination of the review findings will be through publication in a peer-reviewed journal, at conference presentations and in the lead author’s doctoral thesis.

Strengths and limitations of this study

• This is the first systematic literature review to examine types of validity evidence for a range of health literacy assessments within the framework of the authoritative reference for validity testing theory, The Standards for Educational and Psychological Testing .
• The review is grounded in the contemporary definition of validity as a quality of the interpretations and inferences made from measurement scores rather than as solely based on the properties of a measurement instrument.
• The search for the review will be limited only by the end search date (March 2019) because health literacy is a relatively new field and publications are not expected prior to about 30 years ago.
• All definitions of health literacy and all types of health literacy assessment instruments will be included.
• A limitation of the review is that the search will be restricted to studies published and instruments developed in the English language, and this may introduce an English language and culture bias.

Historically, the focus of validation practice has been on the statistical properties of a test or other measurement instrument, and this has been adopted as the basis of validity testing for individual and population assessments in the field of health. 1 However, advancements in validity testing theory hold that validity lies in the justification of a proposed interpretation of test scores for an intended purpose, the evidence for which includes but is not limited to the test’s statistical properties. 2–7 Therefore, to validate means to investigate , through a range of methods, the extent to which a proposed interpretation and use of test scores is justified. 7–9 The term ‘test’ in this paper is used in the same sense as Cronbach uses it in his 1971 Test Validation chapter 8 to refer to all procedures for collecting data about individuals and populations. In health, these procedures include objective tests (eg, clinical assessments) and subjective tests (eg, patient questionnaires) or a combination of both and may involve quantitative (eg, questionnaire) or qualitative methods (eg, interview). The act of testing results in data that require interpretation. In the field of health, such interpretations are usually used for making decisions about individuals or populations. The process of validation needs to provide evidence that these interpretations and decisions are credible, and a theoretical framework to guide this process is warranted. 1 2 10

The authoritative reference for validity testing theory comes from education and psychology: the Standards for Educational and Psychological Testing (the Standards ). 3 The Standards define validity as ‘the degree to which evidence and theory support the interpretations of test scores for proposed uses of tests’ and that ‘the process of validation involves accumulating relevant evidence to provide a sound scientific basis for the proposed score interpretations’ (p.11). 3 A test’s proposed score interpretation and use is described in Kane’s argument-based approach to validation as an interpretation/use argument (IUA; also called an interpretive argument). 11 12 Validity testing theory requires test developers and users to generate and evaluate a range of validity evidence such that a validity argument can determine the plausibility of the IUA. 3 7 9 11 12 Despite this contemporary stance on validity testing theory and practice, the application of validity testing theory and methodology is not common practice for individual and population assessments in the field of health. 1 Furthermore, there are calls for developers, users and translators/adapters of health assessments to establish theoretically driven validation plans for IUAs such that validity evidence can be systematically collected and evaluated. 1 2 7 10

The Standards provide a theoretical framework that can be used or adapted to form a validation plan for development of a new test or to evaluate the validity of an IUA for a new context. 1 2 Based on the notion that construct validity is the foundation of test development and use, the theoretical framework of the Standards outlines five sources of evidence on which validity arguments should be founded: (1) test content, (2) response processes, (3) internal structure, (4) relationship of scores to other variables and (5) validity and the consequences of testing ( table 1 ). 3

The five sources of validity evidence 3

Validity testing in the health context

Two of the five sources of validity evidence defined by the Standards (internal structure and relationship of scores to other variables) have a focus on the statistical properties of a test. However, the other three (test content, response processes and consequences of testing) are strongly reliant on evidence based on qualitative research methods. Greenhalgh et al have called for more credence and publication space to be given to qualitative research in the health sciences. 13 Zumbo and Chan (p.350, 2014) call specifically for more validity evidence from qualitative and mixed methods. 1 It is time to systematically assess if test developers and users in health are generating and integrating a range of quantitative and qualitative evidence to support inferences made from these data. 1

In chapter 1 of their book, Zumbo and Chan report the results of a systematic search of validation studies from the 1960s to 2010. Results from this search for the health sciences categories of ‘life satisfaction, well-being or quality of life’ and ‘health or medicine’, show that there is a dramatic increase in publication of validation studies since the 1990s that produce primarily what is classified as construct validity. 1 Given this was a snapshot review of validation practice during these years, the authors do not delve into the methods used to generate evidence for construct validity. However, Barry et al , in a systematic review investigating the frequency with which psychometric properties were reported for validity and reliability in health education and behaviour (also published in 2014), found that the primary methods used to generate evidence for construct validity were factor analysis, correlation coefficient and χ 2 . 14 This limited view of construct validity as simply correlation between items or tests measuring the same or similar constructs is at odds with the Standards where evaluation and integration of evidence from perhaps several other sources (ie, test content, response processes, internal structure, relationships with theoretically predicted external variables, and intended and unintended consequences) is needed to determine the degree to which a construct is represented by score interpretations (p.11). 3

Health literacy

This literature review will examine validity evidence for health literacy assessments. Health literacy is a relatively new area of measurement, and there has been a rapid development in the definition and measurement of this multi-dimensional concept. 15–18 Health literacy is now a priority of the WHO, 19 and many countries have incorporated it into health policy, 20–24 and are including it in national health surveys. 25–27

Definitions of health literacy include those for functional health literacy (ie, a focus on comprehension and numeric abilities) to multi-dimensional definitions such as that used by the WHO: ‘the cognitive and social skills which determine the motivation and ability of individuals to gain access to, understand and use information in ways which promote and maintain good health’. 28 The general purpose of health literacy assessment is to determine pathways to facilitate access to and improve understanding and use of health information and services, as well as to improve or support the health literacy responsiveness of health services. 28–31 However, these two uses of data (in general, to improve patient outcomes and to improve organisational procedures) may require evaluative integration of different types of evidence to justify score interpretations to inform patient interventions or organisational change. 3 7 9 11 32 A strong and coherent evidence-based conception of the health literacy construct is required to support score interpretations. 14 33–35 Decisions that arise from measurements of health literacy will affect individuals and populations and, as such, there must be strong argument for the validity of score interpretations for each measurement purpose.

To enhance the quality and transparency of the proposed systematic descriptive literature review, this protocol paper outlines the scope and purpose of the review. 36 37 Using the theoretical framework of the five sources of validity evidence of the Standards , and health literacy assessments as an exemplar, the results of this systematic descriptive literature review will indicate current validation practice. The assumptions that underlie this literature review are that, despite the advancement of contemporary validity testing theory in education and psychology, a systematic theoretical framework for validity testing has not been applied in the field of health, and that validation practice for health assessments remains centred on general psychometric properties that typically provide insufficient evidence that the test is fit for its intended use. The purpose of the review is to investigate quantitative and qualitative validity evidence reported for the development and testing of health literacy assessments to describe patterns in the types of validity evidence reported, 38–45 and identify use of theory for validation practice. Specifically, the review will address the following questions:

• What is being reported as validity evidence for health literacy assessment data?
• Do the studies place the validity evidence within a validity testing framework, such as that offered by the Standards ?

Review method

This review is designed to provide the basis for a critique of validation practice for health literacy assessments within the context of the validity testing framework of the Standards . It is not an evaluation of the specific arguments that authors have made about validity from the data that have been gathered for individual measurement instruments. The review is intended to quantify the types of validity evidence being reported so a systematic descriptive literature review was chosen as the most appropriate review technique. Described by King and He (2005) 42 as belonging towards the qualitative end of a continuum of review techniques, a descriptive literature review nevertheless employs a frequency analysis to reveal interpretable patterns in a research area; such as, in this review, in the types of validity evidence being reported for health literacy assessments and in the number of studies that refer to a validity testing framework. A descriptive literature review can include qualitative and quantitative research and is based on a systematic and exhaustive review method. 38–41 43 44 38 39 The method for this review will be guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. 46

Eligibility criteria

This literature review is not an assessment of participant data but a collation of reported validity evidence. As such, the focus is not on the participants in the studies but on the evidence presented in support of the validity of interpretations and uses of health literacy assessment data. This means that it will be the type of study that is considered for inclusion rather than the type of study participant. Inclusion criteria are as follows:

• Development/application/validation studies about health literacy assessments : We expect to find many papers that describe the development and initial validation studies of health literacy assessments. Papers that use an existing health literacy assessment to measure outcomes but do not claim to conduct validity testing will not be included. Studies of comparison (eg, participant groups) or of prediction (eg, health literacy and hospital admissions) will be included only if the authors openly claim that the study results contribute validation evidence for the health literacy assessment instrument.
• Not limited by date : There will be no start date to the search such that papers about validation and health literacy assessments from the early days of health literacy measurement will be included in the search. Health literacy is a relatively new concept and the earliest papers are expected to date back only about 30 years. The end search date was in March 2019.
• Studies published and health literacy assessments developed in the English language : Due to resource limitations, the search will be restricted to studies published in the English language and instruments developed in the English language. Translated instruments will be excluded. We realise that these exclusions introduce an English language and culture bias, and we recommend that a similar descriptive review of published studies about health literacy assessments developed in or translated to other languages is warranted.
• Qualitative and quantitative research methods : Given that comprehensive validity testing includes both qualitative and quantitative methods, studies employing either or both will be included.
• All definitions of health literacy : Definitions of health literacy have been accumulating over the past 30 years and reflect a range of health literacy testing methods as well as contexts, interpretations and uses of the data. We include all definitions of health literacy and all types of health literacy assessment instruments, which may include objective, subjective, uni-dimensional and multi-dimensional measurement instruments.

Exclusion criteria

Systematic reviews and other types of reviews captured by the search will not be included in the analysis. However, before being excluded, the reference lists will be checked for articles that may have been missed by the database search. Predictive, association or other comparative studies that do not explicitly claim in the abstract to contribute validity evidence will also not be included. Instruments developed in languages other than English, and translation studies, will be excluded as noted previously.

Information sources

Systematic electronic searches of the following databases will be conducted in EBSCOhost: MEDLINE Complete, Global Health, CINAHL Complete, PsycINFO and Academic Search Complete. EMBASE will also be searched. The electronic database search will be supplemented by searching for dissertations and theses through proquest.com, dissertation.com and openthesis.org. Reference lists of pertinent systematic reviews that are identified in the search will be scanned, as well as article reference lists and the authors’ personal reference lists, to ensure all relevant articles have been captured. The search terms will use medical subject headings and text words related to types of assessment instruments, health literacy, validation and validity testing. Peer reviewed full articles and examined theses will be included in the search.

Search strategy

An expert university librarian has been consulted as part of planning the literature search strategy. The strategy will focus on health literacy, types of assessment instruments, validation and validity, and methods used to determine the validity of interpretation and use of data from health literacy assessments. The search terms have been determined through scoping searches and examining search terms from other measurement and health literacy systematic reviews. The database searches were completed in March 2019 and the search terms used are described in online supplementary file 1 .

Supplementary data

Study selection.

Literature search results will be saved and the titles and abstracts downloaded to Endnote Reference Manager X9. Titles and abstracts of the search results will be screened for duplicates and according to the inclusion and exclusion criteria. The full texts of articles that seem to meet the eligibility criteria or that are potentially eligible will then be obtained and screened. Excluded articles and reasons for exclusions will be recorded. The PRISMA flow diagram will be used to document the review process. 46

Data extraction

The data extraction framework will be adapted from tables in Hawkins et al 2 (p.1702) and Cox and Owen (p.254). 47 Data extraction from eligible articles will be conducted by one reviewer (MH) and comprehensively checked by a second reviewer (GE).

Subjective and objective health literacy assessments will be identified along with those that combine objective and subjective items or scales. Data to be extracted will include the date and source of publication; the context of the study (eg, country, type of organisation/institution, type of investigation, representative population); statements about the use of a theoretical validity testing framework; the types of validity evidence reported; the methods used to generate the evidence; and the validation claims made by the authors of the papers, as based on their reported evidence.

Data synthesis and analysis

A descriptive analysis of extracted data, as based on the theoretical framework of the Standards , will be used to identify patterns in the types of validity evidence being reported, the methods used to generate the evidence and theoretical frameworks underlying validation practice. Where possible and relevant to the concept of validity, changes in validation practice and assessment of health literacy over time will be explored. It is possible that one study may use more than one method and generate more than one type of validity evidence. Statements about a theoretical underpinning to the generation of validity evidence will be collated.

Patient and public involvement

Patients and the public were not involved in the development or design of this literature review.

With the increasing use of health assessment data for decision-making, the health of individuals and populations relies on test developers and users to provide evidence for validity arguments for the interpretations and uses of these data. This systematic descriptive literature review will collate existing validity evidence for health literacy assessments developed in English and identify patterns of reporting frequency according to the five sources of evidence in the Standards , and establish if the validity evidence is being placed within a theoretical framework for validation planning. 3 The potential implications of this review include finding that, when assessed against the Standards’ theoretical framework, current validation practice in health literacy (and possibly in health assessment in general) has limited capacity for determining valid score interpretation and use. The Standards’ framework challenges the long-held perception in health assessment that validity refers to an assessment tool rather than to the interpretation of data for a specific use. 48 49

The validity of decisions based on research data is a critical aspect of health services research. Our understanding of the phenomena we research is dependent on the quality of our measurement of the constructs of interest, which, in turn, affects the validity of the inferences we make and actions we take from data interpretations. 6 7 Too often the measurement quality is considered separate to the decisions that need to be made. 6 50 However, questionable measurement (perhaps through use of an instrument that was developed using suboptimal methods, was inappropriately applied or through gaps in validity testing) cannot lead to valid inferences. 3 50 To make appropriate and responsible decisions for individuals, communities, health services and policy development, we must consider the integrity of the instruments, and the context and purpose of measurement, to justify decisions and actions based on the data.

A limitation of the review is that the search will be restricted to studies published and instruments developed in the English language, and this may introduce an English language and culture bias. A similar review of health literacy assessments developed in or translated to other languages is warranted. A further limitation is that we rely on the information authors provide in identified articles. It is possible that some authors have an incomplete understanding of the specific methods they are using and reporting, and may not accurately or clearly provide details on validity testing procedures employed. Documentation for decisions made during data extraction will be kept by the researchers.

Health literacy is a relatively new area of research. We are fortunate to be at the start of a burgeoning field and can include all publications about validity testing of English-language health literacy assessments. The inclusion of the earliest to the most recent publications provides the opportunity to understand changes and advancements in health literacy measurement and methods of analysis since the introduction of the concept of health literacy. Using health literacy assessments as an exemplar, the outcomes of this review will guide and inform a theoretical basis for the future practice of validity testing of health assessments in general to ensure, as far as is possible, the integrity of the inferences made from data for individual and population benefits.

Supplementary Material

Acknowledgments.

The authors acknowledge and thank Rachel West, Deakin University Liaison Librarian, for her expertise and advice during the preparation of this systematic literature review.

Twitter: @4MelanieHawkins

Contributors: MH and RHO conceptualised the research question and analytical plan. Under supervision from RHO, MH led the development of the search strategy, selection criteria, data extraction criteria and analysis method, which was then comprehensively assessed and checked by GRE. MH drafted the initial manuscript and led subsequent drafts. GRE and RHO read and provided feedback on manuscript iterations. All authors approved the final manuscript. RHO is the guarantor.

Funding: MH is funded by a National Health and Medical Research Council (NHMRC) of Australia Postgraduate Scholarship (APP1150679). RHO is funded in part through a National Health and Medical Research Council (NHMRC) of Australia Principal Research Fellowship (APP1155125).

Competing interests: None declared.

Patient consent for publication: Not required.

Ethics approval: Ethics approval is not required for this systematic review because only published research will be examined. Dissemination will be through publication in a peer-reviewed journal and at conference presentations, and in the lead author’s doctoral thesis.

Provenance and peer review: Not commissioned; externally peer reviewed.