Keywords
Nursing
Primary Health Care
Psychometrics
Validity and Reliability
Working Conditions
How to Cite
Abstract
Aim
To assess the content validity of an adapted version of the Practice Environment Scale–Nursing Work Index (PES-NWI) for use in Primary Health Care (PHC).
Methods
A qualitative study was conducted using cognitive interviews. Eight PHC nurses with diverse professional profiles completed the adapted version of the PES-NWI and participated in interviews aimed at exploring their interpretation of the items. The analysis was guided by the Question Appraisal System 99 (QAS-99), supplemented with field notes and transcript verification. Identified issues were categorised into four domains: clarity, required knowledge, assumptions, and response format. These findings informed subsequent revisions to the scale.
Results
Twenty-six of the 34 items (76.5%) were identified as problematic by at least one participant, resulting in a total of 56 coded issues. The majority (87.5%) pertained to clarity, primarily due to ambiguous or imprecise wording. All participants considered the items to be relevant and did not perceive any content omissions. Revisions were undertaken to enhance item clarity and contextual appropriateness. The final version was endorsed by the participants, the research team, and the original scale’s author.
Conclusion
The high proportion of items with clarity-related issues underscores the utility of cognitive interviews in detecting interpretive difficulties and refining item wording. Participants judged the adapted scale to be relevant and comprehensive, supporting its content validity and contextual suitability for PHC settings. This study offers a replicable methodology for assessing content validity from the perspective of the target population.
References
Lake ET. Development of the practice environment scale of the nursing work index. Res Nurs Health. 2002 Jun;25(3):176–88.
Aiken LH, Clarke SP, Sloane DM, Lake ET, Cheney T. Effects of Hospital Care Environment on Patient Mortality and Nurse Outcomes. JONA J Nurs Adm. 2009 Jul;39(7/8):S45–51.
Copanitsanou P, Fotos N, Brokalaki H. Effects of work environment on patient and nurse outcomes. Br J Nurs. 2017;26(3):172–6.
Bahlman‐van Ooijen W, Malfait S, Huisman‐de Waal G, Hafsteinsdóttir TB. Nurses’ motivations to leave the nursing profession: A qualitative meta‐aggregation. J Adv Nurs. 2023 Dec;79(12):4455–71.
Paguio JT, Yu DSF, Su JJ. Systematic review of interventions to improve nurses’ work environments. J Adv Nurs. 2020 Oct;76(10):2471–93.
World Health Organization. State of the world’s nursing 2020: investing in education, jobs and leadership [Internet]. Geneva: World Health Organization; 2020 [cited 2025 Jun 10]. Available from: https://iris.who.int/handle/10665/331677
Swiger PA, Patrician PA, Miltner RS (Susie), Raju D, Breckenridge-Sproat S, Loan LA. The Practice Environment Scale of the Nursing Work Index: An updated review and recommendations for use. Int J Nurs Stud. 2017 Sep;74:76–84.
Chang Y, Chang H, Feng J. Appraisal and evaluation of the instruments measuring the nursing work environment: A systematic review. J Nurs Manag. 2022 Apr;30(3):670–83.
Maassen SM, Weggelaar Jansen AMJW, Brekelmans G, Vermeulen H, Van Oostveen CJ. Psychometric evaluation of instruments measuring the work environment of healthcare professionals in hospitals: a systematic literature review. Int J Qual Health Care. 2020 Nov 9;32(8):545–57.
Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res. 2018 May;27(5):1159–70.
Mokkink L, Prinsen C, Patrick D, Alonso J, Bouter L, de Vet H, et al. COSMIN Study Design checklist for Patient-reported outcome measurement instruments. [Internet]. Amsterdam: Amsterdam University Medical Centers; 2019 [cited 2025 Jul 11]. Available from: https://www.cosmin.nl/wp-content/uploads/COSMIN-study-designing-checklist_final.pdf
Alzate LCC, Bayer GLA, Squires A. Validation of a Spanish Version of the Practice Environment Scale of the Nursing Work Index in the Colombian Context. Hisp Health Care Int. 2014 Mar 1;12(1):34–42.
Ambani Z, Al-Hamdan Z, Al-Touby S, Ghanim A, Al Jarameez F, Squires A. Content Validation of the Arabic Translation of the Practice Environment Scale of the Nursing Work Index-Revised. J Nurs Meas. 2019 Aug 1;27(2):234–46.
Fuentelsaz-Gallego C, Moreno-Casbas MT, González-María E. Validation of the Spanish version of the questionnaire Practice Environment Scale of the Nursing Work Index. Int J Nurs Stud. 2013 Feb;50(2):274–80.
Orts-Cortés MI, Moreno-Casbas T, Squires A, Fuentelsaz-Gallego C, Maciá-Soler L, González-María E. Content validity of the Spanish version of the Practice Environment Scale of the Nursing Work Index. Appl Nurs Res. 2013 Nov;26(4):e5–9.
Drennan J. Cognitive interviewing: verbal data in the design and pretesting of questionnaires: Cognitive interviewing. J Adv Nurs. 2003;42(1):57–63.
Willis GB. Cognitive interviewing: a tool for improving questionnaire design. Thousand Oaks, Calif: Sage Publications; 2005. 335 p.
Wright J, Moghaddam N, Dawson DL. Cognitive interviewing in patient-reported outcome measures: A systematic review of methodological processes. Qual Psychol. 2021 Feb;8(1):2–29.
Hirschey R, Nance J, Wangen M, Bryant AL, Wheeler SB, Herrera J, et al. Using Cognitive Interviewing to Design Interventions for Implementation in Oncology Settings. Nurs Res. 2021 May;70(3):206–14.
Rajan D, Rouleau K, Winkelmann M, Kringos D, Khalid F. Implementing the Primary Health Care Approach: a Primer [Internet]. Geneve: World Health Organization; 2024 [cited 2025 Jul 11]. (Global Report on Primary Health Care). Report No.: Volume 1. Available from: https://eurohealthobservatory.who.int/publications/i/implementing-the-primary-health-care-approach-a-primer
Saint-Pierre C, Herskovic V, Sepúlveda M. Multidisciplinary collaboration in primary care: a systematic review. Fam Pract. 2018 Mar 27;35(2):132–41.
de Pedro Gómez J, Morales-Asencio J, Sesé Abad A, Bennasar-Veny M, Artigues Vives G, Pericàs Beltran J. Validación y adaptación al español de la escala del entorno de práctica enfermera del Nursing Work Index. Metas Enferm. 2009;12(7):65–75.
de Pedro-Gómez J, Morales-Asencio JM, Sesé-Abad A, Bennasar-Veny M, Pericas-Beltran J, Miguélez-Chamorro A. Psychometric testing of the Spanish version of the Practice Environment Scale of the Nursing Work Index in a primary healthcare context: PES-NWI in Spanish primary healthcare nurses. J Adv Nurs. 2012 Jan;68(1):212–21.
Gea-Caballero V, Juárez-Vela R, Díaz-Herrera MÁ, Mármol-López MI, Alfaro Blazquez R, Martínez-Riera JR. Development of a short questionnaire based on the Practice Environment Scale-Nursing Work Index in primary health care. PeerJ. 2019 Jul 24;7:e7369.
Lucas P, Jesus E, Almeida S, Araújo B. Validation of the Psychometric Properties of the Practice Environment Scale of Nursing Work Index in Primary Health Care in Portugal. Int J Environ Res Public Health. 2021 Jun 14;18(12):6422.
Boeije H, Willis G. The Cognitive Interviewing Reporting Framework (CIRF): Towards the Harmonization of Cognitive Testing Reports. Methodology. 2013 Aug 1;9(3):87–95.
Zabaleta‐Del‐Olmo E, Rey‐Reñones C, Marchal‐Torralbo A, Sauch‐Valmaña G, Calvet‐Tort G, Planas‐Campmany C, et al. Structural Validity and Reliability of an Adapted Practice Environment Scale‐Nursing Work Index for Primary Health Care Nurses. Res Nurs Health. 2025 Jun 4;nur.22474.
Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the Process of Cross-Cultural Adaptation of Self-Report Measures: Spine. 2000 Dec;25(24):3186–91.
Sousa VD, Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross‐cultural health care research: a clear and user‐friendly guideline. J Eval Clin Pract. 2011 Apr;17(2):268–74.
Willis GB, Lessler JT. Question Appraisal System QAS-99. Rockwille MD: Research Triangle Institute; 1999.
Lessler J, Forsyth B. A coding system for appraising questionnaires. In: Schwarz N, Sudman S, editors. Answering questions: Methodology for determining cognitive and communicative processes in survey research. San Francisco: John Wiley & Sons; 1996. p. 259–91.
Buers C, Triemstra M, Bloemendal E, Zwijnenberg NC, Hendriks M, Delnoij DMJ. The value of cognitive interviewing for optimizing a patient experience survey. Int J Soc Res Methodol. 2014 Jul 4;17(4):325–40.
National Academies of Sciences, Engineering, and Medicine, National Academy of Medicine, Committe on the Future of Nursing 2020-2030. The Role of Nurses in Improving Health Equity. In: Wakefield M, Williams D, Le Menestrel S, Fllaubert J, editors. The Future of Nursing 2020-2030: Charting a Path to Achieve Health Equity. Washington D.C.: National Academies Press (US); 2021.
World Health Organization. Primary Health Care [Internet]. 2025 [cited 2025 Jun 12]. Available from: https://www.who.int/news-room/fact-sheets/detail/primary-health-care
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright (c) 2025 RqR Quantitative and Qualitative Community Nursing Research