A RISK-BASED QUALITY ASSURANCE FRAMEWORK FOR FHIR-BASED CLINICAL APPLICATIONS: ALIGNING VERIFICATION EFFORT WITH CLINICAL CONSEQUENCE
Authors
-
Pavlo Palamarchuk
Palamarchuk Technologies LLC, Sacramento, CA, USA
Author
https://orcid.org/0009-0001-3089-4813
DOI:
https://doi.org/10.69635/mssl.2026.2.4.65Keywords:
FHIR, Risk-Based Testing, Clinical Software Quality Assurance, Patient Safety, Health Information Technology, Interoperability, VerificationAbstract
Background: Healthcare software built on the Fast Healthcare Interoperability Resources (FHIR) standard now mediates prescribing, care coordination, and patient access at national scale, yet the quality assurance practices applied to such systems remain largely undifferentiated: a defect in a practitioner-directory lookup receives the same testing attention as a defect in a medication-dispensing workflow.
Objective: This paper proposes a risk-based quality assurance framework for FHIR-based clinical applications that allocates verification effort in proportion to clinical consequence.
Methods: The framework was constructed through a design-science methodology from three inputs: a case series of production failure modes documented on a HIPAA-regulated telehealth platform; quantified harm evidence from federal patient-safety research, including the finding that computerized provider order entry system-related errors occur in 0.3% to 6.3% of all prescriptions; and risk-management logic adapted from ISO 14971 and healthcare failure mode and effect analysis. The framework’s behavioral layer was then exercised against four FHIR server implementations using an open-source probe as one example realization.
Results: The resulting model classifies FHIR resources and workflows into four risk tiers along three dimensions — clinical consequence class, data-integrity criticality, and failure detectability — and maps each tier to specific verification obligations spanning resource validation, workflow testing, and operational behavior probing. An assistive artificial-intelligence layer for preliminary risk classification is specified with a mandatory human validation gate. Illustrative application demonstrated that the highest-risk behavior identified by the framework — silent data-completeness failure in paginated synchronization — was present on one of the four tested servers while remaining invisible to profile validation and certification testing; the same exercise surfaced a previously undocumented rate-limiting defect, which was reported upstream.
Conclusions: Verification effort for FHIR-based clinical systems can be allocated by clinical consequence rather than by test-suite convention, extending quality obligations into the operational-behavior layer where specification-conformant systems have been shown to fail. Empirical calibration across sites and defect corpora is the necessary next step, and the instrumentation for it is openly available.
References
Agency for Healthcare Research and Quality. (2020). Making healthcare safer III: A critical analysis of existing and emerging patient safety practices (Holmes A, Long A, Wyant B, et al.) (AHRQ Publication No. 20-0029-EF). AHRQ.
Agency for Healthcare Research and Quality. (2023). Making healthcare safer IV: Potential harms resulting from patient-clinician real-time clinical encounters using video-based telehealth: A rapid evidence review (Rosen M, Stewart CM, Kharrazi H, Sharma R, Vass M, Zhang A, Bass EB) (AHRQ Publication No. 23-EHC019-2). AHRQ. https://doi.org/10.23970/AHRQEPC_MHS4TELEHEALTH
Agency for Healthcare Research and Quality. (2024). Making healthcare safer IV: Computerized clinical decision support to prevent medication errors and adverse drug events: Rapid review (Syrowatka A, Motala A, Lawson E, Shekelle PG) (AHRQ Publication No. 24-EHC019-6). AHRQ. https://doi.org/10.23970/AHRQEPC_MHS4MEDERROR
Agency for Healthcare Research and Quality. (2025a). AHRQ-funded patient safety project highlights: Improving healthcare safety by enhancing health information technology and health information exchange. AHRQ. https://www.ahrq.gov/sites/default/files/wysiwyg/patient-safety/highlights/ps-project-highlights-hit-hie-2025.pdf
Agency for Healthcare Research and Quality. (2025b). AHRQ-funded patient safety project highlights: Improving patient safety by enhancing care coordination. AHRQ. https://psnet.ahrq.gov/issue/ahrq-funded-patient-safety-project-highlights-improving-patient-safety-enhancing-care
Augusto, C., Morán, J., Bertolino, A., de la Riva, C., & Tuya, J. (2025). Software system testing assisted by large language models: An exploratory study. In Testing Software and Systems: 36th IFIP WG 6.1 International Conference (ICTSS 2024), London, UK (Lecture Notes in Computer Science, Vol. 15383, Ch. 17). Springer. https://doi.org/10.1007/978-3-031-80889-0_17
Ayaz, M., Pasha, M. F., Alzahrani, M. Y., Budiarto, R., & Stiawan, D. (2021). The Fast Health Interoperability Resources (FHIR) standard: Systematic literature review of implementations, applications, challenges and opportunities. JMIR Medical Informatics, 9(7), e21929. https://doi.org/10.2196/21929
Bates, D. W., Leape, L. L., Cullen, D. J., Laird, N., Petersen, L. A., Teich, J. M., Burdick, E., Hickey, M., Kleefield, S., Shea, B., Vander Vliet, M., & Seger, D. L. (1998). Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA, 280(15), 1311–1316. https://doi.org/10.1001/jama.280.15.1311
Boufahja, A., & Verma, T. (2025). Extensive conformance testing and validation of FHIR data exchange variabilities. In Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC/HEALTHINF) (pp. 297–307). SCITEPRESS. https://doi.org/10.5220/0013370000003911
DeRosier, J., Stalhandske, E., Bagian, J. P., & Nudell, T. (2002). Using health care failure mode and effect analysis: The VA National Center for Patient Safety’s prospective risk analysis system. The Joint Commission Journal on Quality Improvement, 28(5), 248–267. https://doi.org/10.1016/S1070-3241(02)28025-6
Devitt, D. (2026). FHIR architecture decisions: Selecting a FHIR server and choosing between facade, hybrid, and FHIR-native approaches (v1.0) [Self-published practitioner guide]. https://darrendevitt.com/fhir-architecture-decisions-book/
Drummond Group. (2026, August 17). Market study finds barriers to continuous FHIR testing [Industry report]. https://www.drummondgroup.com/blog/market-study-finds-barriers-to-continuous-fhir-testing/
Felderer, M., & Schieferdecker, I. (2014). A taxonomy of risk-based testing. International Journal on Software Tools for Technology Transfer, 16(5), 559–568. https://doi.org/10.1007/s10009-014-0332-3
Hauptfeld, L., Csarmann, A., Grob, M., Liepold, J., Rappelsberger, A., & Adlassnig, K.-P. (2026). Ask not what your server can do for you: A fast query compatibility strategy for navigating FHIR implementation gaps. Studies in Health Technology and Informatics, 338, 658–662. https://doi.org/10.3233/shti260927
Health Resources and Services Administration. (2025). Nurse workforce projections, 2023–2038. National Center for Health Workforce Analysis. https://bhw.hrsa.gov/sites/default/files/bureau-health-workforce/data-research/nursing-projections-factsheet.pdf
Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design science in information systems research. MIS Quarterly, 28(1), 75–105. https://doi.org/10.2307/25148625
HL7 International. (2024). FHIR specification: Validating resources. https://hl7.org/fhir/validation.html
International Electrotechnical Commission. (2015). IEC 62304: Medical device software — Software life cycle processes (Ed. 1.1).
International Organization for Standardization. (2019). ISO 14971: Medical devices — Application of risk management to medical devices.
ISO/IEC/IEEE. (2022). ISO/IEC/IEEE 29119-1: Software and systems engineering — Software testing — Part 1: General concepts (2nd ed.).
Jones, J. R., Gottlieb, D., McMurry, A. J., Atreja, A., Desai, P. M., Dixon, B. E., Payne, P. R. O., Saldanha, A. J., Shankar, P., Solad, Y., Wilcox, A. B., Ali, M. S., Kang, E., Martin, A. M., Sprouse, E., Taylor, D. E., Terry, M., Ignatov, V., & Mandl, K. D. (2024). Real world performance of the 21st Century Cures Act population-level application programming interface. Journal of the American Medical Informatics Association, 31(5), 1144–1150. https://doi.org/10.1093/jamia/ocae040
Khan, M. A., Azim, A., Liscano, R., Smith, K., Chang, Y. K., Seferi, G., & Tauseef, Q. (2024). An end-to-end test case prioritization framework using optimized machine learning models. In 2024 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) (pp. 1–8). IEEE. https://doi.org/10.1109/ICSTW60967.2024.00014
Kostenko, O. (2025). AI law model for ethical legislation: Strategic recommendations for the regulation of artificial intelligence. SciFormat Publishing Inc. https://doi.org/10.69635/978-1-0690482-5-7
Kostenko, O., & Drobakha, A. (2024). Metaverse: A simulacrum platform for digital transformations of the metaverse e-state society. Journal of Educational and Health Studies, (3), 12–20. https://doi.org/10.61345/2734-8873.2025.1.5
Mandel, J. C., Kreda, D. A., Mandl, K. D., Kohane, I. S., & Ramoni, R. B. (2016). SMART on FHIR: A standards-based, interoperable apps platform for electronic health records. Journal of the American Medical Informatics Association, 23(5), 899–908. https://doi.org/10.1093/jamia/ocv189
Peffers, K., Tuunanen, T., Rothenberger, M. A., & Chatterjee, S. (2007). A design science research methodology for information systems research. Journal of Management Information Systems, 24(3), 45–77. https://doi.org/10.2753/MIS0742-1222240302
Sartaj, H., Ali, S., & Gjøby, J. M. (2025). REST API testing in DevOps: A study on an evolving healthcare IoT application. ACM Transactions on Software Engineering and Methodology. Advance online publication. https://doi.org/10.1145/3765744
Sheffer, Y., Hardt, D., & Jones, M. (2020). JSON Web Token best current practices (RFC 8725). Internet Engineering Task Force.
Singh, H., & Sittig, D. F. (2016). Measuring and improving patient safety through health information technology: The Health IT Safety Framework. BMJ Quality & Safety, 25(4), 226–232. https://doi.org/10.1136/bmjqs-2015-004486
Sittig, D. F., & Singh, H. (2010). A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Quality and Safety in Health Care, 19(Suppl. 3), i68–i74. https://doi.org/10.1136/qshc.2010.042085
U.S. Food and Drug Administration. (2026, January 6). Clinical decision support software: Guidance for industry and Food and Drug Administration staff (revised). U.S. Department of Health and Human Services. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software
U.S. Government Accountability Office. (2015). Electronic health records: Nonfederal efforts to help achieve health information interoperability (GAO-15-817).
U.S. Government Accountability Office. (2024a). COVID-19: HHS needs to identify duplicative pandemic IT systems and implement key privacy requirements (GAO-24-106638).
U.S. Government Accountability Office. (2024b). Healthcare cybersecurity: HHS continues to have challenges as lead agency (GAO-25-107755).
Vorisek, C. N., Lehne, M., Klopfenstein, S. A. I., Mayer, P. J., Bartschke, A., Haese, T., & Thun, S. (2022). Fast Healthcare Interoperability Resources (FHIR) for interoperability in health research: Systematic review. JMIR Medical Informatics, 10(7), e35724. https://doi.org/10.2196/35724
Walonoski, J., Kramer, M., Nichols, J., Quina, A., Moesel, C., Hall, D., Duffett, C., Dube, K., Gallagher, T., & McLachlan, S. (2018a). Synthea: An approach, method, and software mechanism for generating synthetic patients and the synthetic electronic health care record. Journal of the American Medical Informatics Association, 25(3), 230–238. https://doi.org/10.1093/jamia/ocx079
Walonoski, J., Scanlon, R., Dowling, C., Hyland, M., Ettema, R., & Posnack, S. (2018b). Validation and testing of Fast Healthcare Interoperability Resources standards compliance: Data analysis. JMIR Medical Informatics, 6(4), e10870. https://doi.org/10.2196/10870
Published
Issue
Section
License
Copyright (c) 2026 Pavlo Palamarchuk (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles are published as open access and are licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). This means that authors retain the copyright to the content of their articles. Under the CC BY 4.0 license, the content can be copied, adapted, displayed, distributed, republished, or otherwise reused for any purpose, including commercial use, provided that proper attribution is given to the original authors.
How to Cite
