Succeeded performing the analysis of a probabilistic model for reducing medication errors with various thresholds. Picking the optimal threshold is both an art and a science — it should be done with careful reference to both specialties and the purpose of the application. The AESOP model was observed over 80% accurate (accuracy) for overall departments. Inappropriate prescriptions were determined with a lower rate (i.e. 1%–3%) and the positive predictive value (PPV) ranged from 40%−60%.
Medication-related clinical decision support systems have already been considered as a sophisticated method to improve healthcare quality, however, its importance has not been fully recognized. This paper’s aim was to validate an existing probabilistic model that can automatically identify medication errors by performing a sensitivity analysis from electronic medical record data.
773.4 million prescriptions were employed to compute the disease-medication and medication-medication associations. We built a knowledge base that consisted of 2.22 million disease-medication (DM) and 0.78 million medication-medication (MM) associations.
We collected 626,710 prescriptions from outpatient visit prescriptions from six departments (Cardiovascular, Neurology, Metabolism, Gastroenterology, Ophthalmology, and Urology department) across five hospitals in Taiwan.
We implemented the AESOP model for all 625,710 prescriptions using various threshold (α); the α is ranged from 0.5 to 1.5. Subsequently, 400 prescriptions with their default threshold of 1 that were randomly selected from each department, were evaluated by three certified physicians by their specialty. In total, 2400 prescriptions (400 per department) were evaluated by 18 physicians (three physicians per department) in our study.
Validation against the gold standard showed a high accuracy (over 80%), sensitivity (80–96%), and positive predictive value (over 85%). The negative predictive values ranged from 45 to 75%. PPV (positive predictive values) means both AESOP model prediction and experts evaluate the prescription as appropriate. NPV (negative predictive values) means both AESOP model prediction and experts evaluate the prescription as inappropriate.
We performed sensitivity analysis and validated the AESOP model at different hospital sites. Our findings show that picking up the optimal threshold of the model would be different in departments and depends on the purpose of the applications. Building an innovative system for detecting medication errors has many potential benefits for patient safety, improving quality healthcare, and conducting research.
773.4 million prescriptions were employed to compute the disease-medication and medication-medication associations. We built a knowledge base that consisted of 2.22 million disease-medication (DM) and 0.78 million medication-medication (MM) associations.
We collected 626,710 prescriptions from outpatient visit prescriptions from six departments (Cardiovascular, Neurology, Metabolism, Gastroenterology, Ophthalmology, and Urology department) across five hospitals in Taiwan.
We implemented the AESOP model for all 625,710 prescriptions using various threshold (α); the α is ranged from 0.5 to 1.5. Subsequently, 400 prescriptions with their default threshold of 1 that were randomly selected from each department, were evaluated by three certified physicians by their specialty. In total, 2400 prescriptions (400 per department) were evaluated by 18 physicians (three physicians per department) in our study.
Validation against the gold standard showed a high accuracy (over 80%), sensitivity (80–96%), and positive predictive value (over 85%). The negative predictive values ranged from 45 to 75%. PPV (positive predictive values) means both AESOP model prediction and experts evaluate the prescription as appropriate. NPV (negative predictive values) means both AESOP model prediction and experts evaluate the prescription as inappropriate.
We performed sensitivity analysis and validated the AESOP model at different hospital sites. Our findings show that picking up the optimal threshold of the model would be different in departments and depends on the purpose of the applications. Building an innovative system for detecting medication errors has many potential benefits for patient safety, improving quality healthcare, and conducting research.
Huang, C. Y., Nguyen, P. A., Yang, H. C., Islam, M. M., Liang, C. W., Lee, F. P., & Li, Y. C. (2019). A probabilistic model for reducing medication errors: A sensitivity analysis using Electronic Health Records data. Computer Methods and Programs in Biomedicine, 170, 31-38.
doi:10.1016/j.cmpb.2018.12.033
doi:10.1016/j.cmpb.2018.12.033
A Probabilistic Model for Reducing Medication Errors.
The AOP model has a variety of applications. It can be used to alert physicians if medication errors are detected while prescribing medications using the CPOE system. Additionally, the model could be used to reduce the size of medication list in the CPOE for a given diagnosis. An automated medication listing systems and clinical decision support system (CDSS) can also be developed by using the AOP model.
We developed a model to detect uncommon or rare medication for a given disease when ordering prescriptions based on disease-medication associations.
This study focuses on medication-disease relationships by applying the association rule mining, and using statistical methods to detect medication errors in the computerized physician order entry (CPOE) systems in order to improve patient’s safety. 103.5 million prescriptions with 204.5 million diagnosis ICD9-CM (International Classification of Disease v.9-Clinical Modification) codes and 347.7 million medications with the Taiwan NHI codes were used in the analysis.
We use Q values to compute association for Disease-medication (DM) and Medication-medication (MM). Q value is the ratio between the joint probability of disease-medication and medication-medication with respect to their expected probability under the independent assumption known as lift (interest) and relative risk (RR) in similar studies dealing with associations. A total of 1.34 million DM and 0.65 million MM pairs with their Q values were computed from 103.5 million prescriptions.
Then we develop a model that can automatically evaluate the Appropriateness of a Prescription (AOP) which is developed based upon following rules:
Finally, we test the model for 100,000 randomly selected prescriptions, and then validating the results using seven human experts. The results showed 96% accuracy for appropriate and 45% accuracy for inappropriate prescriptions, with a sensitivity and specificity of 75.9% and 89.5%, respectively.
We successfully developed, tested, and validated the AOP model, which is able to predict and identify the appropriateness of the prescriptions. The AOP model developed in this study is able to detect accurately the inappropriate medications prescribed via COPE system. Thus, the PPV of the validation results from both physicians and pharmacists were accurate for the appropriate prescriptions. Moreover, this model could be applied in clinical practice to aid in improving prescription appropriateness, accuracy, patient safety, and patient care.
This study focuses on medication-disease relationships by applying the association rule mining, and using statistical methods to detect medication errors in the computerized physician order entry (CPOE) systems in order to improve patient’s safety. 103.5 million prescriptions with 204.5 million diagnosis ICD9-CM (International Classification of Disease v.9-Clinical Modification) codes and 347.7 million medications with the Taiwan NHI codes were used in the analysis.
We use Q values to compute association for Disease-medication (DM) and Medication-medication (MM). Q value is the ratio between the joint probability of disease-medication and medication-medication with respect to their expected probability under the independent assumption known as lift (interest) and relative risk (RR) in similar studies dealing with associations. A total of 1.34 million DM and 0.65 million MM pairs with their Q values were computed from 103.5 million prescriptions.
Then we develop a model that can automatically evaluate the Appropriateness of a Prescription (AOP) which is developed based upon following rules:
- The number of positive DMQs and positive MMQs should be greater than or equal to the number of medications.
- All diagnoses should have at least one positive DMQ.
- Each medication should have at least one positive DMQ or positive MMQ.
Finally, we test the model for 100,000 randomly selected prescriptions, and then validating the results using seven human experts. The results showed 96% accuracy for appropriate and 45% accuracy for inappropriate prescriptions, with a sensitivity and specificity of 75.9% and 89.5%, respectively.
We successfully developed, tested, and validated the AOP model, which is able to predict and identify the appropriateness of the prescriptions. The AOP model developed in this study is able to detect accurately the inappropriate medications prescribed via COPE system. Thus, the PPV of the validation results from both physicians and pharmacists were accurate for the appropriate prescriptions. Moreover, this model could be applied in clinical practice to aid in improving prescription appropriateness, accuracy, patient safety, and patient care.
Nguyen, P. A., Syed-Abdul, S., Iqbal, U., Hsu, M. H., Huang, C. L., Li, H. C., Li, Y. C. J. (2013). A Probabilistic Model for Reducing Medication Errors. PloS one, 8(12).
doi:10.1371/journal.pone.0082401
doi:10.1371/journal.pone.0082401
Related Papers
- Huang, C. Y., Nguyen, P. A. A., Clinciu, D. L., Hsu, C. K., Lu, J. C. R., Yang, H. C., Li, Y. C. J. (2017). A personalized medication management platform (PMMP) to improve medication adherence: A randomized control trial. Computer Methods and Programs in Biomedicine, 140, 275-281. doi:10.1016/j.cmpb.2016.12.012
- Li, Y. C., Yen, J. C., Chiu, W. T., Jian, W. S., Syed-Abdul, S., & Hsu, M. H. (2015). Building a National Electronic Medical Record Exchange System - Experiences in Taiwan. Computer Methods and Programs in Biomedicine, 121(1), 14-20. doi:10.1016/j.cmpb.2015.04.013
- Syed-Abdul, S., Moldovan, M., Nguyen, P. A., Enikeev, R., Jian, W. S., Iqbal, U., Li, Y. C. (2015). Profiling phenome-wide associations: a population-based observational study. Journal of the American Medical Informatics Association, 22(4), 896-899. doi:10.1093/jamia/ocu019
- Syed-Abdul, S., Nguyen, A., Huang, F., Jian, W. S., Iqbal, U., Yang, V., Li, Y. C. (2014). A smart medication recommendation model for the electronic prescription. Computer Methods and Programs in Biomedicine, 117(2), 218-224. doi:10.1016/j.cmpb.2014.06.019
- Yeh, M. L., Chang, Y. J., Wang, P. Y., Li, Y. C., & Hsu, C. Y. (2013). Physicians' responses to computerized drug-drug interaction alerts for outpatients. Computer Methods and Programs in Biomedicine, 111(1), 17-25. doi:10.1016/j.cmpb.2013.02.006
- Chang, Y. J., Yeh, M. L., Li, Y. C., Hsu, C. Y., Yen, Y. T., Wang, P. Y., & Chu, T. W. (2011). Potential drug interactions in dermatologic outpatient prescriptions-experience from nationwide population-based study in Taiwan. Dermatologica Sinica, 29(3), 81-85. doi:10.1016/j.dsi.2011.07.001
- Hsu, M. H., Yen, J. C., Chiu, W. T., Tsai, S. L., Liu, C. T., & Li, Y. C. (2011). Using Health Smart Cards to Check Drug Allergy History: The Perspective from Taiwan's Experiences. Journal of Medical Systems, 35(4), 555-558. doi:10.1007/s10916-009-9391-5
- Long, A. J., Chang, P., Li, Y. C., & Chiu, W. T. (2008). The use of a CPOE log for the analysis of physicians' behavior when responding to drug-duplication reminders. International Journal of Medical Informatics, 77(8), 499-506. doi:10.1016/j.ijmedinf.2007.10.002
- Li, Y. C., Haug, P., Lincoln, M., Turner, C., Pryor, T., & Warner, H. (1995). Assessing the behavioral impact of a diagnostic decision support system. Paper presented at the Proceedings of the Annual Symposium on Computer Application in Medical Care.
- Li, Y. C., Haug, P. J., & Warner, H. R. (1994). Automated Transformation Of Probabilistic Knowledge For A Medical Diagnostic System. Journal Of The American Medical Informatics Association, 765-769.