Kentucky Injury Prevention and Research Center Faculty Publications
Abstract
BACKGROUND: Timely data is key to effective public health responses to epidemics. Drug overdose deaths are identified in surveillance systems through ICD-10 codes present on death certificates. ICD-10 coding takes time, but free-text information is available on death certificates prior to ICD-10 coding. The objective of this study was to develop a machine learning method to classify free-text death certificates as drug overdoses to provide faster drug overdose mortality surveillance.
METHODS: Using 2017–2018 Kentucky death certificate data, free-text fields were tokenized and features were created from these tokens using natural language processing (NLP). Word, bigram, and trigram features were created as well as features indicating the part-of-speech of each word. These features were then used to train machine learning classifiers on 2017 data. The resulting models were tested on 2018 Kentucky data and compared to a simple rule-based classification approach. Documented code for this method is available for reuse and extensions: https://github.com/pjward5656/dcnlp.
RESULTS: The top scoring machine learning model achieved 0.96 positive predictive value (PPV) and 0.98 sensitivity for an F-score of 0.97 in identification of fatal drug overdoses on test data. This machine learning model achieved significantly higher performance for sensitivity (p < 0.001) than the rule-based approach. Additional feature engineering may improve the model’s prediction. This model can be deployed on death certificates as soon as the free-text is available, eliminating the time needed to code the death certificates.
CONCLUSION: Machine learning using natural language processing is a relatively new approach in the context of surveillance of health conditions. This method presents an accessible application of machine learning that improves the timeliness of drug overdose mortality surveillance. As such, it can be employed to inform public health responses to the drug overdose epidemic in near-real time as opposed to several weeks following events.
Document Type
Article
Publication Date
10-16-2019
Digital Object Identifier (DOI)
https://doi.org/10.1371/journal.pone.0223318
Funding Information
This paper was supported by Cooperative Agreement Numbers 5 NU17CE002732-04-00 and 6 NU17CE924880-03-04, funded by the Centers for Disease Control and Prevention.
Support for manuscript preparation was also provided by the National Institute on Drug Abuse, CDC, SAMHSA, and the Appalachian Regional Commission (ARC) (UG3 DA044798; PIs: Young and Cooper).
Related Content
The Kentucky Injury Prevention and Research Center (KIPRC) is not the owner of the data used in our study; the data is owned by the Kentucky State Office of Vital Statistics (OVS). In accordance with KIPRC’s Memorandum of Understanding with the Kentucky OVS, we are legally prohibited from releasing line-level death certificate data. Death certificates contain identifying information (including name, location of residence, social security numbers), and in some cases the free-text describing how an individual died can be identifying. As mentioned in our manuscript, each state houses their own OVS, and data may be requested for research purposes from this body; the address for the Kentucky Office of Vital Statistics is: 275 E. Main St, 1E-A, Frankfort, KY 40621, and data may be requested from here for research purposes. Additionally, the National Centers for Health Statistics have recently made available a death certificate literal-text file, which can be requested and contains the free-text cause-of-death information for all U.S. resident deaths: https://www.cdc.gov/rdc/b1datatype/rdcltf.html.
Repository Citation
Ward, Patrick J.; Rock, Peter J.; Slavova, Svetla; Young, April M.; Bunn, Terry L.; and Kavuluru, Ramakanth, "Enhancing Timeliness of Drug Overdose Mortality Surveillance: A Machine Learning Approach" (2019). Kentucky Injury Prevention and Research Center Faculty Publications. 6.
https://uknowledge.uky.edu/kiprc_facpub/6
S1 Appendix. Phrase list for rule-based method. https://doi.org/10.1371/journal.pone.0223318.s001
pone.0223318.s002.docx (13 kB)
S2 Appendix. Error analysis. https://doi.org/10.1371/journal.pone.0223318.s002
Included in
Bioinformatics Commons, Biostatistics Commons, Computer Sciences Commons, Epidemiology Commons, Preventive Medicine Commons, Substance Abuse and Addiction Commons
Notes/Citation Information
Published in PLOS ONE, v. 14, no. 10, 0223318, p. 1-12.
© 2019 Ward et al.
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.