Abstract
Background: Contouring error is one of the top failure modes in radiation treatment. Multiple efforts have been made to develop tools to automatically detect segmentation errors.Deep learning-based auto-segmentation (DLAS) has been used as a baseline for flagging manual segmentation errors, but those efforts are limited to using only one or two contour comparison metrics.
Purpose: The purpose of this research is to develop an improved contouring quality assurance system to identify and flag manual contouring errors.
Methods and materials: DLAS contours were used as a reference to compare with manually segmented contours. A total of 27 geometric agreement metrics were determined from the comparisons between the two segmentation approaches. Feature selection was performed to optimize the training of a machine learning classification model to identify potential contouring errors. A public dataset with 339 cases was used to train and test the classifier. Four independent classifiers were trained using five-fold cross validation, and the predictions from each classifier were ensembled using soft voting. The trained model was validated on a held-out testing dataset. An additional independent clinical dataset with 60 cases was used to test the generalizability of the model. Model predictions were reviewed by an expert to confirm or reject the findings.
Results: The proposed machine learning multiple features (ML-MF) approach outperformed traditional nonmachine-learning-based approaches that are based on only one or two geometric agreement metrics. The machine learning model achieved recall (precision) values of 0.842 (0.899), 0.762 (0.762), 0.727 (0.842), and 0.773 (0.773) for Brainstem, Parotid_L, Parotid_R, and mandible contours, respectively compared to 0.526 (0.909), 0.619 (0.765), 0.682 (0.882), 0.773 (0.568) for an approach based solely on Dice similarity coefficient values. In the external validation dataset, 66.7, 93.3, 94.1, and 58.8% of flagged cases were confirmed to have contouring errors by an expert for Brainstem, Parotid_L, Parotid_R, and mandible contours, respectively.
Conclusions: The proposed ML-MF approach,which includes multiple geometric agreement metrics to flag manual contouring errors, demonstrated superior performance in comparison to traditional methods. This method is easy to implement in clinical practice and can help to reduce the significant time and labor costs associated with manual segmentation and review
Document Type
Article
Publication Date
2-2023
Digital Object Identifier (DOI)
https://doi.org/10.1002/mp.16299
Funding Information
NIH, Grant/Award Number: R44CA254844; NIH NCI, Grant/Award Number: 75N91020C00048; Research Grant from Varian Inc
Repository Citation
Duan, Jingwei; Bernard, Mark E.; Castle, James; Feng, Xue; Wang, Chi; Kenamond, Mark C.; and Chen, Quan, "Contouring quality assurance methodology based on multiple geometric features against deep learning auto-segmentation" (2023). Markey Cancer Center Faculty Publications. 313.
https://uknowledge.uky.edu/markey_facpub/313
Notes/Citation Information
This is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes. © 2023 The Authors. Medical Physics published by Wiley Periodicals LLC on behalf of American Association of Physicists in Medicine.