Advertisement

Pseudo-siamese network combined with dosimetric and clinical factors, radiomics features, CT images and 3D dose distribution for the prediction of radiation pneumonitis: A feasibility study

Open AccessPublished:November 21, 2022DOI:https://doi.org/10.1016/j.ctro.2022.11.011

      Highlights

      • We firstly proposed a novel method that combined dosimetric and clinical factors, radiomics feature, 3D CT images and dose distribution for predicting RP.
      • A pseudo-siamese network were developed for combination of 1D and 3D information.
      • The used deep learning frame showed a good accuracy (81.8%) for the prediction of RP.

      Abstract

      Purpose

      Radiation pneumonitis (RP)(grade≥2) can have a considerable impact on patient quality-of life. In previous studies, the traditional method commonly used radiomics and clinical factors for RP prediction. This study aims to develop and evaluate a novel pseudo-siamese network (PSN) to assist radiologists predict RP before radiotherapy based on combination of dosimetric and clinical factors, radiomics features, CT (computed tomography) images, and dose distribution (hybrid model).

      Method

      One hundred and ten patients with lung cancer (19 RP≥2) who received radiotherapy between 2016-2020 were retrospectively enrolled in this study. Dosimetric factors were calculated from DVH (dose-volume histogram), such as lung mean dose, lung V5, and prescription dose. Clinical characteristics were recorded, such as age, sex, smoking status, TN stage, and overall stage. A total of 1419 radiomics features were extracted. Cluster analysis was used for detecting radiomics features that associated with RP. Patients were randomly split into a training set (90%, 85 non-RP, and 14 RP) and a validation set (10%, 6 non-RP, and 5 RP). A PSN architecture was designed for combining 1D (dosimetric and clinical factors, radiomics) and 3D (CT images, 3D dose distribution) features. 5-fold cross-validation procedure for estimating the skill of the model on new data.

      Results

      For cluster analysis, totally of 106 radiomics features with high correlation were selected. The accuracy was 0.727, 0.636, 0.545, and 0.727 for input dosimetric and clinical factors, dose distribution, CT images, and radiomics features, respectively. The accuracy of hybrid model was 0.818. The sensitivity of hybrid model was 0.800 (95% confidence interval (CI) [0.299,0.989]), and specificity was 0.833(95% CI [0.364,0.991]). The areas under the receiver operating characteristic curves (AUCs) result in 5-fold cross-validation was 0.77-0.90(mean AUC ± std was 0.85±0.05).

      Conclusion

      This study firstly propose method that the combination of high dimensional and low dimensional features for RP prediction. The results confirm the feasibility of multi-dimensional features predict RP.

      Keywords

      Introduction

      Lung cancer is the most frequently occurring cancer, and leading cause of cancer death (18.0% of the total cancer deaths).[
      • Sung H.
      • Ferlay J.
      • Siegel R.L.
      • Laversanne M.
      • Soerjomataram I.
      • Jemal A.
      • et al.
      Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries.
      ] Radiation therapy (RT) is an importance treatment strategy that plays a crucial role in the lung cancer. Local tumor control and overall survival can be improved by high-dose radiation[

      F.-M. Kong, R.K. Ten Haken, M.J. Schipper, M.A. Sullivan, M. Chen, C. Lopez, et al. High-dose radiation improved local tumor control and overall survival in patients with inoperable/unresectable non–small-cell lung cancer: Long-term results of a radiation dose escalation study, International Journal of Radiation Oncology*Biology*Physics. 63(2) (2005) 324-333. https://doi.org/https://doi.org/10.1016/j.ijrobp.2005.02.010.

      ,
      • Le Q.-T.
      • Loo B.W.
      • Ho A.
      • Cotrutz C.
      • Koong A.C.
      • Wakelee H.
      • et al.
      Results of a Phase I Dose-Escalation Study Using Single-Fraction Stereotactic Radiotherapy for Lung Tumors.
      ]. Nevertheless, the risk of radiation pneumonitis (RP) may simultaneously increase. RP is the most common dose-limiting complication of lung radiation, which can have a considerable impact on patient quality-of-life and respiratory function[

      G. Rodrigues, M. Lock, D. D'Souza, E. Yu, J. Van Dyk Prediction of radiation pneumonitis by dose - volume histogram parameters in lung cancer--a systematic review, Radiother Oncol. 71(2) (2004) 127-138. https://doi.org/10.1016/j.radonc.2004.02.015.

      ].
      Previous studies report that RP have been proved associated with dosimetric factors that were calculated from the dose-volume histogram (DVH), such as plan target volume (PTV) D95 (means dose that covers 95% of the PTV), lung V20(means lung volume receiving ≥20 Gy) and lung mean dose, et al[
      • Pan W.Y.
      • Bian C.
      • Zou G.L.
      • Zhang C.Y.
      • Hai P.
      • Zhao R.
      • et al.
      Combing NLR, V20 and mean lung dose to predict radiation induced lung injury in patients with lung cancer treated with intensity modulated radiation therapy and chemotherapy.
      ,
      • Byhardt R.W.
      • Scott C.
      • Sause W.T.
      • Emami B.
      • Komaki R.
      • Fisher B.
      • et al.
      Response, toxicity, failure patterns, and survival in five Radiation Therapy Oncology Group (RTOG) trials of sequential and/or concurrent chemotherapy and radiotherapy for locally advanced non-small-cell carcinoma of the lung.
      ,

      S. Johansson, L. Bjermer, L. Franzen, R. Henriksson Effects of ongoing smoking on the development of radiation-induced pneumonitis in breast cancer and oesophagus cancer patients, Radiother Oncol. 49(1) (1998) 41-47. https://doi.org/10.1016/s0167-8140(98)00064-4.

      ]. Krafft et al[
      • Krafft S.P.
      • Rao A.
      • Stingo F.
      • Briere T.M.
      • Court L.E.
      • Liao Z.
      • et al.
      The utility of quantitative CT radiomics features for improved prediction of radiation pneumonitis.
      ] developed clinical and dosimetric factors-based model for RP prediction, the result were shown that the areas under receiver operating characteristic curves (AUC) were only 0.68. However, the ideal DVH metric with outperformer operating characteristics, either alone or with other predictive variable in a model, for RP risk prediction before radiotherapy has not yet been identified[

      G. Rodrigues, M. Lock, D. D'Souza, E. Yu, J. Van Dyk Prediction of radiation pneumonitis by dose - volume histogram parameters in lung cancer--a systematic review, Radiother Oncol. 71(2) (2004) 127-138. https://doi.org/10.1016/j.radonc.2004.02.015.

      ]. Radiomics extraction can provide a large number of quantitative images features, which tends to be hard for eyes to recognize[
      • Zheng X.
      • Yao Z.
      • Huang Y.
      • Yu Y.
      • Wang Y.
      • Liu Y.
      • et al.
      Deep learning radiomics can predict axillary lymph node status in early-stage breast cancer.
      ,

      R.J. Gillies, P.E. Kinahan, H. Hricak Radiomics: Images Are More than Pictures, They Are Data, Radiology. 278(2) (2016) 563-577. https://doi.org/10.1148/radiol.2015151169.

      ,
      • Wang K.
      • Lu X.
      • Zhou H.
      • Gao Y.
      • Zheng J.
      • Tong M.
      • et al.
      Deep learning Radiomics of shear wave elastography significantly improved diagnostic performance for assessing liver fibrosis in chronic hepatitis B: a prospective multicentre study.
      ]. Luo et al[
      • Luo Y.
      • McShan D.L.
      • Matuszak M.M.
      • Ray D.
      • Lawrence T.S.
      • Jolly S.
      • et al.
      A multiobjective Bayesian networks approach for joint prediction of tumor local control and radiation pneumonitis in nonsmall-cell lung cancer (NSCLC) for response-adapted radiotherapy.
      ] extracted radiomics features from positron emission tomography and combined with clinical and dosimetric factors, the AUC for predicting RP achieved 0.77. Jiang et al[

      W. Jiang, Y. Song, Z. Sun, J. Qiu, L. Shi Dosimetric Factors and Radiomics Features Within Different Regions of Interest in Planning CT Images for Improving the Prediction of Radiation Pneumonitis, Int J Radiat Oncol Biol Phys. 110(4) (2021) 1161-1170. https://doi.org/10.1016/j.ijrobp.2021.01.049.

      ] combine dosimetric factors and multi-target volume radiomics features from planning CT images for the prediction of symptomatic RP, the AUC has improved significantly comparing with previous studies.
      In addition, the above studies were combined radiomics and dosimetric factors as input, and machine learning algorithm were commonly used. Due to dosimetric factors and radiomics feature are one-dimensional(1D) and discrete variables, some features associated with RP (such as three-dimensional(3D) CT image and dose distribution) were neglected. These factors assume a uniform distribution of pulmonary function and therefore a homogeneous dose response for sub-volume[
      • Bin L.
      • Yuan T.
      • Zhaohui S.
      • Wenting R.
      • Zhiqiang L.
      • Peng H.
      • et al.
      A deep learning-based dual-omics prediction model for radiation pneumonitis.
      ]. On the contrary, the true dose distribution and gradient were heterogeneous yet. Moreover, the main difference between deep learning and traditional machine learning is that its performance increases as the data size increases[
      • Shinde P.P.
      • Shah S.
      A review of machine learning and deep learning applications,Fourth international conference on computing communication control and automation (ICCUBEA).
      ]. In terms of practical application, many studies have that deep learning can reach or exceed that of traditional machine learning if data adequate[

      G. Currie, K.E. Hawk, E. Rohren, A. Vial, R. Klein Machine Learning and Deep Learning in Medical Imaging: Intelligent Imaging, J Med Imaging Radiat Sci. 50(4) (2019) 477-487. https://doi.org/10.1016/j.jmir.2019.09.005.

      ]. Deep learning radiomics, a novel developed method, can support quantitative and high-throughput features from images by convolution[
      • Wang K.
      • Lu X.
      • Zhou H.
      • Gao Y.
      • Zheng J.
      • Tong M.
      • et al.
      Deep learning Radiomics of shear wave elastography significantly improved diagnostic performance for assessing liver fibrosis in chronic hepatitis B: a prospective multicentre study.
      ,
      • Xie Y.
      • Zhang J.
      • Xia Y.
      • Fulham M.
      • Y.
      Zhang Fusing texture, shape and deep model-learned information at decision level for automated classification of lung nodules on chest CT.
      ]. And deep learning algorithm were able to extract spatial features from high-dimensional data. Therefore, 3D dose distribution and CT image may able to improve the performance of RP prediction.
      Here, we proposed a novel network architecture that combination of 1D radiomics features, dosimetric and clinical factors, and 3D CT image and dose distribution features for predicting the incidence of RP after radiotherapy.

      Methods

      2.1 Patients and CT image acquisition

      This study retrospectively 110 patients with stage I-IV lung cancer and received radiotherapy, including 66 IMRT (Intensity modulated radiation therapy) and 44 VMAT (Volumetric modulated arc therapy), at our institution between 2016-2020. The patient characteristics were shown in table 1. A total of 110 patients were enrolled based on the criteria (Table 1). The dosimetric factors include prescription dose, fraction dose, PTV, lung V5, V10, V20, V30, V40, lung Dean dose. National Cancer Institute’s Common Terminology Criteria for Adverse Events (vision 4) report that RP were graded from 1 (asymptomatic) to 5 (death)[
      • Savarese D.
      Common terminology criteria for adverse events.
      ]. Tumor stage was performed based on the clinical staging of the 8th edition of the American Joint Committee on Cancer (AJCC) TNM staging system, including locally advance disease (stage III), inoperable stage I and II disease. Patients were selected if they received a prescription dose of 40-60Gy in 1.5-3.5Gy-daily fractions using 6 MV photons. The endpoint was grade≥2. The RP were determined by a radiologist with 10 years of clinical experience by using follow-up CT images and clinical records. In addition, the general region of interests (ROIs) were segmented, such as Lung, heart, spine et al. All contours were double-checked by physicist to improve the performance of the deep-learning model.
      Table 1Patient characteristics.
      Clinical characteristicsData
      RP(grade≥2)19
      Non-RP91
      Sex
      Male60
      Female50
      T stage
      02
      111
      229
      323
      445
      N stage
      014
      17
      248
      341
      Overall Stage
      I4
      II7
      III68
      IV31
      Age
      Median62
      Average63
      Range39-92
      Smoking status
      Yes45
      No65
      Patients underwent a free-breathing CT scan for radiotherapy planning using Brilliance-16 system (Philips Medical Systems, Inc., Cleveland, OH). The imaging parameters as follows: tube voltage (120kVp), tube current (182∼320mA), pixel size (1.06mm × 1.06mm), reconstruction increment (3.00mm), slice thickness (3.00mm), and matrices of images (X:512, Y:512, Z:79-150).

      2.3 Radiomics features extraction and analysis

      The workflow was shown in Figure.1. This study can be divided into two parts. For Phase 1, patients radiomics features were extracted from CT images. Totally 1419 features were extracted from Dicom (digital imaging and communications in medicine) CT files. The image types include original, wavelet, LoG (Laplacian of Gaussian filter), and available feature classes were first order statistic, shape-based(3D), GLCM (gray level cooccurrence matrix), GLRLM (gray level run length matrix), GLSZM (gray level size zone matrix), NGTDM (Neighbouring gray tone difference matrix), GLDM (gray level dependence matrix). The program was development with Python (version 3.6). Pyradiomics package were used for feature extraction. This is an open-source package for the extraction of radiomics from medical imaging. With this package we could to establish a reference standard for radiomic analysis, and provide a tested and maintained open-source platform for easy and reproducible radiomic feature extraction.
      Figure thumbnail gr1
      Figure 1the workflow of data process and data analysis performed in this study.
      Radiomics features were selected based on significance, correlation, and cluster result. Manhattan distance was widely used for the clustering of multi-variable datasets in computational biology[
      • Pandit S.
      • S.
      • Gupta
      • A,
      Comparative Study on Distance Measuring Approaches for Clustering, International Journal of.
      ]. it was used for selecting features that correlation with RP.

      2.4 Dataset and CT image preprocessing

      Thereafter, radiomics features extraction, a label (RP or non-RP) was assigned to each feature and data were randomly divided into training set and test set, with 90 patients in training set included 85 non-RP and 14 RP. The test set included 6 non-RP and 5 RP, total 11 patients.
      For each endpoint combined radiomics features, dosimetric and clinical factors and 3D CT-dose images (figure 2). To investigate the incremental prediction value of high-dimensional features, five prediction models were designed using the same method: (1) based on clinical and dosimetric factors; (2) based on 3D dose distribution; (3) based on radiomics features; (4) based on 3D CT images; (5) hybrid (1)-(4) features.
      Figure thumbnail gr2
      Figure 2Overall preprocess workflow of feature extraction: CT image and dose distribution data for PSN sub-network 3D input channel; dosimetric and clinical factors and radiomics features for PSN sub-network 1D input channel.

      2.5 Architecture of deep-learning network

      For phase 2 in Figure 1, Figure 3D CT images and dose distribution were exported as dicom file format for image process. Figure 3 was shown the detailed prediction network architecture. The framework backbone was residual block, for processing different dimension of data, we developed a pseudo-siamese network. This network can be set as single mode when input only 1D or 3D.
      Figure thumbnail gr3
      Figure 3Pseudo-siamese network, 3D Conv, 3D convolutional layer;1D Conv, 1D convolutional layer; Batch norm, batch normalization; Maxpool, max-pooling layer; GlobalAvgpool, global average pooling layer; Linear, linear layer.
      The input images were resized to 96×96×Z pixel. Random flipping, rotation, contrast adjustment, color jitter, and affine transform were used for data augmentation. All images pixel value were normalized within range of 0 to 1. The radiomics feature value were normalized within range of -10 to 10. The batch size for training set was 16 and test set was 1. For radiomics feature input, a single 1D Resnet feature extractor were designed for filter useful signature. 1D features and 3D features are finally flattened to assemble by fully connected layers. Modified network layer can improved model performance and avoid overfitting[

      K. Ahmed, L. Hall, D. Goldgof, R. Liu, R. Gatenby, Fine-tuning convolutional deep features for MRI based brain tumor classification, SPIE2017.

      ] (i.e., reduce the number of residual blocks and network layer appropriately). The loss function was cross-entropy, and Adam was used as the optimizer. Epochs were set 1000 with the 1e-4 learning rate. Weight decay was 0.1. An Intel 8700k CPU and Nvidia TITAN GPU were used for model training.

      2.6 The impact of different features for predicting radiation pneumonitis

      To investigate different kind of data for predicting RP, 5 classifiers were built by using (a)Dosimetric and clinical factors, (b)3D dose distribution, (c)3D CT images, (d)Radiomics features, (e)the combination of dosimetric and clinical factors, radiomics features, 3D dose distribution, and 3D CT images (hybrid model). Each dataset was trained separately until the model converged. 5-fold cross-validation were used for testing the model’s ability to predict data that was not used in estimating it, in order to reduce problems like overfitting or selection bias[
      • Cawley G.
      • N.
      Talbot On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation.
      ].

      2.7 Performance evaluation

      The receiver operating characteristic (ROC) curve was a commonly diagnostic tool for evaluating the performance of binary classifiers. This study exist class imbalance may introduce bias to the non-RP group. Cross validation were introduced for verifying the generalization ability of the model.

      Results

      3.1 clustering analysis

      The clustering results were shown as feature heatmap in figure 4, with 110 patients on the x-axis and the expression of 106 radiomics features on the y-axis. The heatmap that patients within the same group cluster displayed similar value patterns. 106 radiomics features were selected from 1419 radiomics features.
      Figure thumbnail gr4
      Figure 4The radiomics features are shown as the heatmap, with 110 patients on the x-axis and 107 radiomics features on the y-axis. The top legend shows the patient RP situation, 1 and 0 represent RP and non-RP, respectively.

      3.2 Performance and index estimation

      Figure 5 shows confusion matrix of the different dataset, each element in the confusion matrix means the number of samples with the true class y that was identified as being in class x. To explore the value of different characteristic of data, the single type dataset was used for training (a-d). Dosimetric and clinical factor model were shown in figure 5(a), totally 8 samples were truly predicted. 3D dose distribution and CT image model were shown in figure(b) and (d), respectively. 3D dose distribution model was shown poor sensitivity than (a). RP and non-RP pattern cannot be identified by only using CT images dataset. Radiomics feature model were shown in figure 5(c), the number of true predictions the same to the dosimetric and clinical factors. The sensitivity, however, were poor than dosimetric and clinical factor model. The hybrid model result was shown outperformance than other models in figure 5(e).
      Figure thumbnail gr5
      Figure 5The confusion matrix for prediction of non-RP patient and patients with RP.
      To summarize, the index that evaluate the performance of model were shown in Table 2 (for example, accuracy, precision, recall, and f1-score). The accuracy of dosimetric and clinical factor model and radiomics feature model both 0.727, while dosimetric and clinical factor model was shown good performance in recall (0.800) and F1-score (0.727) than radiomics feature model. Precision of dosimetirc and clinical factor and radiomics feature model were 0.667 and 0.750, respectively. The accuracy of dose distribution model and CT image model was 0.636 and 0.545, respectively. Recall, precision, and F1-score were 0.600 in dose distribution model, and 0.600, 0.500 and 0.545 in CT image model.
      Table 2summarizes the accuracy, precision, recall, and f1-score in test dataset.
      InputAccuracyRecallPrecisionF1-score
      Dosimetric and clinical factors0.7270.8000.6670.727
      Dose distribution0.6360.6000.6000.600
      CT images0.5450.6000.5000.545
      Radiomics features0.7270.6000.7500.667
      Hybrid model0.8180.8000.8000.800

      3.3 Model validation

      Figure 6 shows the results of 5-fold cross-validation for hybrid model. The mean AUC was 0.85, For 1-5 fold cross-validation, AUC was 0.77-0.90, the mean AUC was 0.85±0.05. The sensitivity and specificity for different model was listed in Table 3. Hybrid model and dosimetric and clinical factor model have the same sensitivity 0.800 (95% CI [0.299, 0.989]), while the sensitivity for dose distribution, CT image, and radiomics feature model were 0.600 [0.170, 0.927]. For specificity, CT image model was 0.500 [0.139,0.860], dosimetric and clinical factor model and dose distribution model are 0.667 [0.241,0.940]. Radiomics feature model and hybrid model are 0.833 [0.364,0.991]. The predictive effect of the hybrid model showed a stable performance in both the sensitivity and specificity.
      Figure thumbnail gr6
      Figure 6ROC for hybrid model with 5-fold cross-validation. Note: the mean value and standard deviation are shown.
      Table 3summarizes the sensitivity and specificity in test dataset
      InputSensitivity [95% CI]Specificity [95% CI]
      Dosimetric and clinical factors0.800 [0.299,0.989]0.667 [0.241,0.940]
      Dose distribution0.600 [0.170,0.927]0.667 [0.241,0.940]
      CT images0.600 [0.170,0.927]0.500 [0.139,0.860]
      Radiomics features0.600 [0.170,0.927]0.833 [0.364,0.991]
      Hybrid model0.800 [0.299,0.989]0.833 [0.364,0.991]

      Discussion

      In this study, we firstly proposed a novel method extracted dosimetric and clinical factors, radiomics features combined with 3D CT images and dose for predicting RP occurrence before delivery. The results were shown the feasibility of the model for predicting RP incidence.
      Previous studies have found clinical markers associated with RP. Zhang et al reported that the multivariate analysis of risk factors predicting grade≥2, higher CD8+ T cell count was shown a risk factor associated with RP (p<0.05)[
      • Zhang X.
      • Yang D.
      • Jiang Y.
      • Huang L.
      • Wang C.
      • Tao D.
      • et al.
      Comparison of Radiation Pneumonitis in Lung Cancer Patients Treated with HT versus IMRT and Circulating Lymphocyte Subsets as Predicting Risk Factors.
      ]. There are many considerable and novel characteristics were not enrolled in this study, such as CD4/8+ T cell count, CD4/8+ T cell ratio, B cell, etc. The useful cell statistics information associate with RP able be collected only after radiotherapy[
      • Zhou P.
      • Chen L.
      • Yan D.
      • Huang C.
      • Chen G.
      • Wang Z.
      • et al.
      Early variations in lymphocytes and T lymphocyte subsets are associated with radiation pneumonitis in lung cancer patients and experimental mice received thoracic irradiation.
      ]. Therefore, we have been excluded these parameters. The advantage of our model is that RP would be predicted before the first delivery. Dosimetric factors were proved considerable for predicting RP in previous study, such as lung V20, mean lung dose, PTV[
      • Saha A.
      • Beasley M.
      • Hatton N.
      • Dickinson P.
      • Franks K.
      • Clarke K.
      • et al.
      Clinical and dosimetric predictors of radiation pneumonitis in early-stage lung cancer treated with Stereotactic Ablative radiotherapy (SABR) - An analysis of UK's largest cohort of lung SABR patients.
      ]. Since 3D dose distribution as input, this study hasn’t compared these dosimetric parameter which have high correlation with RP. These dosimetric characteristic, whereas, still as deep learning model input for improving performance. In our implementation of network, we compared the efficacies of multi-input model, the experiment shows that dosimetric and clinical characteristic model have the same sensitivity than hybrid model while specificity is lower.
      Radiomics features were traditionally combined with machine learning algorithm for the RP prediction. Radiomics features were extracted from different scale ROI in CT images. Although radiomics feature extraction shows a good performance in predicting RP, the high-dimensional information from radiotherapy plan useful for prediction. On the other hand, there is no consensus yet whether the design of radiotherapy plan, beam angle, target, etc, may affect the arise of RP. Deep learning may detect heterogeneity in medical images from high dimensional data point, resulting in greater predictive performance[
      • Lao J.
      • Chen Y.
      • Li Z.C.
      • Li Q.
      • Zhang J.
      • Liu J.
      • et al.
      A Deep Learning-Based Radiomics Model for Prediction of Survival in Glioblastoma Multiforme.
      ,
      • Tanaka S.
      • Kadoya N.
      • Sugai Y.
      • Umeda M.
      • Ishizawa M.
      • Katsuta Y.
      • et al.
      A deep learning-based radiomics approach to predict head and neck tumor regression for adaptive radiotherapy.
      ]. In terms of this, we consider the complete 3D CT images/dose distribution contain potential features which can be extracted by using convolution neural network. To mitigate the limitations of the traditional radiomics predict RP techniques, our approach introduced the 3D CT images and dose distribution, which including the information of the anatomic status, beam angle, and radiotherapy plan could as supplement for model prediction.
      CT has become the main imaging protocol for lung diagnosis because there are many alveoli in the lung, which were filled with gas. In CT image preprocessing, the dose distribution and CT images should be registered. To reduce redundant information, the table was removed from CT image (Figure 2). Lung is crucial structure for prediction RP, therefore, the radiomics features were extracted from lung. Traditional radiomics feature can only reflect texture information. In this study, we also use single feature, like 3D CT image CNN feature or radiomics features, to build model. The results were shown that radiomics model has better performance than 3D CT image model, therefore, we consider radiomics feature still valuable be added in. Using 3D CT images as CNN model input combine with 3D dose map can make them spatial coordinate matching.
      Compared with the previous study, our research yielded a better predict performance by concentrating on the 3D images, dose feature combined deep learning algorithm, which can complement 3D feature with more information[
      • Zheng X.
      • Yao Z.
      • Huang Y.
      • Yu Y.
      • Wang Y.
      • Liu Y.
      • et al.
      Deep learning radiomics can predict axillary lymph node status in early-stage breast cancer.
      ,
      • Xie Y.
      • Zhang J.
      • Xia Y.
      • Fulham M.
      • Y.
      Zhang Fusing texture, shape and deep model-learned information at decision level for automated classification of lung nodules on chest CT.
      ]. Experiments were shown that combine radiomics features and deep learning features can improve performance for predicting RP. Liang et al[
      • Bin L.
      • Yuan T.
      • Zhaohui S.
      • Wenting R.
      • Zhiqiang L.
      • Peng H.
      • et al.
      A deep learning-based dual-omics prediction model for radiation pneumonitis.
      ] designed a feature filter and developed a dual-omics (original dose and ventilation images) prediction model for RP, the AUC was 0.874. The ventilation images, however, should be derived from 4-dimensional CT, this approach certainly raises the cost of diagnosis. The advantage of our approach is that we can used more information which can be collected easily from clinical routine without affiliated step.
      For the training model, we selected PSN for this task, the residual block as the backbone to prevent the gradient disappearance and minimize the degradation problem after many iterations[
      • Xia X.
      • Feng B.
      • Wang J.
      • Hua Q.
      • Yang Y.
      • Sheng L.
      • et al.
      Deep Learning for Differentiating Benign From Malignant Parotid Lesions on MR Images.
      ,
      • He K.
      • Zhang X.
      • Ren S.
      • Sun J.
      Deep Residual Learning for Image Recognition.
      ,
      • Zaheer R.
      • Shaziya H.
      A Study of the Optimization Algorithms in Deep Learning.
      ]. Dual channel solved the data dimensional match problem.
      For the feature standardization, our study was similar to the IBSI (Image Biomarker Standardization Initiative) consortium to processed radiomics features[
      • Zwanenburg A.
      • Vallières M.
      • Abdalah M.A.
      • Aerts H.J.W.L.
      • Andrearczyk V.
      • Apte A.
      • et al.
      The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping.
      ]. The image date was processed by conversion to standardized uptake values, segmentation and create ROI mask. Finally, the radiomics feature were computed via ROI masks and images. PSN network can be reproduced easily refer our architecture.
      This study has several limitations. First, only single institution with a small cohort were enrolled in this retrospective study. We did not validate this model with an external dataset, which could have been valuable in demonstrating the reliable performance of the model. We did many times of 10 folds cross validation to prevent the model overfitting (appendix A). We also compared a pre-trained model for feature extraction. However, the results were no better than our proposed model (appendix B). Therefore, we think our method may suitable for specific medical image analysis area. In this study, we proposed model is not a large CNN network (like Resnet 50, 100, 150 or Dense net-101). It just used the residual blocks. The model parameter was lightweight. The number of layers isn’t too deep. Second, we combined dosimetric and clinical factors, radiomics, CT images and dose which have not involved parameter in feature. For example, in radiomics extraction, the resample pixel spacing, bin width, and sigma should be defined (The default in this study we set 0.5mm per pixel, bin width was 25, sigma was (1,3)). The parameter setting have many method (e.g. SquartRoot, Sturges, Freedman-Diaconis) that have not be consideration in this study. Third, the positive sample in this study was small (only 19 patients), therefore, this study merely explored the feasibility of the PSN model for predicting RP with a few dataset.
      A large-scale, multicenter validation and study are required for full evaluation of model in the future. We will enroll more patients and join multi-scale parameter in this model for assisting physician and physicist design radiotherapy plan.

      5.Conclusion

      In this study, we firstly propose a novel method that combining dosimetric and clinical factors, radiomics features, 3D CT images, and dose maps for RP prediction. The result shows that the feasibility of the proposed method, which introducing high dimensional features and a novel deep-learning frame. Nevertheless, future multi-institution larger-scale studies are required for validation.

      Funding

      This work was supported by the Integrated innovation and application of key technologies for precision prevention and treatment of primary lung cancer [grant number 2019ZX002]; Research and application of FACE radiotherapy management system in patients with malignant tumor [grant number 2022DBXM005].

      Conflict of Interest

      The authors declare that they have no conflict of interest and comply with the ethical standards

      Declaration of Competing Interest

      The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

      References

        • Sung H.
        • Ferlay J.
        • Siegel R.L.
        • Laversanne M.
        • Soerjomataram I.
        • Jemal A.
        • et al.
        Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries.
        CA Cancer J Clin. 2021; 71: 209-249
      1. F.-M. Kong, R.K. Ten Haken, M.J. Schipper, M.A. Sullivan, M. Chen, C. Lopez, et al. High-dose radiation improved local tumor control and overall survival in patients with inoperable/unresectable non–small-cell lung cancer: Long-term results of a radiation dose escalation study, International Journal of Radiation Oncology*Biology*Physics. 63(2) (2005) 324-333. https://doi.org/https://doi.org/10.1016/j.ijrobp.2005.02.010.

        • Le Q.-T.
        • Loo B.W.
        • Ho A.
        • Cotrutz C.
        • Koong A.C.
        • Wakelee H.
        • et al.
        Results of a Phase I Dose-Escalation Study Using Single-Fraction Stereotactic Radiotherapy for Lung Tumors.
        Journal of Thoracic Oncology. 2006; 1: 802-809
      2. G. Rodrigues, M. Lock, D. D'Souza, E. Yu, J. Van Dyk Prediction of radiation pneumonitis by dose - volume histogram parameters in lung cancer--a systematic review, Radiother Oncol. 71(2) (2004) 127-138. https://doi.org/10.1016/j.radonc.2004.02.015.

        • Pan W.Y.
        • Bian C.
        • Zou G.L.
        • Zhang C.Y.
        • Hai P.
        • Zhao R.
        • et al.
        Combing NLR, V20 and mean lung dose to predict radiation induced lung injury in patients with lung cancer treated with intensity modulated radiation therapy and chemotherapy.
        Oncotarget. 2017; 8: 81387-81393
        • Byhardt R.W.
        • Scott C.
        • Sause W.T.
        • Emami B.
        • Komaki R.
        • Fisher B.
        • et al.
        Response, toxicity, failure patterns, and survival in five Radiation Therapy Oncology Group (RTOG) trials of sequential and/or concurrent chemotherapy and radiotherapy for locally advanced non-small-cell carcinoma of the lung.
        Int J Radiat Oncol Biol Phys. 1998; 42: 469-478https://doi.org/10.1016/s0360-3016(98)00251-x
      3. S. Johansson, L. Bjermer, L. Franzen, R. Henriksson Effects of ongoing smoking on the development of radiation-induced pneumonitis in breast cancer and oesophagus cancer patients, Radiother Oncol. 49(1) (1998) 41-47. https://doi.org/10.1016/s0167-8140(98)00064-4.

        • Krafft S.P.
        • Rao A.
        • Stingo F.
        • Briere T.M.
        • Court L.E.
        • Liao Z.
        • et al.
        The utility of quantitative CT radiomics features for improved prediction of radiation pneumonitis.
        Med Phys. 2018; 45: 5317-5324https://doi.org/10.1002/mp.13150
        • Zheng X.
        • Yao Z.
        • Huang Y.
        • Yu Y.
        • Wang Y.
        • Liu Y.
        • et al.
        Deep learning radiomics can predict axillary lymph node status in early-stage breast cancer.
        Nat Commun. 2020; 11: 1236https://doi.org/10.1038/s41467-020-15027-z
      4. R.J. Gillies, P.E. Kinahan, H. Hricak Radiomics: Images Are More than Pictures, They Are Data, Radiology. 278(2) (2016) 563-577. https://doi.org/10.1148/radiol.2015151169.

        • Wang K.
        • Lu X.
        • Zhou H.
        • Gao Y.
        • Zheng J.
        • Tong M.
        • et al.
        Deep learning Radiomics of shear wave elastography significantly improved diagnostic performance for assessing liver fibrosis in chronic hepatitis B: a prospective multicentre study.
        Gut. 2019; 68: 729-741https://doi.org/10.1136/gutjnl-2018-316204
        • Luo Y.
        • McShan D.L.
        • Matuszak M.M.
        • Ray D.
        • Lawrence T.S.
        • Jolly S.
        • et al.
        A multiobjective Bayesian networks approach for joint prediction of tumor local control and radiation pneumonitis in nonsmall-cell lung cancer (NSCLC) for response-adapted radiotherapy.
        Med Phys. 2018; https://doi.org/10.1002/mp.13029
      5. W. Jiang, Y. Song, Z. Sun, J. Qiu, L. Shi Dosimetric Factors and Radiomics Features Within Different Regions of Interest in Planning CT Images for Improving the Prediction of Radiation Pneumonitis, Int J Radiat Oncol Biol Phys. 110(4) (2021) 1161-1170. https://doi.org/10.1016/j.ijrobp.2021.01.049.

        • Bin L.
        • Yuan T.
        • Zhaohui S.
        • Wenting R.
        • Zhiqiang L.
        • Peng H.
        • et al.
        A deep learning-based dual-omics prediction model for radiation pneumonitis.
        Med Phys. 2021; 48: 6247-6256https://doi.org/10.1002/mp.15079
        • Shinde P.P.
        • Shah S.
        A review of machine learning and deep learning applications,Fourth international conference on computing communication control and automation (ICCUBEA).
        IEEE. 2018; 2018: 1-6
      6. G. Currie, K.E. Hawk, E. Rohren, A. Vial, R. Klein Machine Learning and Deep Learning in Medical Imaging: Intelligent Imaging, J Med Imaging Radiat Sci. 50(4) (2019) 477-487. https://doi.org/10.1016/j.jmir.2019.09.005.

        • Xie Y.
        • Zhang J.
        • Xia Y.
        • Fulham M.
        • Y.
        Zhang Fusing texture, shape and deep model-learned information at decision level for automated classification of lung nodules on chest CT.
        Information Fusion. 2018; 42: 102-110
        • Savarese D.
        Common terminology criteria for adverse events.
        UpToDate, UpToDate Waltham, MA2013: 1-9
        • Pandit S.
        • S.
        • Gupta
        • A,
        Comparative Study on Distance Measuring Approaches for Clustering, International Journal of.
        Research. 2011; 2: 29-31
      7. K. Ahmed, L. Hall, D. Goldgof, R. Liu, R. Gatenby, Fine-tuning convolutional deep features for MRI based brain tumor classification, SPIE2017.

        • Cawley G.
        • N.
        Talbot On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation.
        Journal of Machine Learning Research. 2010; 11: 2079-2107
        • Zhang X.
        • Yang D.
        • Jiang Y.
        • Huang L.
        • Wang C.
        • Tao D.
        • et al.
        Comparison of Radiation Pneumonitis in Lung Cancer Patients Treated with HT versus IMRT and Circulating Lymphocyte Subsets as Predicting Risk Factors.
        J Inflamm Res. 2021; 14: 4205-4215https://doi.org/10.2147/jir.S328955
        • Zhou P.
        • Chen L.
        • Yan D.
        • Huang C.
        • Chen G.
        • Wang Z.
        • et al.
        Early variations in lymphocytes and T lymphocyte subsets are associated with radiation pneumonitis in lung cancer patients and experimental mice received thoracic irradiation.
        Cancer Med. 2020; 9: 3437-3444https://doi.org/10.1002/cam4.2987
        • Saha A.
        • Beasley M.
        • Hatton N.
        • Dickinson P.
        • Franks K.
        • Clarke K.
        • et al.
        Clinical and dosimetric predictors of radiation pneumonitis in early-stage lung cancer treated with Stereotactic Ablative radiotherapy (SABR) - An analysis of UK's largest cohort of lung SABR patients.
        Radiother Oncol. 2021; 156: 153-159https://doi.org/10.1016/j.radonc.2020.12.015
        • Lao J.
        • Chen Y.
        • Li Z.C.
        • Li Q.
        • Zhang J.
        • Liu J.
        • et al.
        A Deep Learning-Based Radiomics Model for Prediction of Survival in Glioblastoma Multiforme.
        Sci Rep. 2017; 7: 10353https://doi.org/10.1038/s41598-017-10649-8
        • Tanaka S.
        • Kadoya N.
        • Sugai Y.
        • Umeda M.
        • Ishizawa M.
        • Katsuta Y.
        • et al.
        A deep learning-based radiomics approach to predict head and neck tumor regression for adaptive radiotherapy.
        Sci Rep. 2022; 12: 8899https://doi.org/10.1038/s41598-022-12170-z
        • Xia X.
        • Feng B.
        • Wang J.
        • Hua Q.
        • Yang Y.
        • Sheng L.
        • et al.
        Deep Learning for Differentiating Benign From Malignant Parotid Lesions on MR Images.
        Front Oncol. 2021; 11632104https://doi.org/10.3389/fonc.2021.632104
        • He K.
        • Zhang X.
        • Ren S.
        • Sun J.
        Deep Residual Learning for Image Recognition.
        IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016; 2016: 770-778
        • Zaheer R.
        • Shaziya H.
        A Study of the Optimization Algorithms in Deep Learning.
        Third International Conference on Inventive Systems and Control (ICISC). 2019; 2019: 536-539
        • Zwanenburg A.
        • Vallières M.
        • Abdalah M.A.
        • Aerts H.J.W.L.
        • Andrearczyk V.
        • Apte A.
        • et al.
        The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping.
        Radiology. 2020; 295: 328-338https://doi.org/10.1148/radiol.2020191145