Disease diagnosis in chest radiograph

Classification is a task of predicting a category for a presented example, and one of the perfect applications for this is the task of predicting the diagnosis in chest radiograph images. This task can be done as a binary task in which the categories may be normal versus abnormal (AUC 0.98), ^, normal versus pneumonia (AUC 0.99), pneumothorax versus nonpneumothorax (accuracy 87.3%), and pulmonary tuberculosis versus nontuberculosis (AUC 0.75). A more realistic setting would be to solve this as a multiclass task in which the chest radiograph categories may be critical, urgent, nonurgent, and normal (sensitivities 65%, 76%, 48%, and 71%) or 4 different diseases (AUCs 0.95 for pneumothorax, 0.72 for nodule or mass, 0.91 for opacity, and 0.86 for fracture) or 8 different diseases (average per-class AUC approximately 0.7), or 14 different diseases (average per-class AUC 0.83). The performances for the multiclass algorithms were lower than binary models and were generally good for emphysema, cardiomegaly, and pneumothorax, and poor for infiltration, pneumonia, and mass. However, multiclass algorithms have proven to reduce report time in a simulation study (from 11.2 to 2.7 days for radiographs with critical imaging findings) and improve performance of radiology residents in the emergency department when used as a second reader (sensitivity improved from 66% to 73%). Although perhaps with more limited use, DL-based automatic lung nodule detection algorithms improved nodule detection performances by physicians ^, or show performance superior to that of human readers.

To consider the AI-derived prediction as a second reader, it is also important to understand “how” the algorithm came up with such prediction and radiologic imaging saliency maps are the most common explanation method. Recently, Tang and colleagues used a deep disentangled generative model to generate a patient-specific “normal” chest radiograph image from an abnormal radiograph and obtained residue maps and saliency maps from them ( Fig. 2 ).

Example of synthesized patient-specific normal chest radiograph generated from a deep disentangled generative model. ( A ) Original chest radiograph with abnormalities annotated by radiologists ( pink boxes ). ( B ) Synthesized normal radiograph generated from the original image. ( C , D ) Residual map and saliency map to delineate the potential diseased areas.

( From Tang Y, Tang Y, Zhu Y, Xiao J, Summers RM. A disentangled generative model for disease decomposition in chest X-rays via normal image synthesis. Med Image Anal. 2021;67:101839.)

Lung nodule detection and classification in computed tomography

Low-dose chest CT is now the recommended method for lung cancer screening. However, detecting and classifying pulmonary nodules is a time-consuming task and also prone to interobserver and intraobserver variability. AI-derived algorithms have now gone beyond the old computer-aided detection systems and can suggest a possible diagnosis for the automatically detected lung nodules. With DL-derived methods, the nodule classifications do not have to be limited to the hand-crafted features of the nodule, such as size, density, and growth. DL algorithms can be trained to give a probability of malignancy score with excellent performance in predicting cancerous and noncancerous nodules (sensitivity 94.8%, AUCs more than 0.9 ), surpassing the performance of the Lung CT Screening Reporting and Data System (Lung-RADS) in predicting deaths from lung cancer. Algorithms detecting and classifying nodules into 4 ^, or 6 types (solid, nonsolid, part-solid, calcified, perifissural, and spiculated nodules) have shown performances comparable to human observers, and sometimes better if no previous CT imaging are available. One thing we should keep in mind is that these algorithms were trained on carefully prepared nodule-only data, whereas the real-world data is mixed with nodule and non-nodule diseases. Also, because of the black-box nature of the DL algorithms, it may be difficult to explain why it has come up with such a conclusion or in which step (detection, segmentation, classification) it went wrong.

Classification of diffuse lung disease in computed tomography

Diffuse lung disease includes a group of heterogeneous diseases with considerably different prognoses and treatment. As many of these diseases have similar appearances on CT and tend to have much disagreement between readers, ^, quantitative tools in CT scans are being used to assist in the diagnosis and disease severity assessment of diseases such as emphysema, cystic fibrosis, asthma, idiopathic pulmonary fibrosis, and hypersensitivity pneumonitis. These quantitative tools have been applied with DL-derived algorithms or texture analysis to automatically assess the extent of systemic sclerosis-related interstitial lung disease (ILD), emphysema, and COPD to show good correlation with pulmonary function tests. DL models have also been used to classify lung parenchyma into regional patterns (ground-glass opacity, consolidation, reticular opacity, emphysema, and honeycombing) or classify lung CT scans according to idiopathic pulmonary fibrosis guidelines achieving human-level accuracy. Although with relatively low performance, ILD classifying models that allow the whole lung as input have been reported ^, and should be encouraged.

Predicting prognosis

With the emerging field of radiomics and DL, we can now automatically extract large numbers of features from a lesion of interest and use them to predict pathology, prognosis, and mortality. Risk scores can be obtained from images to predict mortality in chest radiographs (mortality rates were 53.0% and 33.9% for high-risk score groups) and CT (C index 0.72 in smoker group). Radiomics can also find certain features that may have a better response with adjuvant chemotherapy, differentiate squamous cell carcinoma from adenocarcinoma (AUC 0.89), or predict epidermal growth factor receptor genotyping in lung adenocarcinoma CT (AUC 0.8). Studies can be designed to predict the time to progression and overall survival in a baseline CT (hazard ratio 2.8, 2.4) or time-series CT (hazard ratio 6.2 in overall survival).

Disease diagnosis

DL-based diagnosis is outperforming the human-made measurements, such as the hepatorenal index method in liver steatosis or PI-RADS (prostate imaging–reporting and data system) in prostate cancer. With the help of U-Net, radiomics, and ensemble DL algorithms, disease diagnosis can be approached as a binary or multiclass classification problem. The task of classifying abdominal tumors into 2 types or multiple types has achieved variable results. Examples include benign or malignant classification in liver tumors on ultrasound (AUC 0.94), pancreas tumors on CT (sensitivity 97%), adrenal tumors on CT (AUC 0.8), kidney tumors on CT (accuracy 77.3%), prostate tumors on magnetic resonance (MR) images (AUC 0.79), ovary tumors on MR (AUC 0.86), cervical tumors on MR (accuracy 0.9%), 6 liver tumor types on MR (sensitivity 82.9%), and PI-RADS classification on MR (moderate agreement with expert radiologists). The interpretation of these performances can sometimes be confusing, because the imaging modality, tumor types, gold-standard methods, and dataset populations vary between studies. Some of the studies show that they achieve similar or better performances than radiologists in predicting pathology-proven diagnosis, but many of the studies lack comparisons with both radiologists’ evaluation and traditional methods.

Sometimes radiologists are challenged with difficult differential diagnoses that have crucial consequences. This is a good situation to use AI-derived algorithms. Some studies have achieved interesting results in differentiating ambiguous lesions, such as fat-poor renal angiomyolipoma from renal cell carcinoma in CT (AUC 0.96), hepatocellular carcinoma (HCC) from intrahepatic cholangiocarcinoma in CT (AUC 0.72), stromal benign prostatic hyperplasia from transitional zone prostate cancer (AUC 0.9), and benign borderline ovarian tumors from malignant ovarian tumors (AUC 0.89). AI algorithms can also lend a hand in stressful tasks such as detecting vessel invasion of pancreatic cancer in CT (AUC 0.85, comparable performance to senior radiologist), or early-stage cervical cancer in MR (AUC 0.9), detecting extraprostatic extension of prostate cancer (AUC 0.88), detecting local recurrence of rectal cancer anastomosis in MR (AUC 0.86), and detecting HCC in patients with cirrhosis (AUC 0.7). They can even be tried on tasks in which radiologists feel less confident, such as differentiating HCC in noncontrast MR, classifying small hypoattenuating hepatic nodules in colorectal carcinoma CT (AUC 0.84, lower than radiologist), and diagnosing appendicitis in acute abdominal pain CT (accuracy more than 90%).

Disease grading

The classification does not have to be limited to traditional labels. Transfer learning radiomics models can be used to classify liver fibrosis in grayscale and elastogram ultrasounds that perform better than prior liver stiffness measurements. Contrast-enhanced CT scans can be used as inputs to classify grades of liver steatosis (AUC of 0.67, 0.85, and 0.97 for predicting MR proton density fat fraction thresholds of 5%, 10%, and 15%) or liver fibrosis (AUC of 0.96, 0.97, and 0.95 for significant fibrosis, advanced fibrosis, and cirrhosis).

Texture analysis and DL-derived classification algorithms use features beyond the human vision, allowing creative classification tasks, such as predicting estimated glomerular filtration rate (correlation coefficient of 0.74) or interstitial fibrosis and tubular atrophy grades (F1 84%) from ultrasound images, distinguishing aldosterone-producing and cortisol-producing functional adrenocortical adenomas in CT (AUC 0.88), classifying dysplasia grades of intraductal papillary mucinous neoplasia in MR (AUC 0.78), differentiating renal cell carcinoma subtypes in MR (accuracy 81.2%), or grading pancreatic neuroendocrine neoplasm histology types in CT (AUC 0.81). Radiomic features have even shown better results than PI-RADS in diagnosing prostate cancer (AUC 0.96 vs 0.88).

Predicting prognosis

The whole point of classifying any disease into stages or categories is to predict the prognosis and plan a treatment strategy. Radiomics features coupled with machine learning approaches are now used to bypass this hand-crafted staging and directly predict clinical outcome, or predict certain gene expression, and even discover new image biomarkers for well-known diseases. Radiomics features extracted from pretreatment radiologic images have been shown to improve the detection of early recurrence, or predict treatment response or overall survival in HCC, ^, liver metastasis, prostate cancer, and many more malignancies. ^, Models can predict future events of gastro-esophageal variceal bleeding, hepatic encephalopathy in liver cirrhotic CT, or predict intraoperative blood loss in pernicious placenta previa MR (accuracy 75.6%).