Ethical considerations

This study, which is retrospective in nature, was approved by the Institutional Review Board (IRB no. 111199-E and 112136-E). It adhered to the Declaration of Helsinki and complied with the CheckList for EvaluAtion of Radiomics Research (CLEAR).

Inclusion criteria

This study was carried out at a tertiary medical center and we examined the records of patients who visited the out-patient department between January 2007 and December 2021 for SGTs. Our study included adult patients who had undergone US examinations and additional operations or CNBs. Head and neck USs were conducted by two experienced otolaryngologists with over a decade of practice. A Toshiba Aplio 500 (Canon Medical Systems, Tochigi-ken, Japan) was used, which is equipped with a 5–14 MHz linear-array transducer. CNB was carried out when patients were considered unsuitable for open surgery or when they opted against open surgery. Pathological diagnosis, derived from pathological reports, served as the definitive reference for classifying tumors as either malignant or benign. Patients with poor-quality US images were excluded from the study.

Data collection

In this study a total of 294 patients with SGTs were included, 248 of whom had benign tumors and 46 who had malignant tumors. For each patient we collected a single representative US image, specifically the one that depicted the longest axis. These US images were retrieved from the Picture Archiving and Communication System. To maintain patient confidentiality, we removed all identifiable information from the US images, including names, birth dates, admission dates and medical record numbers, using the Snipping Tool (Microsoft). Consequently, we amassed a collection of 248 benign and 46 malignant US images. The clinical data, which included age, gender, smoking status, tumor side, location and size, as well as pathological confirmation and diagnoses, were also collected in this study.

Segmentation and feature extraction

We utilized 3-D Slicer, an open-source medical imaging software equipped with a PyRadiomics package, to select the ROI ( Fig. 1 ). This software facilitates the selection of ROIs and extraction of radiomic features. ROI selection was performed by two seasoned otolaryngologists, each with over 10 y of experience in US practice. They employed manual segmentation techniques, with each checking the other’s work. No disagreement in ROI selection was reported. All 2-D features were extracted from the original US images using PyRadiomics and no image filters were applied in our study.

Selection of region of interest (ROI) in the ultrasound (US) image and extraction of radiomic features using 3D Slicer. Note: The right side is the original US image with the selected ROI. The left side is the panel for selecting and calculating radiomic features.

Number of extracted features

From each of the 296 US images, 105 radiomic features were collected. Subsequently, these images were randomly divided into a training set (235/294, 80%) and a testing set (59/294, 20%). Given the evident data imbalance between benign and malignant cases (248:46), we resampled the training set using ML techniques such as Random Under Sampling (RUS), Random Over Sampling (ROS) and Synthetic Minority Over Sampling Technique (SMOTE) [ ]. These data re-sampling methods balanced both classes by generating new data for the minor class through random duplication (ROS), synthesizing new instances by interpolating among existing samples of the minority class (SMOTE) or reducing the size of majority class via random deletion (RUS). Considering the wide range of these 105 radiomic features, we standardized the training set, resulting in data with a mean value of 0 and a standard deviation of 1. The same standardization was subsequently applied to the testing set.

Feature categories

The 105 radiomic features could be categorized into seven groups [ , ]:

• 1.
  

  Shape-based features (n = 12): These features described the 2-D size and shape of the ROI.

  
  
• 2.
  

  First-order statistics features (n = 18): These features characterized the distribution of pixel intensities within the ROI.

  
  
• 3.
  

  Gray Level Co-occurrence Matrix (GLCM) features (n = 24): These features captured spatial relationships between adjacent pixels.

  
  
• 4.
  

  Gray Level Dependence Matrix (GLDM) features (n = 14): These features quantified the dependence of pixel values by assessing the number of adjacent pixels with the same gray level as the central pixel.

  
  
• 5.
  

  Gray Level Run Length Matrix (GLRLM) features (n = 16): These features analyzed the length of runs of pixel values by calculating the number of consecutive pixels with the same gray level in a specific direction.

  
  
• 6.
  

  Gray Level Size Zone Matrix (GLSZM) features (n = 16): These features quantified gray level zones, defined as the number of connected voxels with the same gray level.

  
  
• 7.
  

  Neighboring Gray Tone Difference Matrix (NGTDM) features (n = 5): These features quantified differences in neighboring pixel intensities.

Feature selection and model establishment

After extracting the radiomic features and pre-processing the data, we employed various ML models for feature selection. These included filter methods (such as ANOVA), wrapper methods (such as recursive feature elimination [RFE]), embedded methods (such as LASSO regression, ElasticNet, Linear Support Vector Classifier [SVC] with L1 penalty) and tree-based methods (such as Random Forest classifier and Extra Trees classifier, both with feature importance) to identify the most relevant features [ ]. Subsequently, we established a diagnostic model using binary classification ML models, including logistic regression, SVC, linear SVC, K-nearest neighbors classifier, Decision Tree classifier, Random Forest classifier and Extra Trees classifier [ , ]. We conducted all possible combinations of data re-sampling, feature selection and binary classification method establishment. All of these steps, encompassing data pre-processing, feature selection and binary classification model development, were executed within the Python framework on Google Colaboratory (Colab), an online Jupyter Notebook platform.

Statistical analysis

A confusion matrix was generated that included measures such as accuracy, sensitivity, specificity, positive-predictive value and negative-predictive value, along with AUC. Sensitivity and specificity pertain to the diagnosis of malignant SGTs. These metrics were obtained by applying the model to the testing set. The model that demonstrated the highest diagnostic accuracy and AUC was selected for application.

Internal validation

After the model was established, we gathered a separate group of patients between January 2023 and June 2023 who fulfilled the same inclusion criteria. This group served as an internal validation for our model and was also assessed using the subjective US model, CT report and fine-needle aspiration cytology (FNAC). A CT report was interpreted as malignant if it reported suspicion of malignancy. An FNAC report was diagnosed as malignant if atypia or malignancy was noted.