Overview
The AI-based system designed to process chest computed tomography (CT) aims to 1) detect the presence of pathologic patterns associated with interstitial changes in pneumonia; 2) highlight areas on the images with the probable presence of pathologies; 3) provide the physician with the results of image processing, including quantitative indicators of suspected viral pneumonia related lung changes according to visual pulmonary lesion grading system (CT0-4).
The retrospective study aims to demonstrate the clinical validation of the AI-based system. Clinical validation measures (sensitivity, specificity, accuracy, and area under the ROC curve) will be determined to provide evidence about the clinical efficacy of the AI-based system.
The hypothesis is that the measures of clinical validation of the AI-based system differ by no more than 8% from those declared by the manufacturer.
Description
The AI-based system designed to process chest CT aims to 1) detect the presence of pathologic patterns associated with interstitial changes in pneumonia; 2) highlight areas on the images with the probable presence of pathologies; 3) provide the physician with the results of image processing, including quantitative indicators of suspected viral pneumonia related lung changes according to visual pulmonary lesion grading system (CT0-4).
This retrospective clinical study will provide the clinical validation of the AI-based system to analyze chest CT images and identify pathological patterns associated with interstitial changes in pneumonia. Clinical validation measures (sensitivity, specificity, accuracy, and area under the ROC curve) will be determined and compared with values declared by the manufacturer to provide evidence about the clinical efficacy of the AI-based system.
The first stage of clinical validation is the collection of a verified labeled dataset. For this purpose, the dataset is collected, labeled, and verified by a research group. The verified dataset should include chest CT images without infiltrative and interstitial lung changes characteristic of viral pneumonia, including COVID-19-associated (CT-0) and chest CT images of all degrees of lung involvement CT-1 (≤25%), CT-2 (25-50%), CT-3 (50-75%), CT-4 (≥75%) [1]. Forming the verified dataset will allow reliable conclusions to be drawn upon completion of the clinical validation. The verified dataset must include a sufficient volume of chest CT images. The verified dataset must be de-identified to ensure the safety of patient personal data.
The second stage of the clinical validation is assessing AI-based system performance by experts. For that purpose, the AI software is analyzed to identify radiological signs of viral pneumonia. Then an examination is made of the correctness of the quantitative assessment of lung damage associated with interstitial changes in pneumonia. The evaluation of both the ability to correctly identify signs of lung damage and to quantify the identified changes is carried out on the same verified dataset.
The third stage of clinical validation is the calculation of clinical efficacy metrics (accuracy, sensitivity, specificity, area under the ROC-curve (AUROC) of the AI-based system by testing it on a verified data set. Testing of the hypothesis to verify the main diagnostic characteristics (sensitivity and specificity) declared by the manufacturer is planned by constructing a two-sided 95% confidence interval (CI), which should not differ by more than 8% from the declared values of 95% and 97%, respectively. Those. the lower limit of the 95% CI for sensitivity should not cross the 87% threshold, and the lower limit of the 95% CI for specificity should not cross the 89% threshold.
All stages of the clinical trial must be under the control of the Principal Investigator.
Randomization of images is not provided in this clinical study, because All CT images will be assessed by the research group and AI software. Also, this design does not involve blinding or masking of the research team. The evaluation of CT images by experts and the software is carried out independently, i.e. the results of each party's assessment are not known to the other party in advance.
Eligibility
Inclusion Criteria:
- General
- Patients over 18 years old;
- Patients who underwent CT without contrast enhancement;
- Patients who underwent a CT scan according to a standardized scanning protocol: 120 kilovolts, slice thickness max. 2 mm, rigid "lung" filter (kernel) reconstruction;
- Patients whose studies should be of acceptable quality, performed with breath-holding, without technical artifacts, and respiratory and motor artifacts;
- Patients whose studies must contain DICOM tags responsible for the orientation and position of the patient in the images during the study, as well as DICOM tags responsible for the size of the scans and image parameters;
- Patients in whom the localization of changes is predominantly bilateral, in the basal and subpleural parts of the lungs, may be located peribronchial;
- For group Normal
- Patients who do not contain COVID-19-related CT patterns;
- For groups Mild, Moderate, Severe, and Critical
- Patients who contain COVID-19-related CT pattern: ground glass opacities (mild, moderate, and higher intensity);
- Patients who contain COVID-19-related CT pattern: pulmonary consolidation;
- Patients who contain COVID-19-related CT pattern: cobblestone infiltration of the lung parenchyma;
- Patients who contain COVID-19-related CT pattern: hydrothorax;
- Patients who contain a combination of one or more patterns.
Exclusion Criteria:
- Patients whose studies contain images with unreported CT patterns;
- Patients whose examinations do not conform to DICOM format;
- Patients whose examinations do not contain imaging of the lung region
- Patients whose examinations contain technical artifacts caused by malfunctions or features of CT scanners;
- Patients whose examinations contain improper patient positioning;
- Patients whose examinations contain studies with deleted DICOM tags responsible for scan size and image parameters;
- Patients whose examinations contain metal artifacts on the patient's body and clothing;
- Patients whose examinations contain the presence of other pathologic changes of lungs in patients - neoplastic, tuberculosis process, bacterial pneumonia, etc.;
- Patients under 18 years old.