Quality Assurance of 3D Printed Anatomic Models

Quality Assurance (QA), defined as the maintenance of a desired level of quality in a service or a product, is a subset of the quality management system (QMS) and provides the framework that ensures quality requirements are met. A proper QA program has many components that objectively demonstrate confidence in product or service quality to the user. A QA program includes components such as document controls, purchasing controls, process controls, traceability, acceptance standards, and training. Ultimately, the QA program must be commensurate with the product and the potential risk of product failure.

QA in Radiology

In radiology, a QA program ensures the correct operation of imaging systems for the delivery of optimal patient care. Facilities should have documented policies and procedures in place to monitor and evaluate the proper performance, management, and safety of imaging equipment. Ultimately, a radiologist’s final report is the final product being evaluated and the images and imaging equipment are auxiliary.

Errors can occur at many stages and may include the following:

1.

Improperly calibrated image acquisition devices
2.

Inappropriate examination technique
3.

Operator and transcriptional errors
4.

Improperly calibrated display devices
5.

Incorrect interpretation of images

There are many stakeholders within a radiology department who should be involved in a QA program ( Table 7.1 ); and a QA committee including all these stakeholders should meet periodically to review QA issues.

Table 7.1

Roles and Responsibilities for QA.

Derived from Samei, E, et al. Assessment of display performance for medical imaging systems: executive summary of AAPM TG18 report. Med Phys . 2005;32(4): 1205–1225 and Mawlawi OS, Kemp B, Jordan DW, et al. PET/CT Acceptance Testing and Quality Assurance: The Report of AAPM Task Group 126. AAPM; 2019.

Personnel Type	Responsibilities
Radiologic technologist	1. Perform imaging exam and ensure images are delivered to physician 2. Process must be in place to verify arrival of images and a mechanism to detect and correct errors in delivery
Imaging informatics professional	3. Ensure appropriate imaging display, transfer, and archiving into PACS
Medical physicist	4. Acceptance testing of imaging equipment (verifies compliance with local regulatory requirements, compliance with special contractual terms, and compliance with manufacturer’s specifications) 5. Ensure proper calibration and performance of imaging equipment 6. Implement Quality Control (QC) program
Radiologist	7. Support QA program by controlling resources and priorities in imaging departments 8. Demand accountability for image quality and availability
Radiology administrator	9. Allocate resources for QA program and coordinate QA efforts 10. Enforce QA policies and procedures

A process map is a graphical representation of a process including the sequence of all steps that transform inputs to outputs. For a radiological imaging exam, the process is outlined in Fig. 7.1 .

Medical physicists are responsible for acceptance testing and verifying the appropriate setup and performance of imaging equipment. The American Association of Physicists in Medicine (AAPM) has established many Task Groups which focus on QA of medical imaging systems for each specific imaging modality.

In one example, Task Group 18 of the Diagnostic Committee of the AAPM has established a standard of performance for Quality Control (QC) of medical imaging displays. As compared to QA, which is process oriented, QC is part of a quality system which is product oriented and focuses on defect identification. In another example, Task Group 126 is responsible for Positron Emission Tomography/Computed Tomography (PET/CT) acceptance testing and QA; and this group provides recommendations for testing preparations, PET spatial resolution, and PET/CT registration evaluation, PET sensitivity evaluation, PET count rate performance and accuracy of corrections evaluation, PET image contrast and scatter/attenuation correction evaluation, PET image uniformity assessment, and PET scanner QC.

In the United States (US), medical imaging equipment used for diagnosis is considered a medical device and governed by the Food and Drug Administration (FDA). Medical imaging equipment such as CT and magnetic resonance imaging (MRI) scanners are accredited by an accreditation organization, such as the American College of Radiology (ACR) or joint commission. Measurable indicators of the quality of a diagnostic image can be described by contrast, resolution, and noise. Imaging phantoms, objects with specific dimensions and imaging properties, are used as standards to ensure that imaging systems are operating correctly. Tests are performed daily, monthly, and annually depending on the accreditation requirements. Geometric accuracy and spatial resolution are routinely assessed ^, ( Fig. 7.2 ).

Proper interpretation of radiological exams can be assessed by peer review. In peer review, a peer radiologist is asked to re-read cases and determine if he or she agrees with initial report. Radiologists who are outliers among their peers for the number of radiology reports in disagreement can be identified and improvement plans can be implemented. An alternative method for measuring professional outcomes is to score radiological examinations that have a reference standard proof such as knee MRI which can be compared to knee arthroscopy, coronary CT angiography compared to conventional coronary angiography, or CT colonography which is compared with colonoscopy. These methods are preferred since they measure true outcomes for radiologic reports.

QA of 3D Printed Parts in Medicine

To be useful to the clinician, 3D printed models must be a reliable representation of the patient’s anatomy. To address this need, the 3D printing process and output should be covered by a QA program. Similar to QA in radiology, each step of the process must be analyzed from image acquisition to printed part to identify potential pitfalls and risks that could affect part performance. Key steps in the process include image acquisition, image data segmentation, model design, printing, and part post-processing. At minimum, finished parts should go through a QC process to ensure compliance to the device’s quality requirements. The QC process could be a simple visual inspection ensuring that the model is free from manufacturing defects or could be more complex with verification of mechanical performance and measurement of critical model features for accuracy. The level of QC needed will be dependent on the QC plan established for the model as dictated by the QA program.

In the medical device industry, QA is a requirement of the quality system regulation law in many jurisdictions. For example, in the US, a manufacturer of medical devices is charged with establishing and maintaining a quality system that is appropriate for the scope of medical devices that are being produced. The QMS directs the entire scope of activities necessary to ensure confidence in the finished device inclusive of management responsibilities, QA, and QC. Within the QMS, a set of procedures and documents describe the product realization lifecycle from concept to development to production. An important input to product realization during the design control process is the concept of risk. Risk analysis is utilized to identify, assess, and mitigate all potential hazards associated with the medical device. Oftentimes, risk mitigation strategies will involve verification of product requirements through QC inspection.

The internationally recognized QMS standard for medical devices is ISO 13485:2016. This standard specifies the QMS requirements needed by an organization to demonstrate its ability to provide medical devices and related services that consistently meet customer and regulatory obligations. Application of specific clauses in the standard is dependent on the organization’s activities; some requirements can be excluded based on organizational scope.

At 3D Systems, a 3D printing company with medical device development and manufacturing operations, the requirements of ISO 13485:2016 and their relationships are depicted as shown in Fig. 7.3 .

As Fig. 7.3 illustrates, quality is ingrained into all aspects of the organization, which is essential to the production of high quality safe and effective devices that are intended to diagnose, treat, cure, mitigate, or prevent disease. Inherent within the quality system is the application of risk management as described in ISO 14971:2019. Importantly, considerations regarding QA and QC of a medical device need to be done with proper identification of the hazards, evaluation of the associated risks, and implementation of controls to mitigate risks to acceptable levels with monitoring of the effectiveness of the controls.

Bridging the Gap Between Radiology and Manufacturing

With the growing use of 3D printing in hospitals, ensuring the accuracy and reproducibility of the final 3D printed models with a comprehensive QA program is essential. Since 3D printing in hospitals is a relatively new area, there is currently no guidance for the application of relevant standards. Therefore, the onus of responsibility to ensure patient safety is placed on the clinical team. For example, the fabrication of an anatomic model from imaging data intended to aid the clinical team with surgical planning typically includes the following process steps:

1.

Image Acquisition
2.

Image Segmentation and Processing
3.

Anatomic Model Design
4.

3D Printing Preparation
5.

3D Printing
6.

Anatomic Model Post-processing
7.

Packaging and Delivery to Clinical Team

Within each step, there is potential for failure that could propagate throughout the process and yield a model that does not meet device requirements for completeness and accuracy. Therefore, the 3D printing personnel at the hospital will need to evaluate each step and the potential for error to develop an acceptable control plan. Additionally, training of the hospital 3D printing staff on the QA procedures, control plan, and QC requirements is necessary to ensure compliance.

All hospital personnel involved in the 3D printing process (radiologic technologists, biomedical engineers, imaging informatics professionals, medical physicists, and radiologists) should be involved in developing the QA plan. Responsibilities for QA will vary across institutions, but should include at least the following:

1.

Acceptance testing to verify appropriate setup of software, printing hardware, and postprocessing equipment
2.

A process for verifying the accuracy of image acquisition, segmentation, and modeling
3.

Methods to ensure that 3D printed parts are accurate and have the correct specifications
4.

Methods to ensure appropriate biocompatibility and sterilization of models (for models that will be brought into the operating room (OR))
5.

Methods to collect and analyze the QC data
6.

A system for review, approval, and release of the model from the production team to the clinical team.

QA and Optimization of Image Acquisition and Segmentation for Anatomic Models

The first step in creating 3D printed patient-specific anatomic models is to acquire volumetric imaging of the patient. As described above, imaging systems such as CT and MRI machines are medical devices governed by FDA and accredited by an accreditation organization, such as the ACR, and QA of these systems is routinely performed by medical physicists.

Image acquisition and reconstruction parameters can affect image suitability for use in 3D printing, therefore protocols should be optimized. Major factors to consider include slice thickness, spatial resolution, signal-to-noise ratio and contrast-to-noise ratio, and image artifacts. More information regarding image acquisition and specific imaging artifacts can be found in Chapter 2 and will not be discussed here. However, in order to properly visualize and reconstruct the anatomy of interest and be suitable for 3D printing, the image dataset should be free of any major artifacts and visual inspection of the dataset should be performed prior to any modeling to ensure only acceptable artifacts are present. Image segmentation (discussed in detail in Chapter 3) involves separating the appropriate anatomic regions of interest from the surrounding anatomy and is an important step for converting medical images into 3D printed models. The accuracy of the image segmentation is critical to the process and requires a high attention to detail. After converting segmented regions of interest from medical images to 3D polygonal meshes, the mesh representation should be checked for accuracy. Accuracy of the image segmentation may be verified by overlying the final computer-aided design (CAD) file onto the original source images ( Fig. 7.4 ).