AI-Powered Radiomics for Breast Cancer Prognosis and Aggressiveness Assessment via Ultrasound Imaging
Abstract: Breast cancer is one of the leading causes of cancer-related deaths globally, and breast ultrasound (BUS) is a critical modality for early detection. This thesis introduces several advanced deep learning-based approaches aimed at improving the accuracy and robustness of automated tumor segmentation and classification systems for breast ultrasound images. The first contribution is a computer-aided diagnostic (CAD) system consisting of two stages: segmentation and classification. In the segmentation stage, an encoder-decoder network utilizing various loss functions is proposed to segment breast tumors. The classification stage fine-tunes the MobileNetv2 network to distinguish between benign and malignant tumors. Experimental results show that the WideResNet architecture, combined with Binary Cross-Entropy (BCE) and Dice loss functions, achieves superior segmentation results, with a Dice score of 77.32%, and the CAD system attains a classification accuracy of 86%.
The second contribution presents a novel two-stage deep neural network designed for automatic tumor segmentation. The model comprises a consecutive encoder-decoder (autoencoder) networks, with cost-sensitive learning applied to focus on minimizing segmentation errors for the minority class (tumor). This model outperforms other tumor segmentation methods, achieving Dice indices of 84.49% and 78.94% on the UDIAT and BUSI datasets, respectively. Building on these advancements, the third contribution introduces an extension of the CoAtNet architecture. This dual-stream model applies both convolutional and self-attention blocks at the final layers, allowing the model to aggregate local and global features more effectively, resulting in significant improvements in segmentation accuracy for breast ultrasound images. The fourth contribution introduces CoAtUNet, a hybrid model combining convolutional neural networks (CNNs) and Transformers within a symmetric encoder-decoder architecture for semantic segmentation.
CoAtUNet effectively addresses the challenges of noise, low contrast, and variability in tumor shapes in breast ultrasound images. Evaluations on UDIAT and BUSI datasets demonstrate the superior performance of CoAtUNet, achieving Dice coefficients of 79.12% and 76.21%, respectively. These results indicate the model’s potential in assisting clinicians with accurate tumor detection and reliable diagnosis. The findings of this research highlight the significant potential of deep learning models, particularly CoAtUNet, in advancing automated tumor detection and segmentation in breast ultrasound images. The fifth contribution addresses the inherent low contrast of grayscale breast ultrasound images, which presents significant challenges for accurate tumor segmentation. We propose a novel architecture—Dual-Stream CoAtNet (DSCoAtNet)—enhanced with a pseudocolor transformation layer to improve boundary delineation and emphasize high-contrast features. This model incorporates a preprocessing step that transforms grayscale images into pseudocolored representations using perceptually optimized colormaps, such as Plasma, Magma, Viridis, and Cividis. The enhanced images are then processed through the dual-stream architecture, which captures both fine-grained local details and abstract global context. Experimental results consistently demonstrate improved segmentation accuracy, with DSCoAtNet.v2 using the Cividis colormap achieving a Jaccard index of 72.66% and an F1 score of 80.98%, outperforming the grayscale counterparts. These findings highlight the efficacy of pseudocolor transformations for enhancing deep learning models in low-contrast imaging domains.
Finally, to address inconsistent image quality in breast ultrasound diagnostics, we propose a supervised GAN-based enhancement framework using the Pix2Pix model for image-to-image translation. By leveraging the high-quality UDIAT dataset as a reference and the moderate-quality BUSI dataset as the target, the framework successfully reduces noise and enhances image quality. Experimental results show a PSNR of 25.24 dB, SSIM of 0.8155, and LPIPS of 0.1340, with an average noise reduction of 25.2%. These improvements significantly enhance segmentation performance and provide a reliable preprocessing step for clinical AI applications.