Tri-M2MT: Multi-modalities based effective acute bilirubin encephalopathy diagnosis through multi-transformer using neonatal Magnetic Resonance Imaging

Acute Bilirubin Encephalopathy (ABE) is a significant threat to neonates and it leads to disability and high mortality rates. Detecting and treating ABE promptly is important to prevent further complications and long-term issues. Recent studies have explored ABE diagnosis. However, they often face limitations in classification due to reliance on a single modality of Magnetic Resonance Imaging (MRI). To tackle this problem, the authors propose a Tri-M2MT model for precise ABE detection by using tri-modality MRI scans. The scans include T1-weighted imaging (T1WI), T2-weighted imaging (T2WI), and apparent diffusion coefficient maps to get indepth information. Initially, the tri-modality MRI scans are collected and preprocessesed by using an Advanced Gaussian Filter for noise reduction and Z-score normalisation for data standardisation. An Advanced Capsule Network was utilised to extract relevant features by using Snake Optimization Algorithm to select optimal features based on feature correlation with the aim of minimising complexity and enhancing detection accuracy. Furthermore, a multi-transformer approach was used for feature fusion and identify feature correlations effectively. Finally, accurate ABE diagnosis is achieved through the utilisation of a SoftMax layer. The performance of the proposed Tri-M2MT model is evaluated across various metrics, including accuracy, specificity, sensitivity, F1-score, and ROC curve analysis, and the proposed methodology provides better performance compared to existing methodologies.