Multi-Scale Attention-Fusion Classification with an LLM-Driven Clinical Recommendation Assistant for Brain Tumor MRI

Abstract

Brain tumor MRI interpretation remains constrained by radiologist shortages, inter-reader variability, delayed specialist access, and high diagnostic cost, particularly in low-resource settings. Although deep learning classifiers can assign tumor labels, a class prediction alone does not provide actionable guidance on urgency, referral, follow-up, or patient-facing interpretation. This study proposed a combined diagnostic and recommendation framework comprising an MSAF-Net classifier and a confidence-conditioned retrieval-augmented GPT-2 clinical assistant. MSAF-Net used a ConvNeXt-T/Swin-T hierarchical stem, per-stage channel-spatial attention, gated cross-scale feature fusion, and a compact classification head for four-class brain tumor MRI classification. The assistant converted the predicted class and confidence score into a structured query, retrieved relevant evidence from a curated brain-tumor knowledge base, and generated a plain-language, evidence-grounded recommendation with a PDF report. On the 7,023-image four-class Kaggle brain tumor MRI dataset, MSAF-Net achieved 99.16% accuracy, 0.9897 macro-F1, and 0.9985 AUC. Cross-dataset evaluation showed 99.46% accuracy on Figshare CE-MRI and 99.74% accuracy on BR35H. The model remained computationally efficient, with 22.9M parameters, 3.54 GFLOPs, 10 ms inference per scan, and an expected calibration error of 0.0179. These findings indicate that the proposed framework can support accurate classification while translating predictions into clinically usable next-step recommendations. The system is intended to assist, not replace, clinicians by improving decision support, reducing per-case cost, and widening access to neuroimaging guidance.