An Interpretable Multimodal Deep Learning Framework for Liver Disease Classification Using CT Imaging and Clinical Data

Abstract:

Liver disease accounts for a large share of global mortality, and it frequently progresses without visible symptoms until the damage is advanced. Many computer aided systems for this problem work from a single data source or from opaque models that give a prediction without a reason, which limits their use as a clinical decision support aid. This study presents an interpretable multimodal deep learning framework that classifies liver disease from computed tomography (CT) images and routine liver funct ion tests, using a three class target (benign, malignant, normal) with a softmax output layer, inside a convolutional neural network (CNN). CT scans were resized, normalized, and segmented, while clinical records were imputed and scaled; the training, validati on, and test partitions were created before any of these steps were fitted, so that no information from the held out data could influence preprocessing or feature selection. Feature selection reduced the clinical set through correlation analysis and recurs ive feature elimination, with principal component analysis used only as a variance retention check rather than as the final transform, so the six clinically named variables that passed this pipeline, not derived components, were fused with the image featur es. A MobileNetV3 (a lightweight, mobile optimized convolutional neural network architecture) backbone extracted image features that were concatenated with the selected clinical features before dense layers produced the classification. Three interpretabili ty methods were applied: Grad CAM (Gradient weighted Class Activation Mapping) for image regions, SHAP (SHapley Additive exPlanations) for clinical feature contributions, and LIME (Local Interpretable Model agnostic Explanations) for local per case checks. On a held out test set the framework reached 98.13% accuracy, 93.2% precision, 94.62% recall, and a 93.5% F1 score; whether these figures are macro averaged across the three classes has not been independently confirmed, and a genuine confusion matrix, rec eiver operating characteristic (ROC) and precision recall (PR) curves, and per class breakdown are not yet available from verified test set predictions, so they are reported here as a provisional aggregate result rather than a fully audited one. A represen tative Grad CAM heatmap over a CT image and a rendered LIME example are likewise not yet available in verified form; both methods are described as applied, and their absence as demonstrated figures is stated plainly rather than filled with placeholder imag es. The clinical feature explanations that are available (SHAP and a per instance feature importance chart) point in a clinically plausible direction. Read together, these results are a provisional case for a framework that is both accurate and interpretab le on this task, pending the verification steps detailed in the paper.

Keywords: liver disease classification, convolutional neural network, explainable AI, multimodal learning, Grad CAM, SHAP, LIME