Deconstructing Big Data Complexity: A Multimodal Knowledge Graph-Based Interactive Visualization System for Enhanced Learning

Abstract

This study proposes a novel interactive visualization system that addresses the growing complexity of big data in educational contexts by integrating multimodal knowledge graphs (KGs) with adaptive user interfaces. The system unifies structured and unstructured data from diverse sources, including course ontologies, lecture transcripts, and code repositories, into a cohesive KG framework. Entity extraction, summarization, and cross-modal embedding techniques are employed to construct the KG, which is then visualized through a combination of 2D and 3D rendering engines. The visualization dynamically maps conceptual relationships, semantic similarities, and centrality metrics, enabling learners to explore complex data hierarchies intuitively. Furthermore, the interface supports multimodal interactions such as concept lensing, comparative overlays, and path tracing, while adaptive features like guided tours and annotation tools cater to varying expertise levels. A prototype extension integrates augmented and virtual reality for immersive graph manipulation via hand gestures and spatial audio. The proposed method not only enhances comprehension of intricate data structures but also fosters active engagement through multimodal exploration. Experimental validation demonstrates its efficacy in reducing cognitive load and improving knowledge retention, positioning it as a scalable solution for modern learning environments. The system’s modular design ensures compatibility with existing educational technologies, offering broad applicability across disciplines.