A Data-Correlation and Load-Balancing-Based Optimization Framework for Metadata Migration in Distributed Storage Systems

Abstract

Against the backdrop of the rapid development of information technology, the in-depth integration of big data, artificial intelligence, and cloud computing has driven explosive growth in data volume. Distributed storage systems have become core infrastructure, and the efficiency and reliability of metadata management play a decisive role in system performance. This study proposes a metadata migration management architecture based on data correlation and cluster load balancing. By constructing a data correlation graph and a multi-objective optimization model, it realizes the intelligent optimization of metadata layout. This method comprehensively considers access coordination, load balancing, and migration costs; meanwhile, it introduces a bandwidth-aware mechanism and a data popularity prediction mechanism to improve dynamic adaptability. Experimental verification shows that this algorithm can effectively improve system performance and resource utilization. Experimental results show that in scenarios of high-frequency synergetic access and periodic large-scale access, the algorithm achieves a hot metadata recognition accuracy of 93.5%; it mitigates the average metadata access delay to 7.4 milliseconds, and shortens the migration trigger response time to 0.15 seconds. All performance indicators outperform traditional methods such as hash mapping and round-robin migration. The innovation of this study lies in the first deep coupling of metadata semantic association with dynamic cluster load balancing, breaking through the single-dimensional limitations of traditional strategies. The study provides a more intelligent and efficient solution for metadata management in distributed storage systems, effectively improving the system's overall service quality and scalability. This has important theoretical and application value for promoting the evolution of distributed storage towards intelligence and adaptability.