Multi-task Learning for Intelligent Portrait Composition: Deep Residual Networks and Human Pose Estimation

Abstract

Traditional photography composition is difficult to meet the current needs and aesthetic preferences of portrait photography. Therefore, to improve the composition quality and efficiency of portrait photography, an intelligent composition technology combining multi-task learning framework and deep learning technology is proposed. Firstly, taking the deep residual network as the basic skeleton, the lightweight classification network MobileNet V2 is introduced to improve the portrait detection performance. Afterwards, image quality is improved through the multi-scale feature fusion and multitask learning, and the scale feature extraction is performed on the input image. Moreover, the human pose estimation network is used to detect human keypoints, dynamically adapting to different human poses and scales. The relay node is used to associate the relationship between human instances and keypoints, improving the intelligent composition effect. The effectiveness of the research method is analyzed from three aspects: intelligent cropping for portrait detection, pose recognition analysis, and synthesized image quality. The MS COCO and MuPoTS-3D datasets are selected for evaluation, including recall rate, F1-value, average Intersection over Union ratio (Avg IoU), mismatch, Area under Curve (AUC), and precision-recall curve (PCK). The results showed that the proposed MobileNet V2- ResNet50 achieved an accuracy of 94.51% in extracting facial image information, with an Avg IoU of 0.69, while the Avg IoU of the other three comparison methods was less than 0.65. The proposed fusion algorithm had information entropy, structural similarity, average structural similarity, peak signal-tonoise ratio, brightness relationship factor, and mutual information of 7.596, 1.129, 7.081, 1.828, 1.078, and 8.826, respectively. The overall image quality fusion effect was significantly better than other algorithms. The MobileNet V2-ResNet50 network significantly outperforms the UNet-TransformerCBAM and MCTN models in terms of computational efficiency, with the floating-point operations of only 2.5B, parameter count of 5M, and inference time of 15ms. The research method achieved a human pose detection accuracy of 88.94% on the dataset, with AUC and PCK scores of 44.99 and 83.24, respectively. The detection accuracy of RSC-MS, OP-GAN, and PGF-HPE models did not exceed 88.5%, and their AUC and PCK scores differed significantly from the research method. This method can effectively identify and optimize portrait composition, enhancing the visual effect and artistic expression of photos.