Secure Face Recognition Using Fully Homomorphic Encryption and Convolutional Neural Networks

Abstract

As a unique physiological characteristic, facial information is considered privacy information. This paper combines fully homomorphic encryption technology with a convolutional neural network (CNN) algorithm to develop a face recognition system. The CNN used for extracting facial features was a conventional structure of an CNN, consisting of input layer, convolutional layer, pooling layer, and output layer. The only difference is that during training, triplet samples are used. The differences in convolutional features between the triplet samples were directly utilized as the loss function to train the algorithm. The trained CNN used the convolutional features as facial features. The facial feature vector was encrypted using fully homomorphic encryption technology. Then, the ciphertext was directly used for matching operations to achieve face recognition under encrypted conditions. Finally, simulation experiments were carried out. The simulation experiment used facial data from the Public Figures Face Database. The experiment tested the impact of encryption parameters on encryption effectiveness, as well as the matching performance and security of the face recognition system. The results showed that a high polynomial modulus combined with a low ciphertext coefficient modulus in the encryption parameters led to a decline in both recognition accuracy and efficiency of face recognition. When the ciphertext coefficient modulus was 256 and the polynomial modulus was 1,024, the performance of the system based on fully homomorphic encryption was optimal, achieving a recognition accuracy of 97.7% and an efficiency of 3.34 faces per second. When the matching threshold was 0.8, the recognition accuracy of the system under full-homomorphic encryption was the highest (98.7%). Under the same matching threshold, the recognition efficiency of the system under fully homomorphic encryption was higher than that of the traditional encryption (3.21 per second). In the face of third-party attacks, the face recognition system with fully homomorphic encryption realized the recognition and matching of face feature vectors without exposing plaintext.