A Novel Approach to Fuzzy-Based Facial Feature Extraction and Face Recognition

Generalized two-dimensional Fisher’s linear discriminant (G-2DFLD) is an effective feature extraction technique that maximizes class separability along row and column directions simultaneously. In this paper, we have presented a fuzzy-based feature extraction technique, named fuzzy generalized two-dimensional Fisher’s linear discriminant analysis (FG-2DLDA) method. The FG-2DLDA is extended version of the G-2DFLD method. In this study, we also have demonstrated the face recognition using the presented method with radial basis function (RBF) as a classifier. In this context, it is to be noted that the fuzzy membership matrix for the training samples is computed by means of fuzzy k-nearest neighbour (Fk-NN) algorithm. The global mean and class-wise mean training images are generated by combining the fuzzy membership values with the training samples. These mean images are used to compute the fuzzy intraand inter-class scatter matrices along xand y-directions. Finally, by solving the Eigen value problems of these scatter matrices, we find the optimal fuzzy projection vectors, which actually used to generate more discriminant features. The presented method has been validated over three public face databases using RBF neural network and establish that the proposed FG-2DLDA method provides favourable recognition rates than some contemporary face recognition methods. Povzetek: V prispevku je opisana metoda dvodimenzionalne Fisherjeve linearne diskriminacijske analize na osnovi mehkih množic (FG-2DLDA).


Introduction
Facial feature extraction technique has developed as a popular research area in last 20 years in the field of computer vision, and machine learning [1][2][3][4][5][6]. Very popular linear methods include principal component analysis (PCA) [5][6], linear discriminant analysis (LDA) [7] and their variants, which use Eigen faces and/or Fisherfaces to compute features, fall under this category. In particular, PCA maximizes the total scatter matrix across all face images. However, undesirable variations caused by lighting, facial expression and other factors are retained through PCA techniques [6]. Many researchers argue that the PCA techniques do not provide any information for class discrimination; only perform dimension reduction [6,7]. The LDA has been proposed as a better alternative to the PCA to provide class discrimination information [8,9]. The main objective of the LDA is to find best discrimination of vectors among the classes by maximizing the between-class differences and minimizing the within-class ones [8]. The disadvantage of LDA technique is that, it suffers from the "small sample size (SSS)" problem [9]. The aforementioned problem mainly occurs in case of few numbers of sample than the sample dimension. The dimension of face images is generally very high; as a results, the within-class scatter matrix become singular that makes the FLD method infeasible. The SSS problem in LDA can be solved by sampling down the face images into smaller size [10]. LDA is one of the most important linear approaches for feature extraction which maximizes the ratio of the between-and within-class scatter matrix. However, for a task with very high-dimensional facial images, LDA method may suffer from the problem of singularity. To solve this problem, PCA has been applied to reduce the dimensions of the high dimensional vector space before employing the LDA method [11]. While the PCA seeks projections which are optimal for image reform from a low dimensional space, it may remove dimensions that contain discriminant information required for face recognition. R-LDA method introduces to solve the singularity problem [12]. The main drawback of R-LDA is that the dimensionality of covariance matrix is often more than ten thousand. It is not useful for R-LDA to procedure such large covariance matrix, when the computing platform is not sufficiently powerful. Huang et al. [13] introduced a more efficient null space IDA method. The key idea of this technique is that the with-class scatter matrix ( ) is more effective for calculating discriminant feature, whereas, between-class scatter matrix ( ) is useless. Though, the method is often criticized for the high storage requirement and computational cost in facial feature extraction and recognition. Chen et al. [8] claimed that eigenvectors corresponding to eigenvalues equal to zero of contain the maximum discriminant information. Yu and Yang [14] proposed a direct linear discriminant analysis method by diagonal the between-and within-class scatter matrix. It is well-known that between-and within-class scatter are two important measures of the separability of the projected samples. Independent component analysis (ICA) is also proposed as an effective feature extraction technique [15]. ICA computes discriminant features from covariance matrix by considering high-order statistics. The two-dimensional PCA (2DPCA) directly works on the 2D image matrices and found to be computationally efficient and more superior for face recognition and reconstruction than PCA [16]. Two-dimensional FLD (2DFLD) method maximizes the class separability in one direction (row or column) at a time [17]. The significant characteristic of 2DFLD method is that it directly works on the 2D image matrices. The projection vectors are extracted from the row and, by the G-2DFLD method [18]. The discriminant feature matrices are found by linearly projecting an image matrix on aforementioned directions. Therefore, the discriminative information is maximized by this method among the classes while minimizing it column direction of the training images simultaneously within a class [18]. To increase its pertinence, many LDA extensions, such as direct LDA [19], complete LDA [20], LDA/QR [21] or LDA/GSVD [22], have been developed in the last decades. These extensions try to preserve the same validation and overcome singularity problems either by first projecting the problem in a convenient subspace, using alternative indirect or approximate optimizations.
Very recently, several researchers presented fuzzybased methods, such as fuzzy k-nearest neighbour (Fk-NN) [23], fuzzy two dimensional Fisher's linear discrimination (F-2DFLD) [25], fuzzy maximum scatter difference (F-MSD) [28], fuzzy two dimensional principal component analysis (F-2DPCA) [32], fuzzy two dimensional linear discriminant analysis (GPG-2DLDA), Generalized multiple maximum scatter difference (GMMSD) [33], fuzzy local mean discriminant analysis (FLMDA) [36], and fuzzy linear regression discriminant projection (FLRDP) [37] for feature extraction. Keller et al. (1985) presented the fuzzy k-nearest neighbour (Fk-NN) approach, which fuzzifies the class assignment [23]. This method, popularly known as fuzzy Fisherface [24] (Fuzzy-FLD), which incorporates the fuzzy membership grades into the within-and between-class scatter matrices for binary labelled patterns to extract features and are used for face recognition [25]. The fuzzy 2DFLD (F-2DFLD) is an extension of the fuzzy Fisherface [26]. The scatter matrices were redefined by introducing membership values into each training sample. Yang et al. proposed feature extraction using fuzzy inverse FDA [26]. The Fk-NN was also incorporated in fuzzy inverse FDA for calculating membership degree matrices. The Fk-NN is used to calculate the membership matrix, which is incorporated within the definition of between class and within class scatter matrix [26]. Reformative LDA method is used along with the Fk-NN method to redefine the scatter matrices [27]. A weighted maximum scatter difference algorithm is used for face recognition [28]. Fuzzy LDA algorithm is derived by incorporating the fuzzy membership into learning and random walk method is introduced to reduce the effect of outliers [29]. Fuzzy set theory is integrated with the scatter difference discriminant criterion (SDDC) algorithm where Fk-NN method is used to compute the membership grade which is utilized to redefine the scatter matrices [30]. Fuzzy maximum scatter difference model is proposed where Fk-NN is used to calculate the membership degree matrix of training sample [31]. The Fuzzy 2DPCA method was introduced where Fk-NN method is applied to compute the membership matrix for training sample which was utilized to obtain fuzzy mean of each class. The average of the mean was calculated to define the scatter matrices [32]. Generalized multiple maximum scatter difference discriminant criterion has been introduced for effective feature extraction and classification [33]. Gaussian probability distribution information was incorporated in defining of between class and within class scatter matrices [34]. The membership grade and label information were used to define the scatter matrices [35]. Fuzzy local mean discriminant analysis was employed to construct the scatter matrices by redefining the fuzzy local class means [36]. Fuzzy linear regression discriminant projection method is proposed to compute the fuzzy membership grade for each sample and incorporated in the definition of within class and between class scatter matrices [37].
In the proposed method, we have incorporated the fuzzy membership values in different classes which are computed from the training images (samples). To obtain the membership degrees of each training sample, we have used the fuzzy k-NN and used them for calculating the global and class-wise mean training image matrices. Finally, fuzzy scatter matrices (between and within) are computed distinctly in row and column wise direction. To solve the eigenvalue problem of aforementioned scatter matrices, the features are extracted.
The remaining sections of this paper are organized as follows. In Section 2, we give brief overview of G-2DFLD method. In Section 3, we propose a novel method for feature extraction based on G-2DFLD method, called FG-2DLDA method. The simulation results on three public face image datasets are demonstrated in Section 4. Concluding remarks is given Section 5.

Brief summary of the generalized 2DFLD method
Our presented technique is extended version of the G-2DFLD feature extraction technique [18]. G-2DFLD method is briefly presented in this section. Let, the face images are of × dimension which are represented in the form of 2D vectors ( = 1, 2, … , ). The total number of " " classes comprises face images. The ℎ class is represented by having total samples of and also satisfying the condition (∑ = ) =1 . Given an image , the G-2DFLD-based 2D feature matrix is generated by the following linear transformation: where and are the two optimal projection matrices.
The two Fisher's criteria (objective function) along row and column direction ( ( ), ( )) have been expressed as stated below: The optimal projection vectors and can be obtained by finding the normalized eigenvalues the eigenvectors of −1 and −1 , respectively. The eigenvalues are sorted in descending order and the eigenvectors are also rearranged accordingly [18]. The optimal projection (eigenvector) matrix and can be stated as follows: The between-class and within-class scatter matrices along row direction ( and ) and column direction ( and ) are computed as follows : In above expression, the global mean training image ( = 1 ∑ =1 ) and class-wise mean training image dimensions of the row-wise scatter matrices ( ) and the column-wise scatter matrices ( ) are found to be r×r and s×s, respectively.

Proposed fuzzy generalized twodimensional linear discriminant analysis (FG-2DLDA) method
Human faces are highly susceptible to vary under different environmental conditions, such as illumination, pose, etc. As a result, sometimes, images of a person may look alike to that of a different person. In addition, variability among the images of a person may differ quite significantly. The proposed FG-2DLDA method is basically based on the concept of fuzzy class assignment, where a face image belongs to different classes as characterized by its fuzzy membership values. The idea of fuzzification using fuzzy k-nearest neighbour (Fk-NN) was conceived by Keller et al. and found to be more effective [23]. In the present study, we have used the Fk-NN for generating fuzzy membership values for training images; resulting a fuzzy membership matrix. The fuzzy membership values are incorporated with the training images to obtain global and class-wise mean images, which in turn used to form fuzzy (between-and withinclass) scatter matrices. Therefore, these scatter matrices yield useful information regarding association of each training image into several classes. The optimal fuzzy 2D projection vectors are obtained by solving the eigenvalue problems of these scatter matrices. Finally, the FG-2DLDA-based features are extracted by projecting a face image onto these optimal fuzzy 2D projection vectors. The different steps of the FG-2DLDA method are presented in details in the following sub-sections.

Generation of membership matrix by fuzzy k-nearest neighbour (Fk-NN)
Let, there are C classes and N training images; each one is represented in the form of 2D vectors ( = 1, 2, … , ). A fuzzy k-NN-based decision algorithm has been performed for assigning membership values (degree) to the training images [23,24]. This Fuzzy k-Nearest Neighbour (Fk-NN) method redefines the membership values of the labelled face images. When, all of the neighbours belong to the ℎ class which is equal to the class of ℎ image under consideration, then = and µ returns 1, making membership values for the other classes as zero. In addition, µ also satisfies two obvious ). So, the fuzzy membership matrix using the Fk-NN can be demonstrated as given below: = [ ]; = 1, 2, 3, … , ; = 1, 2, 3, … ,

Fuzzy generalized two dimensional linear discriminant analysis (FG-2DLDA) algorithm
FG-2DLDA methods has employed the fuzzy membership values with the training images and redefine the scatter matrices along row and column directions. Finally, the optimal fuzzy projection vectors are generated by solving the eigenvalue problems of these scatter matrices. Let the training set contains N images of C classes (subjects) and each one is denoted as ( = 1, 2, 3 … , ) having dimension as r×s. The ℎ class , has total images and satisfies ∑ = =1 .
For an image , the FG-2DLDA-based features in the form of 2D matrix of size × is generated by projecting it onto the optimal fuzzy projection matrices and can be achieved by the following linear transformation as defined below: The Fisher's criteria (objective function) ( ) and ( ) along row and column directions are defined as follows: The ratio is maximized in the above equations (7)

Simulation results and discussion
We have assessed the performance of the proposed FG-2DLDA on three publicly available databases namely, FERET [39,40] AT&T [41], and UMIST [42]. The equation for calculating the recognition rate is represented below: where, denotes total number of experimental runs. Correct recognition number in the ℎ run is represented by .
indicates the whole number of test face images.
FERET face database is used to evaluate the FG-2DLDA method under several facial expressions, pose and lighting conditions. AT&T and UMIST database are used to access the presented method under the condition of minor variations of rotation and scaling. In these experiments, we have used a RBFNN classifier due to its superiority and simplicity over the other types of neural networks. As discussed in Section 3 of the proposed FG-2DLDA method, the experiments are performed to validate our claim. The FG-2DLDA algorithm is implemented in C programming language on the Linux operational system with Intel Core i5 (2.4 GHz) and DDR3 (8 GB, 1333 MHz). The suggested method is evaluated on a subset of FERET face database [39,40]. The database consists of 1400 images of 200 individuals and each individual is having 7 images. The images differ in facial expression, illumination and pose. In our study, the facial portion of each original image was lopped and resized to 80×80 pixels based on the location of the eyes. Here, the values of s are taken as 2, 3 and 4 and our method is tried out 10 times with each value of s with the different training sets and test sets. Some examples of images of a person are shown in Fig. 1 (i). A set of 400 images of 40 persons comprise AT&T face database. There are 10 dissimilar images for each person. In our present study, from the set of images for each person, s images are picked out in random from the database to generate the training set and remaining (10s) images are considered as the test set. Hence, a distinct set of images encompasses the training and test set. 3, 4, 5, and 6 are taken as the values of s to form different pairs of training and test sets. Some examples of images of a individual are shown in Fig. 1 (ii). A total of 575 grey-scaled images of 20 different individuals covering a variety of race, sex, and appearance is contained in the multi-view UMIST database. The Face database of images per individual varies from 19 to 48 images. In recent studies, we have diminished each image to 112 × 92 pixels. Fig. 1 (iii) shows one person face image from the database.
We have compared the performance of the proposed FG-2DLDA method with other competent related methods. FG-2DLDA method extracts discriminative feature by calculating the within class and between class

s = 10
The dimension of feature vector Avg. recognition rate ( % ) scatter matrix in row and column direction. Thus, the results again demonstrate the superiority of the FG-2DLDA method over other methods.
In this study, we have validated the performance of our method with 20 different pairs of training and test sets for each value of s on the AT&T face database. Since the present method considers that a face image may simultaneously belong to different classes with possibly different membership values, the class-wise mean images may differ from the actual ones. Fig. 3 Minimum, maximum and average recognition rates of the FG-2DLDA method for different values of s by varying feature size on the AT&T face database. The proposed method yields the best average recognition rates of 93.41% (14×14), 96.08% (16×16), 98.08% (14×14), and 98.68% (18×18) for s = 3, 4, 5, and 6, respectively. Table 3 demonstrates the best average recognition rates achieved by this algorithm for different combination of training and test set. Moreover, we also have compared the result of our method with the other competent methods. In general the face images are severely affected by the different environmental condition. These factors need to be investigated to measure their impact on the intra-class assignment. The scatter matrices involve the overlapping sample distribution information for classification.
In this experiment, UMIST database, to generate distinct pair of the training and test sets we have taken the s as 4, 6, 8 and 10. In this context, each pair of training and test sets is disjoint in nature. The performance of the proposed technique is performed by considering each value of s with 20 dissimilar pairs of training and test sets on the UMIST face database. Fig. 4 also shows the minimum, maximum and average recognition rates (%) of the FG-2DLDA method for different values of s by varying feature size. Table 4 shows a comparative presentation of the FG-2DLDA method along with other contemporary methods in terms of best average recognition rates. The proposed method yields the best average recognition rates (dimension of feature vector) of 86.81% (18×18), 92.75% (20×20), 96.83% (14×14), and 97.3% (14×14) for s = 4, 6, 8 and 10, respectively. In this case, the discriminative information is extracted by calculating fuzzy scatter matrices. The discriminative projection vectors are obtained when the fuzzy scatter matrices are singular. The results show that in all the cases, the performance of the FG-2DLDA method is superior to the other methods.

Conclusion
In this paper, fuzzy generalized two-dimensional Fisher's linear discriminant analysis (FG-2DLDA) method for face recognition is presented. This method assumes that a face image may belong to several classes with possibility of different membership values. These membership values are generated by fuzzy k-NN algorithm and used to generate fuzzy global mean image and fuzzy classwise mean images. Finally these mean images are used to generate fuzzy intra-class and inter-class scatter matrices along row and column directions. The projection matrices obtained by solving these scatter matrices, satisfying the two Fisher's criteria, yield rich information leading to generation of superior discriminant features. Image classification and recognition is performed using a RBF neural network. The performance of our method is validated on the FERET, AT&T and UMIST and face databases. The experimental results demonstrate that the FG-2DLDA method outperforms the competent methods.