The Theories Used in the Area of Facial Recognition - Research Paper Example

Humans recognise individual through faces and perform the necessary tasks of identifying them. Modern requirements and applications in areassuch as crime, security, and scientific research have required the need to scan across thousands of images and be able to identify individuals through faces. In many cases, the identification process has to scan across thousands of images that are stored in databases for such purposes. The task of scanning, matching and identifying faces is performed by special software applications that are fast and efficient in achieving their objectives. The current paper discusses some of the theories used in the area of Facial recognition and uses their provisions to develop and discuss some of the features of a corresponding software application that was developed. Introduction The challenge of recognising people from pictures has been an intuitive process for humans from our very existence. Although recognising people from the slightest features from images is a simple task for human beings, it is rather difficult to achieve a similar functionality in computers through automated mechanisms. The science of facial recognition, as an integral component of biometrics helps render the recognition of humans an automated process that can be performed by intelligent computer systems and associated software. However, facial recognition has a unique provision by helping in purposes requiring surveillance and is thus an essential practice particularly for governments and securities. Facial recognition is used by Security agencies to track criminals, terrorists, kidnapped individuals including children as well as help identify and secure any important installation or infrastructure from suspicious elements through easy and credible recognition. The technology of facial recognition provides several benefits as will be highlighted through the current discussion whereby systems such as cameras and close circuit television cameras (CCTVs) help capture images of people in public locations such as shops and streets. Further, as images of the faces of people can be captured as far as the resolution and range of the camera extends, facial recognition is a unique tool that helps in securing information on individuals without coming into any sort of physical contact with them and in many cases, such a retrieval is performed in a clandestine and covert manner without alerting the potential suspect (Anil Jain, 2005). Another important aspect of Facial recognition techniques is the development and maintenance of systems that can store and manage records in a database that can be used by the facial recognition software to compare against the retrieved faces from fresh images. In several cases, Facial recognition systems are sophisticated enough to utilise existing and legacy databases that contain the facial details of several individuals (Alan Bovik, 2006). One such example can be found in the case of the records available with law enforcement agencies, which maintain mug shots of criminals as and when they are caught and store them in such systems for further reference. In a similar fashion, facial recognition help perform a speedy comparison of the images obtained from such systems with the faces that have been retrieved on a continuous basis from surveillance cameras on a constant basis. In terms of biometrics, Facial recognition is a way utilising the vision of computers to single out and retrieve the faces of persons and verify them against a set of faces that are held by the system. Irrespective of the procedure used or the methods that are followed to undertake a facial recognition task, the process progresses along a set of 5 sequential steps. Firstly, the requirement for an image of a face is fulfilled by the digital scanning of a photograph or through the use of a camera to obtain the still image of a subject (Harry Wechsler, 2007). In the case of a video input, the input is broken down into a series of time spaced stills that can then be used as the source. The Facial recognition software then tried to detect any faces within the acquired image (Shaun Li, 2004). This is the most complex task in the entire process as a range of generalised patterns and related calculations are employed to identify any faces. Any retrieved face is then analysed in terms of spatial geometry to retrieve any distinguishing features in the face and this is one of the phases where several approaches from different vendors are employed. One of the most popular ways of achieving this face is through the ‘Eigenfaces’ method, which is being combined with neural networks as well as feature analysis to make the related analysis more qualitative. The resulting process helps generate a template, which is simply a reduced data set with only the unique features of the face of the individual (David Connelly, 2007). The fourth step in the process comprises the task of comparing this template with the database of known individuals, wherein the comparison is used to determine the relative closeness among pictures in the form of a percentage rather than absolute resemblance. The ultimate step is where all matches beyond a certain level of comparability are presented, usually sorted by prominence, thereby helping in estimating whether such samples are considerable enough to declare any match (Massimo Tistarelli, 2005). The aim of the current report is discussing and elaborating on a project that was undertaken to develop an application that achieves the task of recognising faces received through a video. The current paper will provide a detailed description of the stages involved and the methods used in face recognition program. Literature Review Linear Discriminant Analysis One of the most important components of the logical approach taken towards the facial recognition program is the method of Linear Discriminant Analysis (LDA), which along with the Fisher’s face method is used to determine the linear combination within features of entities such as images that help differentiate between two or more classes of objects. Further, the combination of these two methods is used for the reduction of dimensionality prior to any subsequent linear classification. Linear Discriminant Analysis is closely associated with regression analysis as well as ANOVA (the analysis of variance). This approach helps estimate the value of a dependent variable as a linear combination that takes other measurements into account. While earlier statistical approaches consider such dependent variables as mere numerical values, the LDA based approach signifies such variables as being categorical in nature. Further, LDA is often associated with PCA (Principal component analysis) as both approaches aim to look at the linear combinations of the variables to determine the nature of the data (Mitchell, 1997). LDA works to model the deviations within classes of data in an explicit fashion, which qualifies it as one of the best approaches for deriving data out of observations in a continuous fashion, thus rendering the value of the output dependent variable to be the most updated. In the case of Facial recognition, every face is comprised of a large quantity of pixel, whose quantity is reduced to a much lesser and manageable quantity through the use of LDA prior to classification. As such, every new dimension serves as a linear combination of pixel values thereby helping in the formation of a template. All such linear combinations that are a resultant of the Fisher’s discriminant method are known as Fisher faces. On the other hand, such combinations that are produced through the PCA method are referred to as Eigenfaces (Turk and Pentland, 2002). Hidden Markov Models Under the Markov process, every system undergoing modelling is treated as a Markov process that possesses an unobserved state. In this context, Hidden Markov Models (HMM) are the simplest versions of a Bayesian network. In the case of a regular Markov model, the observer is clearly aware of the model’s state through the probabilities of the State Transition. However, the hidden version of the Markov Model is one where the state cannot be visualised or perceived clearly and can only be sensed through dependent output. In this scenario, every state is a probability distribution of all possible tokens of output whereby any relevant information of the progression of states is given by these sequences of tokens (Ayinde and Yang, 2002). Such models are termed ‘Hidden’ as the state is studied on the basis of the sequence through which it passes and not the parameters that help define the model. Even when the parameters of the model are known, the method of study still follows along the ‘hidden’ approach. Every face of a specific width W and height H is segregated into overlapping blocks that have a lesser height L and the same width. Additionally, the length of overlap between these images is denoted by P. the pattern is as shown below (Kyrki and Kalviainen, 2004): Max Allen (2007, pg. 43) All such blocks that are denoted by observation vectors T are given by the following formula: In this context, the choice of L and P have a predominant effect on the recognition rate of the system wherein a high rate of overlap leads to a significant leap in the rate of recognition as it allows the capture of features that is more independent of the position. In each of the observation vectors that make up T, the values of the pixels help determine the dimension of the observation vector which is given by (L X W) (Menser and Muller, 1999). However, the use of HMM has two disadvantages when used on the basis of pixels. Firstly, pixels are not known to represent robust features and are thus highly sensitive to any occurrence of rotation or noise. Further, illumination also has influence on the quality of the pixel value. Secondly, the large dimensionality associated with the observation vector tends to induce a high degree of complexity in terms of computation thereby increasing the amount of resources, processing power and time required to produce satisfactory results (Eli Saber, 1998). This is known to be a major problem when using face recognition over large databases and even in situations where facial recognition has to work over real time data and related applications. OpenCV OpenCV is an open source image processing library that was developed by Intel that has simplified the domain of vision programming. The library helps in undertaking advanced tasks such as face detection and face recognition. Additionally, OpenCV can also be used for tracking the movement of faces and is also capable of performing any advanced calculations on the lines of Artificial intelligence methods. Further, the library is available in standard form and can be installed as an additional component to popular development tools such as Microsoft Visual Studio. Further, the user does not have to delve deep into the specifics of advanced algorithms under computer vision, thus allowing them to concentrate on the important task of developing systems for facial detection. As is common case with any software development libraries (popularly known as APIs, Application Programming Interfaces), one needs to be thorough in knowing about all the capabilities of OpenCV is the maximum benefit is to be derived from the provisions of the library. The library is available predominantly in C/C++ and has been downloaded from http://sourceforge.net, a popular repository for open source initiatives. Further, OpenCV is versatile as it is a standalone entity thus eliminating the need to install and configure any additional components in order to develop the required system. besides, OpenCV is also usable across several platforms thus allowing it to be used in Windows, Linux as well as Mac OS X (Gary Bradski, 2008). Since its release in 1999, Open CV has undergone a constant and thorough process of optimization that has made it usable under real time constraints. Further, the library is independent of the platform and facilitates the use of images in several formats. Developers can develop comprehensive face recognition systems through OpenCV that can help in the allocation, copy, setting and conversion of the image. The inclusion of vector manipulation in addition to solvers as well as eigenvalues helps provide for both face detection as well as face recognition (Branislav Kisačanin, 2007). One of the primary reasons for the high performance of any system developed through OpenCV is attributed to the speed with which any image can be processed for any facial patterns. The documentation provided for OpenCV reveals that the library helps achieve this through the use of several data structures such as queues, trees, lists and graphs that help in a fast selection, storage and retrieval of any intermediary variables that are used in several lengthy and complex algorithms. Requirements such as edge detection, corners, interpolation, morphology and image pyramids have all been provided through appropriate procedural calls (Marek Kurzyński, 2007). Another reason for choosing OpenCV for the implementation was due to its capability to make the necessary adjustments for different camera calibrations that helps in the tracking and determination of calibration patterns. Further, OpenCV allows an easy integration of several image manipulations that are required by the user as part of any standard user interface which is facilitated through the display of images and video, handling events from the mouse and the keyboard and the ability to view large and detailed images through resizing without loss of information (George Bebis, 2008). Users of OpenCV at companies such as Magna Donnelly Inc. have acknowledged that the library has helped in rapid application development of face recognition and surveillance systems in a seamless fashion. Kristian Kirk from the same company is of the view that OpenCV is ideal for prototyping new advances in face recognition primarily for its high speed in terms of responsiveness. The Brig Brat Ltd., a maker of innovative surveillance solutions has attributed the vast depth of algorithms provided by OpenCV as the main reason for being able to develop successful products for face and number plate recognition. The provision of numerous mathematical techniques has enabled the development of fast and qualitative products over time (Max Allen, 2007). Methodology As the proposed study involves the incorporation of several theories on Facial recognition and its practices, it is necessary to highlight some of the theories that will be used to develop the corresponding facial recognition application. The application performs several tasks in a sequential manner, wherein each phase works towards achieving a specific purpose for the identification of the faces from captured or retrieved images. In order to study the relevant theories that will go into developing and understanding the working of the application, it is necessary to provide details of such methods that can be sourced from relevant secondary sources of literature which are available in the form of books, journals and magazines. Secondly, in order to develop the application, it is necessary to use certain software libraries such as OpenCV that provide readymade techniques to achieve most of the common purposes of facial recognition applications. As such, it is necessary to provide a brief insight into the provisions of such libraries. In addition to discussing the relevant theories and libraries, the study will also provide details on the specifics and working of the developed software application and substantiate its efficiency through an appropriate discussion of standard tests performed on it. With these discussions, the study aims to provide a thorough overview on the entire application and the elements that have gone into developing it. The application that is discussed by the current report performs the two basic tasks of face detection and face recognition in a sequential workflow. While the former requirement focuses specifically on the Viola-Jones method, the use of Eigenfaces has helped achieve the task of face recognition. The Viola-Jones method represents a new face detection method that is radically faster and is based on the concepts of machine learning such as the AdaBoost algorithm. The use of this method is known to be up to 15 times faster than conventional approaches and can be used to run the detector on complete images without the need to tone down on the search area. Eigenfaces have several advantages over other techniques for facial recognition in that they can be used to achieve the requirements with better speed, accuracy and efficiency. Further, Eigenfaces have proved to operate successfully over large sets of faces in very little time thereby making it an ideal method to implement in the application. Description The application being developed as part of the study is an attempt to create an application that can provide the requirements of facial recognition. The process of recognising faces is achieved through a series of steps, each of which utilises a specific method to provide a required improvement over the data obtained from preceding phases. The task of facial recognition can be divided into three different stages namely the learning stage during application start up, the task of capturing an image within which faces need to be recognised and the last stage of recognising the image. Each of these stages are implemented within the application through three distinct methods. The learning stage is initiated during the start of the application wherein a training set is loaded into the application in order to provide the application with an initial understanding of a few model faces thereby helping it provide with a requisite starting point. This task is achieved through the supply of the data on faces through a text file, which contains information of a person as well as the name of the file containing the image of the face. The relevant images are retrieved and stored in an appropriate array for later use. it is necessary in this context that the application requires at least two training faces, without which an appropriate error is generated. This set of images then undergoes the process of principal component analysis, wherein the correlated variables in the form of the image height and width are converted into corresponding principal (uncorrelated) elements through the use of the eigenvalue decomposition process, which helps provide a single resultant value. These resultant values are stored in a separate ‘projected’ training set matrix, which is stored in an xml file named as ‘facedata.xml’. The next stage is used to detect the image for conducting the facial recognition in an appropriate manner, whereby the source for the input is achieved through a webcam or an AVI movie file. In this context, the application uses the detect_and_draw method to handle nested objects within images through the use of nested cascades. The cascade used for this purpose is loaded into an appropriate object by casting it against the CvHaarClassifierCascade modifier provided by openCV. In addition, the captured image is placed into an object defined by cvCapture and is stored using the cvCaptureFromCAM method call. In cases where a CAM input is not available, the application searches for any available image or an AVI resource for any available image input. Any detected image is then queried for frames using the IplImage object and stored in a required format. Further, in order to improve the tolerance of the faces in the captured images, it was necessary to align all such faces, for which the detectEyes function was included into the code. This function helps detect the left or the right eye and uses its position to align the face within the rectangle thereby making it bigger in size. All training faces used in the test phase had eyes located at (31, 35) for the left and (81, 35) for the right. Such an approach also makes use of an appropriate classifier cascade. In cases where the source is not based on capture and obtained through regular images, the application is capable of making such a distinction and uses inbuilt routines to open all such images that are referenced through a supplied text file. The task of storing such images in an IplImage object and segregating all cascaded objects concludes the capture method of the application. This is followed by the recognition stage wherein the training set generated through the learning phase is first loaded into an appropriate matrix. These eigenvalues are then compared against the captured images from the data stored in the IplImage object. The output then shows the nearest neighbour of every test face that is used in the application in the form of a series of lines printing the details of such neighbours to a relevant test face. Results The current section presents a few details on the tests conducted to evaluate the face recognition application. The testing has been done through the variation of two main parameters namely the angle of the face with respect to the plane of the screen as well as the distance of the face from the screen in cm. The results of the tests are specified depending on the success rate of recognition for a particular combination of angle and distance. This evaluation has also been done for several faces, which are specified by FaceID as shown below. Finally, the average success rate of recognition for a particular angle and distance is specified in the end of the every table. Condition: 1 Angel (Face to plane of screen): 90 degrees Distance: 20-80 cm Face ID Success (%) 1 93 2 79 3 91 4 84 5 76 6 73 7 87 8 92 9 86 10 83 Average 84.40 Condition: 2 Angel (Face to plane of screen): 70-80 degrees Distance: 20-50 cm Face ID Success (%) 1 71 2 58 3 59 4 61 5 43 6 62 7 63 8 62 9 56 10 59 Average 59.40 Condition: 3 Angel (Face to plane of screen): ~45 degrees Distance: 20-40 cm Face ID Success (%) 1 3 2 25 3 0 4 6 5 2 6 12 7 8 8 3 9 6 10 8 Average 7.3 Discussion The development of the application was then followed by the testing phase, wherein the performance of the application was evaluated through successive tests and the code fine tuned to attain satisfactory levels of performance and efficiency. The tests were performed through the use of three different sets of training sets in a successive manner, which were then used to compare against standard input through a webcam. The size of the detector box was set to 112, which was found to work best even in the case of opposite angles although some issues still persist with several other angles of capture. Smaller boxes (30X30) were found to have a higher detection rate for false object (non-faces), while larger ones (of the range 100X100) could not detect faces far away from the camera (smaller faces). As such, the optimal size was found to be 30X30 although this brought the rate of recognition down. The evaluation of the software was performed keeping in mind several potential scenarios that the application could be faced with. Apart from the distance of a potential face from the capture source, it was also essential to test for cases when the face was not aligned completely with the place of the capture source. Testing the application from the perspective of the distance was important to ascertain whether it was possible to recognize faces that had a different size (on the camera view) in comparison to the size of faces used in the training set. Aside from testing this capability, the alignment of the face with the camera plane was also necessary to evaluate as this would help determine the angle to which it was possible to recognize faces successfully. Situations in this regard would be quite commonplace as such an alignment could be evident from a face that was turned away from the camera, or looking up, down or sideways from the camera plane. It is important for the application to recognize faces in all these given conditions. The training sets used in the analysis was a set of 9 different images of the same person. The difference in each of these 9 images was the variation in facial expressions of the individual, which was done purposefully to provide the necessary diversity in the training set. An example of the training set is provided in the appendix. Upon having loaded this training set, the application was connected to a webcam, which served as the input source for the image capture. The video feed through the webcam was captured by the application into successive images at regular intervals and then used for comparison against the images from the training set using the procedures described in the preceding section. A successful match for a given image captured from the webcam against the training set was indicated by a circle over the image from the webcam, which signified that a known face had been detected. The absence of such a red circle around the webcam feed indicated the absence of any match. It was witnessed that beyond a certain inclination of the face from the plane of the webcam, the application was unable to detect any face in the feed although it belonged to the same person. This can be attributed to the relative flat alignment of the faces from the training set, which made it rather difficult to achieve a positive match at steep angles. Further, it was observed that the application succeeded in recognition with near distances with 85% efficiency. However, an increase in this distance caused the recognition rate to come down to a probability of 60% as a result of the box being bigger than the size of the face to be recognized. Also, when testing for efficiency in terms of face alignment, it was observed that a modest rate of success could be achieved for up to an angle of about 30 degrees in any direction from the plane. In all such cases, the face was recognized although the current responsiveness of the application is a little slower in this case. However, there is a scope for further improvement as faces could not be recognized for larger angles of alignment, which necessitates an improvement in existing approaches taken. Despite these limitations, the face recognition application, under its current implementation can be recognized to have achieved its most significant objective of being able to recognize faces up to a basic level and can be considered as a working application from the test results. Conclusion The application developed using the OpenCV library has made use of simple techniques to achieve the purposes of face detection and face recognition. While faces were detected through the webcam input, the subsequent comparison against images from a training set helped simulate the comparison of images from a large database of faces. The use of the library has shown the extreme responsiveness of the application in terms of speed and its effectiveness in capturing images out of a continuous video feed. The application has made use of approaches such as the Viola-Jones method for face detection and the use of Eigenfaces for achieving the necessary face recognition. The main reasons behind choosing these techniques arises from their proven track record in earlier implementation in several research and industrial applications, that has helped establish their credibility as fast and responsive techniques. Much work needs to be done to improve the quality of the application especially in the case of lack of recognition for greater angles with the webcam plane, which is planned to be taken up for improvement in the coming period.  References 1. Anil Jain (2005), Handbook of face recognition. New York: Springer. 2. Alan Bovik (2006), Handbook of image and video processing. London: Academic Press. 3. Shaun Li (2004), Advances in biometric person authentication: 5th Chinese Conference on Biometric Recognition. Guangzhou: Prentice. 4. David Connelly (2007), Intelligent biometric techniques in fingerprint and face recognition. London: CRC Press. 5. Massimo Tistarelli (2005), Advanced studies in biometrics: Summer School on Biometrics, Alghero, Italy. 6. Harry Wechsler (2007), Reliable face recognition methods: system design, implementation and evaluation. New York: McGraw Hill. 7. Gary Bradski (2008), Learning OpenCV: computer vision with the OpenCV library. New York: OReilly. 8. Branislav Kisačanin (2007), Embedded Computer Vision. London: Springer. 9. Marek Kurzyński (2007), Computer Recognition Systems 2. London: Springer. 10. George Bebis (2008), Advances in visual computing: 4th international symposium, ISVC 2008. Las Vegas: Springer. 11. Mitchell (1997), Machine Learning. McGraw-Hill International Editions. 12. Turk and Pentland (2002), Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1), 1991a. Found at URL: http://www.cs.ucsb.edu/~mturk/Papers/jcn.pdf. 13. Ayinde and Yang (2002), Face recognition approach based on rank correlation of Gabor- filtered images. New York: Prentice. 14. Kyrki and Kalviainen (2004), Simple Gabor feature space for invariant object recognition. London: Routledge. 15. Menser and Muller (1999), Face Detection in Colour Images Using Principal Components Analysis. Proceedings Seventh International Conference on Image Processing and its Applications, vol. 2, pp.620–624. 16. Eli Saber (1998), Frontal-view Face Detection and Facial Feature Extraction Using Colour, Shape and Symmetry Based Cost Function, Pattern Recognition Letters. Boston: Prentice. 17. Max Allen (2007), Advances in Facial recognition technologies. New York: McGraw Hill. Read More