Facial Landmarking on the FBI Facial Catalogue - Literature review Example

?Review of Related Literature Crime Trends According to the Committee on Understanding Crime Trends (2008), fluctuations in the number and patterns of crimes have serious implications for the policy-making sector of nations. Increased levels of crime affect not only the criminal justice system but also economic activity. Investors and businessmen always consider the prevalence of crime in a country they wish to expand to. Hence the first thing that attracts their attention is the crime rate, a statistical survey of the criminal offenses that occur in the selected society. Owing to the sub rosa nature of criminal activity, such measurements are often inaccurate, but indicative nonetheless. There are several sources used to gauge crime rate in a country, the first being the official crime statistics drawn from criminal records compiled by the police and law enforcement agencies. However, unlike health, employment and housing, police records are not exactly comprehensive and complete, as some offenses, especially victims under threat and experiencing humiliation (i.e. rape and abuse), are not reported. Thus, governments like that of the United States institute public surveys throughout the different states, along with collating hospital and insurance records to fill the gaps that official police records are afflicted by. Having both economic and social welfare in mind, international organizations like the United Nations (UN) and the International Criminal Police Organization (Interpol) gather and compile crime statistics from different countries to publish worldwide crime indices. One such index was created by the United Nations Office on Drugs and Crime (2009) which features a statistical compilation of various violent crimes that occurred from 2003 to 2008, classified according to the continent and country where they occurred. According to the index, there has been a steady decline of at least 5 – 10% in violent crimes that occurred in the European regions, specifically those involving crimes against property. This decreasing trend in the crime rate can be explained by the widespread installation of surveillance cameras that occurred at the turn of the millennia. Today, surveillance cameras are set up in main throughways, business establishments as well as in households, waiting to record the criminal activities of unsuspecting wrongdoers and help bring them to justice. Facial Recognition Where available, surveillance cameras are invaluable to the police and law enforcement agencies as the video record narrows down their suspect list to persons that closely resemble the perpetrator caught on camera. According to Enciso et al. (n.d), however, matching the 3D image of a person to 2D photographs in a database presents several problems because it involves the processes of visualization and analysis. The procedure is therefore prone to asymmetries and deviations from the norm. Additionally certain video surveillance equipment have poor contrast and resolution or they are mounted at an angle that distorts the face of the criminal. Adding insult to injury, criminals have learned to wear masks to hide their faces. All these hinder precise identification. The following sections discuss the methods, current and previous, used to identify criminals in 2D photograph databases as well as the technology required to automate 3D facial recognition. Identification Methods Anthropometry is the field of science that defines the physical measurements of a person (i.e. size, form, stature) for the purpose of studying human physical and functional variations. According to De Angelis et al. (2008), the identification of the living began in the late 19th century and early 20th century. Discovered in 1883 by Alphonse Bertillon and accordingly termed “Bertillonage”, anthropometric identification was based on the unchanging measurements and characteristics of human body parts. This scientific field therefore worked on the premise that if a database of measurements of specific parts of every individual was recorded, it would be found that every person is unique and distinguishable. This was a revolutionary discovery that was soon adopted by France and other European nations that appreciated how it was particularly helpful to law enforcement agencies. Prior to the discovery of Bertillonage, the police had trouble with the identification of recidivist criminals as their records consisted only of names and general descriptions, which could easily be faked and changed. By that time, photography was already in use as a means to identify criminals. Such efforts proved counterproductive and futile since police officers had to sort through thousands of photos in search of a positive match. Although initially perceived as effective, Bertillon’s system lost favor and was eventually discarded because taking down the measurements was cumbersome compared to the replacement method suggested by Francis Galton. Following the German anatomist Johann Christoph Andreas Mayer’s inspired discovery, Galton postulated that fingerprints were unique to each individual. Using statistical analysis, he proved that the chance of two people having the same fingerprints was 64 billion to one. Thus fingerprinting replaced Bertillonage as a method of identifying criminals, a situation that which persists even in contemporary times. Today fingerprinting is commonplace in police and law enforcement agencies, especially with the development of an electronic database called the Integrated Automated Fingerprint Identification System, which makes serial or second-time offender identification virtually straightforward. The key phrase, of course, is “second-time offender”. One of the limitations of fingerprint identification of criminals is that a previous record of the offender must exist. An even greater problem with fingerprinting is the need for discover and record fingerprints in a crime scene that positively place the suspect on the scene of the crime. Absent eyewitnesses and fingerprints, the evidence is weak. A perpetrator who makes it a point not to leave behind evidence is either experienced or a professional felon and therefore conventional methods of identification becomes useless. Another method of identification is through photograph comparison. The earliest known case that used this method was an English court in 1871. The case of the “Tichborne claimant” concerned a certain Roger Tichborne who was lost at sea in a holiday voyage and declared dead. His mother, Lady Tichborne, refused to accept the truth and continued to search for his lost son. Eleven years later, a man from Wagga Wagga, Australia claimed to be Roger and Lady Tichborne truly believed him to be her son. However, certain “concerned” individuals decided to dispute the man’s claim to the family estate. The trial involved comparing a daguerreotype of the young Roger to an Albumen print of the claimant. At the time however, the technology and methods for proper facial recognition were non-existent and the man was denied his claim and incarcerated for perjury. Today cases concerned with facial recognition like the Tichborne claimant place pressure on the prosecution to properly match the suspect to a photograph, a surveillance tape, or even an artist’s sketch. The latter need calls for the development of technology for identifying criminals accurately and rapidly to ensure that even the slyest criminals can be brought to justice. Facial Measurement and Surface Anthropometry A review by Douglas (2004) describes the different methods used for facial measurement. The most primitive is direct measurement and follows traditional anthropometry where facial measurements are taken using rulers, measuring tapes and calipers. Secondly, there is photogrammetry, the process of obtaining facial measurements from 2D images. Finally, there is the even more complex undertaking called “stereo photogrammetry” based on 3D information of a face being obtained with the use of two or more cameras. When used by experts in the field of anthropometry, the word “photogrammetry” generally refers to obtaining measurements from photographs. However, in the field of architecture and geology. the term is used to refer to the construction of 3D models from a 2D image. When applied to medicine, photogrammetry refers to obtaining measurements from single photographs, while stereo photogrammetry is defined as deriving 3D information of objects from multiple 2D versions. Photogrammetry Farkas and Deutsch (1996) propose indirect measurement methods owing to the numerous drawbacks of anthropometric direct measurement. Direct measurements often lead to inaccuracies, especially when the part being measured consists of soft tissue. Contact made by anthropometric measuring tools cause deformities and therefore imprecise measurements. Moreover, direct measurement is often tedious as facial landmarks have to be located repeatedly for different measurements. This makes the procedure considerably more time consuming for subjects compared to indirect measurements, in which facial measurements are taken from the images. Additionally indirect methods reduce the discomfort experienced by patients because it reduces the time patients have to keep still. Furthermore the said methods fit perfectly for patients with tendencies to misbehave during the procedure, particularly children, thus reducing the inaccuracies it can cause. At the same time indirect methods also reduce the risk of injury to delicate parts (i.e. orbital region) that conventional direct methods cannot. A landmark is defined as a point of correspondence on each object that matches between and within populations (Dreden & Mardia, 1998). Farkas (1994) defined approximately 45 facial landmarks and the methods of measurements along with the required compensation to obtain accurate data from a 2D photograph. He states that facial anthropometry cannot be done accurately on a single photo alone; instead, accuracy is enhanced by taking measurements from different photographs taken at different angles. However, an investigation conducted by Tanner and Weiner (1949) suggests that simple 2D photogrammetry is inadequate for obtaining the measurements of 3D objects as complex as the face. They argue that a great degree of error in measurements occur due to posing and that compared with direct measurements, those obtained by photogrammetry are erroneous and unreliable. This is attributed to the low visibility of facial landmarks and the nature of the print of the photograph. To resolve this shortcoming, Astley and Clarren (2001) trust that modern digital technology has considerably increased the clarity and resolution of 2D photographs. Still, they conceded, adjustments have to be made when measurements are taken from fontal photographs. Ratios, angles, depth and other values are given proxy measurements to compensate for the lack of dimensional or depth measuring scales in photographs. Inconsistency in measurement processes along with the use of proxy measurements to compensate for 2D image analysis makes traditional photogrammetry unreliable at best. The solution is stereo photogrammetry which uses multiple images to create a unique 3D object from two or more projections and therefore allows more accurate measurements to be taken. However this requires elaborate camera calibration to determine the parameters of the 2D images as well as their position and point of reference in the 3D coordinate system. The two most commonly applied photogrammetric procedures used in 3D construction are bundle adjustment and direct linear transformation. Typically, the instrument used for stereo photogrammetry is composed of two or more cameras set at different angles and positions. These take a photo of the subject synchronously or sequentially, to ensure that the position of the subject remains constant throughout the procedure. As mentioned earlier, camera calibration is achieved by either placing calibration articles on the subject, or the subject is placed in a pre-calibrated zone. Allanson (1997) explains that stereo photogrammetry does away with the loss of depth that is the inevitable limitation of single-camera photogrammetry. More than a decade has passed since Aung, Ngim, and Lee (1995) devised a way to increase accuracy significantly by conducting a landmark-based face measurement experiment using laser scanners instead of cameras. The researchers cross-referenced measurements with standard anthropometric measurements to check whether laser scanning was reliable for rapid facial anthropometry. Results of the experiment showed that 12 laser scan measurements were highly reliable to the extent of less than 1 nm offset from physical anthropometric assessment and 16 more measurements were sufficiently “reliable” with less than 1.5 nm variance from their direct measurement reference. In total, the two researchers advanced the science of facial mapping by demonstrating a total of 28 reliable measurements, a development that made laser scanning 33% effective as a rapid recording measurement medium. More crucially, the advantage of using a laser scanner is that surface and contour measurements are taken accurately so long as the subject is positioned at a pre-measured, pre-calibrated distance and a fixed position according to the surface contour of the target face. The rapid collection of data unique to laser scanning also allows the documenting of large populations without the known risk of cumulative radiation in the long term. Although the technology is not completely accurate, laser scanning did boast a high degree of accuracy in measuring specific regions of the face. Facial Landmarks Shi, Samal, and Marx (2006) proposed a different approach to face modeling based on three systems of landmarks. One method involved placing a specific set of landmarks and taking two distance measures based on those. Landmarks are classified into three categories: “pseudo-landmark”, mathematical, and anatomical. Anatomical landmarks are biological locations or points defined by scientific experts. Mathematical landmarks, on the other hand, are defined according to a mathematical or geometric property. Pseudo-landmarks are intermediate, combining two existing anatomical or mathematical landmarks. Of the three types of landmarks suggested by the researcher, only mathematical landmarks represented a significant advance. Since this type of landmark could be defined as mathematical functions, they are easily programmed into computers which can then process, readily locate the landmarks and create 2D or 3D representations of faces. That is the theoretical outcome, at least. A complete review of the equations and solutions that guide the automation of computer facial mapping is beyond the scope of this paper. However, Shi and colleagues had clearly advanced the field for being able to prove that a human face can be effectively modeled by well-defined and biologically meaningful landmarks and their geometrical representations. Although their research was constrained to a number of landmarks, their results show that landmark-based face models and algorithms can reduce the number of people in a search through face recognition. Their method also improved the ability to differentiate subjects according to age and expression. On the other hand Introma, De Donno, Santoro and Carbonara (2007) question the soundness of facial indices as a tool for comparative metrical facial identification. Results showed that craniometric indices are vulnerable to high variability when the subjects’ faces are differently angled, looking in other directions or of different sizes. The latter group of researchers maintain that, however valid the metrical analysis methods, they cannot stand alone. Numerical comparison will be unreliable unless prior parametrical somato-physical superimposition of the subject faces has been done. Simply put, it is possible to make a reliable comparison using mathematical formulas and craniometric indices provided that prior parametric superimposition or direct measurements of facial features of the subject are obtained. Only by resorting to the latter will automated and rapid facial recognition techniques be possible. Facial Recognition Efficiency Zhao et al. (2003) states that human face recognition research has increased drastically since the 1990s as a result of the rising need for commercial security as well as the law enforcement applications it engenders. The development of better hardware and software technology built on such established face databases and standardized face landmarks as has been discussed in the previous sections. Since the bulk of the research on human face recognition automation involves the analysis of face-focused videos and photographs, automated face recognition is indeed a possibility. An earlier meta-review by Chellappa, Wilson and Sirohey (1995) covered the different applications of automated Face Recognition Technology (FRT). Among this is static matching of controlled-format photographs which include passports, driver’s license, company IDs, credit cards and mug shots. Then there is FRT involving dynamic or video matching, which has time and again proven to be complicated even for humans, much less for computers. The potential applications of Automated FRT are endless. However, the technology is far from perfect and can even be considered questionable. Static matching is considerably less complicated than dynamic matching. Yet, even the development of a system to accurately recognize individual photos is already a daunting task. The most common application of static matching are mug shots, which rely on existing databases to obtain a match. Applying FRT in search of a photo match in a database presents the problem of recognition. Typically, a mug shot consists of two or more photos of suspects from different views. Unfortunately, recognition is hindered by the facts there is no uniform standard when these photos are taken; databases also contain different backgrounds, lighting, and subject poses. This problem has been recognized for decades now and has taxed the ingenuity of researchers. Baron (1981) contended that the success of image processing depends on uniform lighting conditions as well as placement of the initial calibration in close proximity to the actual landmark or feature to be recorded. The lack of uniformity in mug shots paired with the inconsistency in the collection of IDs and driver’s licenses makes automated recognition increasingly difficult as the photo of a suspect and their photo references vary greatly. Another potential challenge FRT has to overcome is the feature matching in a large database, as subjects can age, change in appearance, and even undergo dermal (cosmetic surgery) procedures. An experiment by Catterick (1992) tackled the subject of the discriminating power of FRTs by using a camera to record passport shots, other photographs of about the same quality as driver’s licenses, and comparing them to magazine photographs. The experiment tested the discrimination power of FRTs and whether they are impaired by poor quality recordings and irregular angles. The magazines provided a criterion for quality. Results revealed that 66% of the images produced non-discriminate results which are only slightly lower than those produced by passport photos at 72%. This suggests that image quality does not affect FRT reliability to a great degree However, results of the statistical analysis suggests that different camera angles impair reliability, the ability of FRTs to reproduce the same measurements on a single photograph. Additionally, the use of formulas and ratios set to compensate for the difference in measurements caused by variable angles were ineffective and still caused a reduction in discriminatory power. Considering that the performance of FRTs when the only variable was camera angle were already highly unsatisfactory, it stands to reason that confounding features such as facial hair, hair length and accessories would considerably reduce the 66% benchmark even more. To help hurdle this difficult, Iannarelli (1989) argues that facial measurement is only a part of facial recognition. Comparison procedure and factors should expand, he maintains, to such elements like ear shape. Human ears exhibit enough variability in their morphology to allow individualization between subjects; hence, suspect recognition can be done even if an attempt at disguise is made. Hoogstrate, Van Den Heuvel and Huyben (2001) tested this possibility by focusing on the ears of individuals in surveillance camera footage. The findings being inconclusive, the research team argued that more evidence is required to support claims of forensic scientists that ear prints and images have enough variability to allow individualization. In spite of the latter claim, results from their experiment showed that forensically-trained individuals and laymen were able to distinguish a person from surveillance footage through ear morphology alone with over a 65% success rate. Moreover, better surveillance equipment installed at the proper locations and angles further increased the success of identification. Case Study FRT could indeed be groundbreaking when it is perfected. As it stands, however, the technology is limited to subjective observations that only humans can make. Moreover, scientists have yet to agree on the mathematical scales, the method, the technology, and a database on which forensic experts can base their decisions on. As discussed in the previous sections, algorithms, landmark placements, laser scanning and direct measurements were among the parameters tested to increase the accuracy and validity of facial recognition. All of these offered reasonable good, but far from perfect, results. A science like facial recognition or “face mapping” needs to become bullet-proof and infallible, if it is to be used in the justice system especially in cases that determine the fate of an innocent man or restitution to a wronged family. In the case D. Atkins and M. Atkins v The Queen (2009) facial recognition expert Mr. Neave, was called to analyze closed circuit television (CCTV) footage of the robbery in question. His testimony was not given credence by the court and marked inadmissible as evidence against the defendants. Prior to the case of Atkins v. The Queen, the general admissibility of photo-comparison or facial comparison / mapping evidence has been ruled admissible by the court without consideration of the limitations that are suggested in this case. Such cases include Clarke [1995] 2 Cr App R 425 and Briddick [2001] EWCA Crim 984 as summarized by Lord Rose in the Attorney-General’s Reference 2 of 2002 [2003] 1 Cr App R 321 (even though in that case the court was concerned with non-expert evidence of recognition of camera footage). It is recognized in Clarke that a suitably qualified expert with facial mapping skills may properly render an opinion based on his comparison of images from the scene, enhanced or otherwise, and a reasonably contemporary photograph of the accused. For this to be admissible, all the subject photos they should be available for the jury to see. In the present case, the extent and limitations of the evidence did not permit an expert like Mr. Neave to render positive identification. The evidence was good enough to exclude but it could conclusively identify the perpetrator. There was no unique identifying feature in this case and comparisons were not made with any database which could give statistical foundation for his expression of opinion. Accordingly, the evidence given was considered expert but subjective opinion. The lesson to be learned from Atkins is that where a photographic comparison expert gives evidence, properly based upon study and experience, of similarities and or dissimilarities between a questioned photograph and a known person, that expert is not disabled either by authority or principle from expressing his conclusions as to the significance of his findings. In the said case, the expert’s testimony was made in expressions like “Lends strong support” which the court found to be backhandedly conclusive in manner that was considered “forcing the conclusion” of guilt on the jury. According to the said case, he who asserts admissibility must demonstrate it; evidence of opinion is rarely admissible in court. For the expert’s testimony to have been admissible, there must be solid proof that his statement could not be rebuked or falsified. Support was crucially wanting in point of FRT, mathematical equations and procedures that prove with absolute certainty that the person in the surveillance footage was indeed Dean Atkins. Therefore the inefficiency of Facial Recognition Technologies as well as the inability of facial recognition experts to develop techniques that will identify an individual through photos and video footages with 100% precision makes it useless in the courts of Justice. Despite its patent weakness where the judiciary is concerned, face recognition technology has come into use among the police and law enforcement agencies. In the latter case, identification does not have to be 100% precise; it need only be accurate enough for officers to narrow down the suspect list to a manageable number. A good example of benefit for law enforcement is the study conducted by Vanesiz and Brierly (1996) on the use of facial recognition technology as a tool for facial image comparison. Given a sample of 46 crime suspects, facial recognition technology paired with video superimposition was dubbed as “excellent to fairly useful” in 50% of the cases. Such chancy results (equivalent to coin-toss odds) can be ascribed to the diverse problems facial recognition experts encounter. In the course of the study, it was found that the use of video superimposition limited the distinguishing power of the FRT when camera angles and subject orientation differed from the profiled frontal view. As mentioned earlier, photos or footages that differ in angle or position can still be used provided that the expert conducting the evaluation is experienced enough with the equations necessary to compensate for the difference. Where the courts look askance at expert opinion heavily laced with judgment, law enforcement authorities are happy enough with 50% validity if it means reducing the possible number of false negatives. The greater the case load and the larger the suspect list – as may be prevalent for the FBI, Interpol, Scotland Yard and attached agencies of Homeland Security – the higher the likelihood that FRT will be employed to good effect. Bibliography Allanson, J. E. (1997) Objective techniques for craniofacial assessment: what are the choices? American Journal of Medical Genetics. 70 (1), pp. 1 – 5 Astley, S. J. & Clarrenn, S. K. (2001) Measuring the facial phenotype of individuals with prenatal alcohol exposure: correlation with brain dysfunction. Alcohol. 36 (2), pp. 147- 59 Aung, S. C., Ngim, R. C. K. & Lee, S. T. (1995) Evaluation of the laser scanner as a surface measuring tool and its accuracy compared with direct facial anthropometric measurements. British Journal of Plastic Surgery. 48, pp. 551-58. Baron, R. (1981) Mechanisms of human facial recognition. International Journal of Man-Machine Studies. 15, pp. 137 – 178 Catterick, T. (1992) Facial measurements as an aid to recognition. Forensic Science International. 56, pp. 23-27 Chellappa, R., Wilson, C. L. & Sirohey, S. (1995) Human and matching recognition of faces: a survey. Proceedings of the IEEE. 83 (5) pp. 705 - 740 Committee on Understanding Crime Trends. (2008) Understanding crime trends: workshop report[Internet]. Washington D.C., National Academies Press. Available from [Last Accessed 29th July 2011] Enciso, R., Shaw, A., Neumann, U. & Mah, J. (n.d) 3D head anthropometric analysis. Los Angeles, University of South California. De Angelis, D., Sala, R., Cantatore, A., Grandi, M. & Cattaneo, C. (2008) A new computer-assisted technique to aid personal identification. International Journal of Legal Medicine. 123 December, pp. 351 – 356. DOI 10.1007/s00414-008-0311-x Douglas, T, S. (2004) Image processing for craniofacial landmark identification and measurement: a review of photogrammetry and cephalometry. Computerized Medical Imaging and Graphics. 28 June, pp. 401-409. Dreden, I. L. & Mardia, K. V. (1998) Statistical Shape Analysis. New York: John Wiley Farkas, L. G. & Deutsch, C. K. (1996) Anthropometric determination of craniofacial morphology. American Journal of Medical Genetics. 65, pp. 1-4. Farkas, L. G. (1994) Anthropometry of the head and the face. New York: Raven Press. Hoogstrate, A. J., Van Den Heuvel, H. & Huyben, E. (2001) Ear identification based on surveillance camera images. Proceeding of the Forensic Science Society Meeting. 41, pp. 167-72. Hughes, Q. C. (2009) EWCA Crim 1876. Royal Court of Justice. London, Royal Courts of Justice. Case No: 200801604 D4 200801607 D4 Iannarelli, A. (1989) Ear identification. Forensic Identification Series. Fremont, Paramount Publishing Company. Introna, F., De Donno, A., Santoro, V. & Carbonara, M. (2007) Poor reliability of facial indices for comparative facial identification without parametrical superimposition. Forensic Science Medical Pathology. 3, pp. 275-82. DOI 10.1007/s 12024-007-0037-y Tanner, J. M. & Weiner, J. S. (1949) The reliability of photogrammetric method of anthropometry. American Journal of Physical Anthropology. 7, pp. 145 – 186 Shi, J., Samal, A. & Marx, D. (2006) How effective are landmarks and their geometry for face recognition. Computer Vision and Image Understanding. 102, pp. 117- 133. United Nations Office on Drugs and Crime. (2009) The twelfth United Nations survey of crime trends and operations of criminal justice systems [Internet]. New York, United Nations. Available from: [Last accessed 29th July 2011] Venesiz, P., Brierly, C. (1996) Facial image comparison of crime suspects using video superimposition. Science and Justice. 36, pp. 27-34. Zhao, W., Chellappa, R., Phillips, P. J. & Rosenfeld, A. (2003) Face recognition: a literature survey. ACM Computing Surveys. 35 (4), pp. 399-458 Read More