Individualization of People from Images Richard W. Vorder Bruegge FBI Operational Technology Division Forensic Audio, Video and Image Analysis Unit (FAVIAU) Quantico, VA 22135 703-632-6315 rvorderbruegge@fbiacademy.edu Operational Technology Division DIGITAL EVIDENCE SECTION FORENSIC AUDIO, VIDEO AND IMAGE ANALYSIS UNIT IMAGE ANALYSIS VIDEO FACIAL & CLOTHING COMPARISIONS VIDEO ENHANCEMENT AUTHENTICITY VIDEO RESTORATION AUDIO ENHANCEMENT ON VIDEO SPECIAL EFFECTS HEIGHT DETERMINATION IMAGE AUTHENTICTY IMAGE ENHACEMENT AUDIO AUDIO ENHANCEMENT SIGNAL ANALYSIS AUTHENTICITY VOICE COMPARISON MISCELLANEOUS REPAIRS CASES SUBMITTED „ „ „ „ „ „ „ Terrorism Homicide Armed Robbery Financial Fraud Public Corruption Health Care Fraud Money Laundering EVIDENCE EXAMINED „ „ „ „ „ Film Photographs Video tapes Digital Images Digital videos Forensic Photographic Examinations „ Comparisons of Questioned Images with Known Objects, Places, or Images – Facial Comparisons/Personal Identification – Identification of camera as source of image – Clothing, vehicles, rooms, etc. (Anything that can be photographed can be compared…) Detection of Image Manipulation Artifacts / Image Authentication „ Photogrammetric Examinations „ Information Extraction (Enhancements) „ Forensic Photographic Examinations FBI Laboratory has conducted this work since the 1930’s. „ Bank Protection Act of 1967 „ Expert Witness Testimony „ Open Court „ Daubert & Kumho Tire Standards „ – Accepted field, peer-reviewed, known error rate, testable. – Facial ID Statistics? Personal Identification from Photos / Facial Identification Lack of statistics means: „ Conclusions are ultimately opinion-based. „ – 100% certainty (ID or Elimination) based on knowledge, training, and experience. – Same as fingerprint testimony! – Error rates calculated for practitioners, not technique. „ Forensic science needs more data. Iom DeLuise . . . HIdanti?cation and chef Paul Prudhomm Personal Identification through Photographic Analysis „ Positive Identification of Individuals depends upon presence and visibility of individual identifying characteristics – Ear patterns – Moles, Skin Tags, Birthmarks – Freckle Patterns – Scars – Tattoos – Knuckle Crease Patterns Techniques of Photographic Facial Comparisons „ Morphological Analysis – Point-by-point analysis – Measurements secondary/approximate „ Photo-Anthropometric Technique – Dimensional analysis – Measurements primary, points secondary „ Photographic Video Superimposition – “Real-time” combination of other two – Inherently dimensional, but not “measured” Photographic Comparisons: Principle of Individualization „ The individualization of an item of evidence is established by finding agreement of corresponding individual characteristics of such number and significance as to preclude the possibility (or probability) of their having occurred by mere coincidence, and establishing that there are no inexplicable differences. (Adapted from Tuthill, 1994) Class Characteristics in Photographic Facial Comparisons „ Characteristics shared by persons in a group – Overall shape of face, chin, mouth, nose, eyes, ears … – General shape and characteristics of the hairline, facial hair, eyebrows, glasses... Individual Identifying Characteristics (“Minutiae”) „ Individual Identifying Characteristics differentiate persons from others in a group and can make a person “unique” either alone or in combination „ Moles, freckles, ear patterns, birthmarks, chipped teeth, tattoos … „ Irregularities of hairline, eyebrows, facial hair... Critical Factors in Photographic Facial Comparisons „ Comparable views of Questioned and Known individuals is crucial – Camera-subject geometry » Angle » Distance/Perspective – Lighting/shadows – Image resolution (feature visibility) „ SAME AS FACIAL RECOGNITION Critical Factors in Photographic Facial Comparisons „ Transient nature of human face can affect analysis: – – – – „ Facial expression Changes in weight or age Addition/removal of facial hair Transience of blemishes (acne, freckles, moles…) Intentional alteration: – Make-up – Surgery Visibility of Features in Questioned Images Must be able to see features to identify people and object. „ Visibility is Dependent upon: – Size of characteristic – Shape of characteristic – Contrast of characteristic and background „ » Dark marks on light skin vs dark marks on dark skin allSlde?by?31de comparlson of ear Questioned Known Facial Comparison Mole on Neck uestioned Known FBI Facial Comparisons General Procedure (1/2) Review questioned & known images „ Ensure quality is best available „ – ask for more if needed Matching views selected for best 1:1 „ Consider following for potential overlay: „ – Nostrils’ visibility (tilt of face) – Eyes’ & Ears’ visibility (rotation of head) – If similar in Q & K, then „ Attempt image rotation and resizing for overlay FBI Facial Comparisons General Procedure (2/2) „ Select similar points for rotation of image – Center of pupils or outer corners of eyes for rotation „ Once rotated, resize one (or both) images to scale – Center of pupils, typically, but others sometimes „ Create overlay in Photoshop and compare characteristics: – Relative spacing and size of features – Class and Individual Identifying characteristics – One-page checklist utilized. FBI Facial Comparisons Best Case Scenario „ High Quality Images (Q & K) – Passport-quality photos – Passport-quality lighting (no shadows) „ Same Camera-Subject Geometry „ Potential to photograph subject repeatedly – Multiple photos, multiple angles FBI Facial Comparisons Best Case Scenario „ Recent Case in Washington, D.C. „ Questioned Passport Photo „ Subject in custody thought to be one in passport. „ Differences, however, in skin coloration, hair style and other characteristics led to defense position “Not the same person” Questioned Individual (photograph from Q1 passport) Known Individual photograh (frame 14) from K10 film negatives Side-by-side comparison of (from Q1) and (from K10) i I .q I - auorl?h'. a 1" Side-by-side comparison of (from Q1) and from K10 rv, ?g?.?nxl In. . 4i ~15. Side-by-side comparison of (from Q1) and (from K10) Side-by-side comparison of (from Q1) and (from K10) Reasons for Poor Quality Imagery -Multi-generational copies -Digital image compression -Improperly photographed originals -All of the above -Intentional alterations of images Comparison of Twins ?u'll "th?i? 01:? Pomts 0 omparlson D1 erentlatlon Freckles on order of 1mm or less 12% mm fix? ?3 - awn-n Automated Facial Identification Challenges/Problems/Issues „ Target-Camera geometry – Changes in perspective affect measurements » Great promise in 3D-morphable models » But where will we get 3D images of the bad guys? Changes in facial expression „ Differences in imaging system (cameras and recording devices). „ – Resolution affects ability to identify landmarks – Imperfect optics and other system calibrations... Long Range Challenge (for Law Enforcement and Intel Communities) „ LINKABLE/SEARCHABLE DATABASES: – Multiple agencies, states, and jurisdictions have, or are building, known offender databases, but systems not necessarily compatible or shareable. – Quality of images usually poor » 640 x 480 is NOT GOOD ENOUGH FOR ID! » JPEG damages images – ANSI/NIST STANDARD WILL IMPROVE QUALITY AND HELP DATA SHARING GOAL FOR STANDARD: FACIAL EQUIVALENT OF THE CARD Can new ANSI/NIST Standard Addresses our Requirement? GOAL: Resolution on Face = 0.1 mm (Requires ~ 1700 x 2515 pixels on face) Proposed Level 50/51 Requirements „ 0.1 mm resolution on face requires approximately 1700 x 2515 pixels ON face. – 3300 x 4400 pixels will meet for head & shoulder (50) – 2400 x 3200 pixels will meet for head only photo (51) „ 5-views needed for comparison purposes: – Full Frontal (1) – Full Profiles (2) – 3/4 Profiles (2) Proposed Level 50/51 Requirements „ Full scope of effect of compression remains to be determined. Therefore, for full frontal images, lossless compression is requested for now. – (Further research remains to be done) – 15:1 for profile views is acceptable „ Ears MUST be visible in all photos (if possible). – Ear provides strong potential for individualization and is prominent visible feature in questioned images. Proposed Level 50/51 Requirements „ Acquisition of all images at level 50 should meet or exceed requirements for all other levels. „ In other words, if Level 50 is met, all potential users and applications should have their requirements met as well. „ Note that the 5-photo requirement would also allow for 3-D facial models to be constructed with better quality than with fewer images. Other means of Personal Identification „ Positive Identification of Individuals depends upon presence and visibility of individual identifying characteristics – Ear patterns – Moles, Skin Tags, Birthmarks – Freckle Patterns – Scars – Tattoos – Knuckle Crease Patterns 5? 1. Known Photo of Suspoot . 1 .1 a a 6 Quest Photographic Comparisons and Knuckle and Freckle Patterns Some images do not depict questioned individual’s face but do depict their hands „ Case examples have included: „ – Child abuse – Kidnapping – Murder „ Possible to positively identify suspects if other identifying characteristics are present Questicmed Thumb 0 Side-by-side Comparison Questioned and Known Thumbs Side?by?side Comparison Questioned and Known IndeX Fingers Side?by?side Comparison Questioned and Known IndeX Fingers Side?by?side comparison Questioned and Known Thumbs Raw Image Cropped Processed version of ?DSC00008.jpg? Left Hand of Suspect II .II. I ?f w. Cropped version of Left Hand of Suspect 4" Comparison of Questioned Hand (left) [from ?DSC00008.jpg?] With Known Hand (right) Comparison of Questioned Hand (left) [from ?DSC00008.jpg?] With Known Hand (right) QUESTIONS? Richard W. Vorder Bruegge FBI - FAVIAU - ERF Building 27958-A Quantico, VA 22135 703-632-6315 rvorderbruegge@fbiacademy.edu