Elsevier

Science & Justice

Volume 53, Issue 2, June 2013, Pages 89-97
Science & Justice

Whole-face procedures for recovering facial images from memory

https://doi.org/10.1016/j.scijus.2012.12.004Get rights and content

Abstract

Research has indicated that traditional methods for accessing facial memories usually yield unidentifiable images. Recent research, however, has made important improvements in this area to the witness interview, method used for constructing the face and recognition of finished composites. Here, we investigated whether three of these improvements would produce even-more recognisable images when used in conjunction with each other. The techniques are holistic in nature: they involve processes which operate on an entire face. Forty participants first inspected an unfamiliar target face. Nominally 24 h later, they were interviewed using a standard type of cognitive interview (CI) to recall the appearance of the target, or an enhanced ‘holistic’ interview where the CI was followed by procedures for focussing on the target's character. Participants then constructed a composite using EvoFIT, a recognition-type system that requires repeatedly selecting items from face arrays, with ‘breeding’, to ‘evolve’ a composite. They either saw faces in these arrays with blurred external features, or an enhanced method where these faces were presented with masked external features. Then, further participants attempted to name the composites, first by looking at the face front-on, the normal method, and then for a second time by looking at the face side-on, which research demonstrates facilitates recognition. All techniques improved correct naming on their own, but together promoted highly-recognisable composites with mean naming at 74% correct. The implication is that these techniques, if used together by practitioners, should substantially increase the detection of suspects using this forensic method of person identification.

Introduction

It is common practice for police to invite eyewitnesses to help them solve crime. These witnesses and victims can be a valuable resource at the initial stage of an investigation when there is no apparent evidence available to locate an offender (e.g. from CCTV footage or DNA analysis). If a witness or victim has clearly seen either the offender's face or another person who is unknown but nevertheless may potentially have valuable evidence, he or she may be able to describe the appearance of the face and construct a facial composite. Composite images produced in this way can be circulated within a police force, or more generally in the media, with the aim that someone will name the face and thereby generate new lines of enquiry. Potential suspects can be identified or eliminated and ideally the offender brought to justice.

Facial composites have been in regular use for the principal purpose of detecting offenders for about 40 years (for recent reviews, see [18], [20], [21]). There are three basic methods currently available to police to construct the face. First, there are feature-based computer systems which involve the selection of individual facial features (eyes, nose, mouth, etc.). Witnesses describe the face they have seen and this description is used to locate matching features within the composite system. They view these choices and are asked to select the best matches, which are then sized and positioned on the face with the aim of producing the best likeness. Software systems for constructing composites include E-FIT and PRO-fit in the UK and Europe, and FACES and Identikit 2000 in the US. Second, there are sketch artists who produce composites using pencils or crayons using a similar feature-by-feature method.

Research has indicated that feature-based methods produce fairly identifiable composites when the delay is fairly short between a person (observer) seeing a target face and constructing a composite (e.g. [6], [7], [13], [22], [26], [35]). When the retention interval is up to a few hours in duration, for example, composites are named with a mean accuracy of around 10 to 20%.

Considerable research effort has been spent attempting to improve the performance of composites, and this has been met with success in three areas.

The first area of success relates to the interview. It is normal for witnesses (including victims) to initially undergo a cognitive interview (CI) to help them recall details of the target's (offender's) face; this allows appropriate facial features to be located within the composite system for witnesses to select from. The CI has been extensively developed (e.g. [54]) for use in forensic and other applications and contains a set of memory-facilitating techniques or mnemonics to facilitate information recall. For face construction, mnemonics normally include a request to describe the face (a) in an uninterrupted (free-recall) format, (b) in as much detail as possible but (c) without guessing (for more details, see [25]).

The first improvement involves administering an additional ‘holistic’ mnemonic to the CI (e.g. [20]). This aims to improve witnesses' face recognition and consequently allow them to select facial features with greater accuracy. The technique is based on the established ‘deep’ over ‘shallow’ processing advantage for encoding a target: face recognition is facilitated when a target face is remembered by attributing overall (holistic) judgements (e.g. attractiveness, masculinity, distinctiveness) rather than by physical attribution such as size of mouth or length or nose (e.g. [46]). The benefit of holistic over physical-feature attribution has similarly been found when carried out immediately prior to face recognition (e.g. [5]).

With face construction, the holistic element of the interview requires witnesses to (i) think freely about the character of this face (silently to themselves for 1 min) and (ii) make seven overall (holistic) judgements about it (e.g. intelligence, friendliness and distinctiveness). In [24], composites constructed using the PRO-fit feature system after this ‘holistic’ cognitive interview (H-CI) were correctly named at 41.2% compared with 8.6% for composites constructed after the traditional CI.

The second area of success relates to the naming stage—that is, when another person attempts to recognise a composite, as would be carried out by a police officer or member of the public in a police appeal for information. The main problem is that composites constructed from memory contain error, with inaccuracies present in the composites' shape and placement of facial features: for this reason, composites do not evoke perfect recognition. Despite developing methods to reduce error, such as the H-CI, there are often marked differences between a person's appearance and a constructed likeness of him or her, the outcome of which is to limit recognition (e.g. [34]). Two techniques have successfully facilitated recognition.

One technique is to artificially increase the level of distinctiveness by exaggerating distinctive features and configural relations (distances between features) through caricature. Positive caricature facilitates recognition of line drawings of faces (e.g. [4]) and briefly-presented photographs of faces (e.g. [41]). Observing a composite with a range of caricature levels, from negative to positive, provides one frame (one level of exaggeration) that is a better probe to memory than the veridical composite, facilitating recognition. In [23], multi-frame caricature improved correct naming of composites by about 50%.

The other technique emerged from a curious line of research involving transforming faces by linear stretch, an affine transformation. When applied to photographs, by doubling the height, [39] found that recognition accuracy of familiar (celebrity) faces did not decrease relative to unaltered images (although reaction times were slower in one of their four experiments). However, as described in [2], doubling height or width of familiar-face (celebrity) composites substantially improved an observer's ability to correctly name the face. The technique is impractical for publishing witness composites in the media, as stretched images look silly for the serious application of identifying offenders. It was found, however, that the technique was also effective when participants were asked to look at the face sideways, to give the appearance of a thin face (there is also a smaller effect of perspective, discussed in Section 3.2). Relative to looking at the face front-on, this ‘perceptual’ stretch about doubled correct naming, from 17% to 36%.

[39] argued that, for photographs of faces, an observer's cognitive system may be normalising a stretched face prior to recognition, to fit with how a face usually appears, or that the memory itself may be altered to accommodate the transformation, a ‘deformable template’ theory. More recent research by the first author (unpublished) has found that the null effect of stretching extends to unfamiliar-face recognition, which argues for the former case: normalisation of an incoming face not deformation of memory. For composites, be it physical or perceptual stretch, the advantage appears to be based on reducing the perception of error: in the process of normalising the face for comparison with existing (un-stretched) memories, some inaccuracies in the composite are reduced. In particular, information in the horizontal plane is condensed with a side-on perspective, making it ambiguous: this ambiguity can be resolved in an observer's cognitive system in effect by ‘filling in the details’, facilitating recognition. In a related study, [19] simplified information in a normal photographic-type composite, by converting it into a sketch-like image. For the same reason as being suggested here, this image transform facilitated naming of composites that were poorly recognised (i.e. images with inaccurate facial texture). The most likely candidates underlying the benefit of (perceptual) stretch are the individual features, whose likeness is known to positively correlate with composite naming [24], and the region around eyes, in particular the eye spacing, known to be important for recognition (e.g. [14], [40], [55]). So, inaccuracies present in both individual features and spacing in the eye region may be reduced with side-on viewing, augmenting recognition.

Unfortunately, in spite of these improvements, when the retention interval is upwards of 24 h, the normal situation in police investigations, most people struggle to recall details of an offender's face and select individual features. It is under these circumstances that identifications arising from feature-based composites tend to be infrequent in police investigations [27], an observation supported by laboratory research (e.g. [25], [30], [32], [42], [43]). In fact, many practitioners find it extremely hard to create a composite with witnesses that have poor face recall; also, under these circumstances, UK police guidelines advise against production of a feature-based composite [1].

While the difficulty with face recall is by no means new (e.g. [12]), there is another important theoretical issue: construction usually involves an unfamiliar face, a person that the witness has not knowingly seen previously, but recognition of the composite is carried out by a person familiar with the offender [22]. It is known (e.g. [15]) that while familiar-face recognition tends to involve internal facial features (the region including eyes, brows, nose and mouth), recognition of unfamiliar faces is biased strongly by external facial features (hair, face shape and ears). For face construction, the consequence is that the internal region tends not to be created accurately, which is arguably the reason why traditional composites are not often recognised correctly (e.g. [29], [35], [22]).

Alternative methods have been sought to access facial memory: the resulting recognition-based (or ‘holistic’) systems form the third area of success emerging from research. These systems do not have a database of individual facial features; instead, they create a face space (model) constructed from faces in their entirety (for a discussion on different types of face spaces, see [52]). These models are built using the statistical technique, Principal Components Analysis (for a review, see [28]). The face space is holistic in nature, with each parameter representing an overall aspect of the face (e.g. [37]); one parameter might code for face width, for example, while others attend to age and complexion. When these holistic parameters are given random values, the result is realised as a synthetic yet plausible-looking face—it is, in fact, a point in face space. This can be repeated to produce other plausible-looking unfamiliar faces—other points in face space. The goal of the method is to search the space to locate a set of parameters which together represent the appearance of the offender's face, as seen by the witness. Estimating these parameter values accurately, based on feedback from a witness (user) during face construction, is the challenge.

The approach is generally based on the idea that faces are recognised as whole entities rather than by their isolated facial features (e.g. [11], [49]). Also, while face recall decays quickly (e.g. [16]), which is arguably the Achilles' heel of the feature systems (e.g. [25]), face recognition remains largely intact after a forensically-relevant interval of several weeks (e.g. [47]). Accordingly, these systems principally use a face-recognition task: witnesses are asked to select the overall closest matches from an array of randomly-generated faces of appropriate age, gender and ethnicity. These choices are ‘bred’ together, usually using an Evolutionary Algorithm to combine parameter values for pairs of faces, to produce further choices for selection. When repeated a few times, a composite is ‘evolved’—or, more specifically, the space is searched to locate a point in multidimensional space (a set of parameter values) that best represents the target.

There are three main recognition-based systems: ID [51], EFIT-V [36] and EvoFIT [20]. While there does not appear to be much research published on the first two, EFIT-V does appear to be somewhat successful, producing composites after a short target delay that are named about as accurately as feature-based composites [53].

We have been researching and developing EvoFIT for about 15 years now (for a review, see [18]). In the original design, face constructors selected external features to match those of the target, and these were presented in arrays of 18 faces from which users made repeated selections. Choices were based initially on facial shapes, then facial textures and finally on the overall appearance; selected items were ‘bred’ together, to combine characteristics, as described above, and the process repeated. There have been three milestones in development.

The first development facilitated construction of the internal features, the important region for composite naming [35]. A Gaussian (blur) filter was applied to external features in the face arrays. The blurring level was such (8 cycles per face width) as to render recognition difficult if extended across the image (e.g. [50]); here, it was used to de-emphasise the external features and increase the saliency of the internal features. After the face had been evolved, blurring was disabled to allow the entire face to be seen clearly. Alongside this development, a set of ‘holistic’ tools were created for a user to enhance the overall likeness of an evolved face. These could be used to change age, weight, masculinity and seven other overall properties of the face. In [32], composites constructed after a 2 day delay using blur and holistic tools were correctly named with a mean of 24.5% compared to 4.2% for composites where only holistic tools were used (i.e. with non-blurred external features presented in the face arrays).

The second development considered in more detail the importance of the exterior region with face construction. It was expected that external features would, to some extent, provide a beneficial context in which to select internal features: some level of blur would be optimal for face selection. The idea is similar to the recognition benefit achieved when the background scene (context) does not differ greatly between a person seeing a target face and attempting to recognise it (e.g. [44]). The outcome for composites was counterintuitive: higher levels of blur led to more identifiable images, with an “infinite” level of blur emerging the best—filtering that revealed only the internal features. In [34], correct naming approximately doubled from 22.7% for composites created using the previous external-features' blur method to 45.6% with internals only. So, the mere presence of external features appears to be a distraction to the person constructing the face: best identification is achieved by presenting just internal features in the face arrays for selection (and adding external features after holistic-tool use).

A recent police-field trial using a version of EvoFIT including this development indicates very-good performance in the real world: 60% of EvoFIT composites constructed by witnesses and victims in a 12 month period directly led to identification of a suspect, and 29% of these identifications resulted in conviction [33].

The third development applied the H-CI to face construction using EvoFIT. In [31], participants looked at a target video and, the following day, received either CI or H-CI and then evolved the face using the blurred external-features procedure. Correct naming of composites increased from 24.1% made after a CI to 39.4% after an H-CI. We argued that the procedure helped a participant-witness's ability to select faces from the arrays based on the overall appearance of the face, to create a representation that was more recognisable (since recognition is based on the perception of the entire face).

The aim of the current experiment was to explore the impact of combining the above techniques at face construction and naming. We did not know quite how recognisable composites would be were the H-CI to be used in conjunction with internals-only construction. Similarly, would performance increase further using techniques for facilitating recognition of finished composites? One would anticipate that effects would be additive; if this is the case, then augmenting face construction in this way is theoretically interesting as well as providing a worthwhile improvement for police practitioners. In the following experiment, we systematically investigated three factors that have previously been found to increase composite naming: interview type (CI/H-CI), construction type (external-features blur/internal-features only), and one of the procedures for facilitating recognition of a finished composite, angle of view at naming (front on/side on).

Section snippets

Experiment

Two stages were required to conduct the investigation, which are described below, composite face construction (Stage 1) and composite face naming (Stage 2).

Discussion

Recovery of an identifiable face from memory can be of enormous value for identifying people who commit crime. Until quite recently, methods available to the police had not consistently produced identifiable faces. The feature systems in particular do not appear to provide a good interface to memory, especially when deployed after long intervals of time. In the current project, we investigated three techniques which have been found to improve the effectiveness of composites created using

Acknowledgments

This research was supported in part by an Economic and Social Research Council grant (RES-000-22-4150) awarded to Dr Charity Brown and Dr Charlie Frowd.

References (57)

  • R. Cabeza et al.

    Features are also important: contributions of featural and configural processing to face recognition

    Psychological Science

    (2000)
  • S. Coren et al.

    Sensation and Perception

    (1999)
  • G.M. Davies et al.

    Face recall: an examination of some factors limiting composite production accuracy

    Journal of Applied Psychology

    (1982)
  • G.M. Davies et al.

    Facial composite production: a comparison of mechanical and computer-driven systems

    Journal of Applied Psychology

    (2000)
  • R. Diamond et al.

    Why faces, are and are not special: an effect of expertise

    Journal of Experimental Psychology. General

    (1986)
  • H.D. Ellis et al.

    Identification of familiar and unfamiliar faces from internal and external features: some implications for theories of face recognition

    Perception

    (1979)
  • H.D. Ellis et al.

    The deterioration of verbal descriptions of faces over different delay intervals

    Journal of Police Science and Administration

    (1980)
  • F. Faul et al.

    G*Power 3: A flexible statistical power analysis program for the social, behavioural, and biomedical Sciences

    Behavior Research Methods

    (2007)
  • C.D. Frowd

    Facial recall and computer composites

  • C.D. Frowd et al.

    Evolving the face of a criminal: how to search a face space more effectively

  • C.D. Frowd et al.

    Changing the face of criminal identification

    The Psychologist

    (2008)
  • C.D. Frowd et al.

    Evolving facial composite systems

    Forensic Update

    (2009)
  • C.D. Frowd et al.

    Parallel approaches to composite production

    Ergonomics

    (2007)
  • C.D. Frowd et al.

    An application of caricature: how to improve the recognition of facial composites

    Visual Cognition

    (2007)
  • C.D. Frowd et al.

    Improving the quality of facial composites using a holistic cognitive interview

    Journal of Experimental Psychology. Applied

    (2008)
  • C.D. Frowd et al.

    Contemporary composite techniques: the impact of a forensically-relevant target delay

    Legal and Criminological Psychology

    (2005)
  • C.D. Frowd et al.

    A forensically valid comparison of facial composite systems

    Psychology, Crime & Law

    (2005)
  • C.D. Frowd et al.

    Catching more offenders with EvoFIT facial composites: lab research and police field trials

    Global Journal of Human Social Science

    (2011)
  • Cited by (37)

    • Composite sketch recognition using saliency and attribute feedback

      2017, Information Fusion
      Citation Excerpt :

      Researchers have proposed using attribute filtering for sketch recognition with reliable attributes such as gender and race [8,16,17]. The use of attributes has also been extensively studied for composite sketch generation [21,32]. However, not all the attributes are reliable enough to perform filtering and at the same time, they are not completely incorrect.

    • Construction of Facial Composites from Eyewitness Memory

      2023, Advances in Experimental Medicine and Biology
    View all citing articles on Scopus
    View full text