What can our eyes and face really tell us about customer experiences?

Eye tracking

The largest part of the human brain involved with sensory perception is the visual cortex at the back of the skull and our eyes have evolved to be overwhelmingly our most developed organ in terms of keeping tabs on the world around us. With advances in camera technology, science is able to monitor both the movement of our eyes and also measure their ‘fixation’, that is how long we dwell on a particular point in our field of vision, through both mobile eye-tracking glasses and devices fixed to monitors (PC and TV) in front of subjects. By analyzing eye movement and fixation and superimposing this data onto a video of what the subject is looking at, we can build a very accurate picture of what is grabbing and holding people’s visual attention. Fixed eye-tracking is ideal for monitoring how people ‘see’ websites and advertising material, as subjects are sat facing the screen. Mobile eye-tracking glasses are useful for monitoring more ‘real-world’ scenarios as people carry out everyday tasks like shopping, although the devices themselves (though increasingly less bulky and more like normal glasses) are still noticeable and may therefore detract from a completely natural experience. An issue with this methodology however is that, on its own, it only measures overt visual attention and gives us no clue as to the ‘why’. We can see what people notice, how quickly and for now long they look at it, but not any of the reasons why that might be or how it relates to other sensory input (sound, temperature, touch, taste, etc) and the impact of this on behaviour. However, by linking eye tracking with other internal physiological measurement – like heart rate, EDA and EEG (the measurement of electrical activity in the brain) – we can understand both what people notice and how it affects them, giving us the evidence to support hypotheses as to why this is happening.

Pupillometry

Monitoring the eye can also allow us to measure the size of the subject’s pupils as they navigate the world. The scientific study of the attributes of our eyes have revealed that our pupils change size in response to both external stimuli (the amount of light) and also as a result of internal mental processes. Thus our pupils dilate in response to increased cognitive processing as well as emotional arousal, giving us an indicator of the intensity of response that may be independent of something we are actually looking at. For example, regardless of what we are noticing visually, our pupils may be responding to other sensory inputs like an unpleasant smell, a change in temperature, pain, etc. However, although pupil response may indicate arousal, it gives us no clue as to the valence (the direction) of that arousal – i.e. whether it is positive or negative. There is also some experimental evidence to show that an increase in pupil size may reflect greater interest in something that we observe. What’s interesting about pupillometry is that pupil dilation does indicate cognitive surprise (Tripping Points® again!), which is very difficult to control voluntarily and so gives us a useful indicator of the impact of such an event on subsequent behaviour. Combining pupil measurement with other physiological methods provides the best opportunity of understanding both the response to an experience and the reasons for that response.

Facial Expression Analysis (FEA)

By capturing the facial expressions of people as they interact with different stimuli – presented in front of them, such as websites, TV screens, events, etc – researchers can, either manually or by using automatic facial coding software, monitor the change in people’s reactions to what they see and experience. Through recording all the facial features of a subject, we may be able to detect the valence of that person’s response, i.e. whether that response is positive or negative. FEA relies on a universal set of facial expressions for all humans (which is indicative of the emotion that person is experiencing) so that a response can be successfully coded and also, for accurate measurement, that the subject is clear, well-lit and directly facing the camera. There is some evidence that FEA correlates with the self-reported response of people, but would appear better at identifying negative responses to stimuli.

Within the field of FEA, there is also a more objective measure of human response using electromyography (EMG). This involves the use of extremely sensitive devices, affixed to the skin, which can monitor minute changes in facial muscles and offers researchers the opportunity to understand how stimuli can affect subjects emotionally. By measuring the contraction of two specific muscle groups – at the eyebows (to monitor frowning) and on the cheek (to monitor smiling), scientists are able to record the implicit affect of stimuli – both positive or negative – on subjects, even when no overt change is facial expression can be detected and people are not consciously aware of their reaction. However, because of the nature of the measurement, attaching electrodes to the skin, facial EMG cannot be done covertly and can only really occur in a lab situation.

Again, both FEA and EMG are best used in combination with other physiological methods to more accurately correlate the ‘what’ with the ‘why’ of a particular event or experience. It is also worth noting that some in neuroscience have recently cast doubt on the efficacy of a universal set of facial expressions that indicate specific emotions. This is despite the 40-year history of research into this area and a number of software manufacturers offering facial coding as a way of ‘measuring’ emotion. In a future post we will return to this BIG question of whether what we are feeling really is ‘written all over our face’ and the controversy over where science has got to in understanding and measuring emotion.

In our next post, we’ll look at some of the neuroscientific methods for investigating human behaviour involving scanning the brain to measure electrical activity and blood flow as indicators of mental activity.

CX Lab help uncover the unconscious and conscious behaviour of your customers and design and implement customer and employee experience strategies that deliver a significant ROI. Please get in touch