by Devina Sen

Categorization is crucial to shaping our individual experience of the world around us. When our eyes fall on an object we have never seen before, we use prior knowledge of features and old encounters to best categorize and identify the object as something we are familiar with, like concluding that a fluffy creature with four legs that barks is a dog or a long stick with a button on the top than makes ink come out the other side is a pen. Similarly, people must learn to categorize others’ facial expressions and identify them as a specific emotion, a basic component of human empathy. So far, we know that people can recognize emotions on people’s faces in as little as 0.3 to 0.5 seconds and that people’s perception of factors like orientation and numerosity are serially dependent or biased toward visual input seen in the recent past. This begs the question: is our perception of people’s emotions also based on the facial expressions we recognized previously? A study from the Whitney laboratory at UC Berkeley finds that perceived emotional expression is influenced by the faces previously seen only if the current face and a previously seen face share similar physical characteristics.

The overall design of the experiment is simple: have participants identify a series of emotional expressions and determine whether the facial characteristics seen before impact the identification the participant makes. The faces used in the experiments were generated by a computer software called Morph 2.5 that took three images of the same face representing happiness, anger, and sadness, and meshed them together to different degrees. For each trial, the participant was shown a random target expression, then another random expression that they adjusted on the screen to match the target expression. The adjusted face that the participants decided matched the target best was called the match expression. The researchers determined response error as the distance between the target expression and the match expression along a gradient and used the response errors from hundreds of trials in statistical analyses that determined whether factors such as seeing faces with the same expression back to back or faces of people with similar ethnic origin affected the response error. Specifically, the researchers isolated the gender and ethnicity of recently seen faces as possible causes for serial dependence in the participants’ responses as well.

The research concluded that emotional expressions seen up to two faces ago had a pull on the perception of the current emotional expression, giving evidence that indeed people have an innate method of stabilizing emotional expression perception. They found that this pull, or serial dependence effect, was especially selective in the gender of previously seen faces and somewhat selective when previously-seen faces were dissimilar in ethnicity. This seemingly contradictory result reveals that the stabilizing mechanism might not encode specific characteristics but rely on the perceived similarity of faces. For instance, a participant may find two faces of the same gender to be more similar than two faces of different ethnicities which would result in a greater positive perceptual pull in trials featuring a black woman and a white woman than a trial featuring a white woman and a white man, while these results would be flipped for a participant who finds people of the same ethnicity to look more alike.  

But why does this matter? While we may not know how serial dependence effects come about, this finding implies that perception of identity and perception of expression may not be as dissociated as we previously thought. Serial dependence therefore changes the way we might interpret a person’s reaction or state of being based on a feature such as gender and suggests the mechanism for stable emotion perception, whether memory-based or cognition-based, considers previously seen expressions and identities to make expression perception in the future less neural computationally heavy or simply a more familiar, streamlined process. In other words, serial dependence allows us to quickly match emotions to the physical features of a face we have visually processed instead of reinterpreting a facial expression again and again (and again). This ability is especially significant in people’s lives as emotion perception is crucial to guiding our social interactions as well as our own emotional regulation in different situations. What do we do with this information over time? Do we have a lifelong storage of different facial expressions hidden in our brain as a reference throughout our lives or does this serial dependence only play a part for a few seconds at a time? Why are some features more selective for serial dependence while others are not? These are just some of the questions researchers must further investigate as we uncover more and more about the elusive inner workings of our minds.

Source:

Liberman, A., Manassi, M., & Whitney, D. (2018). Serial dependence promotes the stability of perceived emotional expression depending on face similarity(Publication). The Psychonomic Society.

Image Source: https://pixabay.com/photos/smiley-emoticon-anger-angry-2979107/

[Digital image]. (n.d.). Retrieved from https://pixabay.com/photos/smiley-emoticon-anger-angry-2979107/