Online retail images reveal discrepancies in skin tone

When Lee Humphreys ’99 noticed that models at a retail company – known for its diversity – looked lighter in photos than in videos, she didn’t think it was intentional.

She contacted the CEO, who promised to get back to her. He was eventually told that the discrepancy must be due to his own computer.

But she knew that wasn’t true – she had also asked her colleagues to check their computers. The experience sparked an investigation by Humphreyprofessor and chair of the Department of Communication at the College of Agriculture and Life Sciences, and doctoral student Chelsea Butkowski, on the role of skin color representation of models in the online retail environment.

Online retail images of popular fashion brands show a difference in models’ skin tones. The image on the left is the main product photo for the dress pictured; to the right is a still image from the product video of the dress shown.

their study, Colorism Calculation: Skin Tone in Online Retail Imagery», published on March 13 in Visual Communication, found that still images of models had statistically lighter skin tones than in videos of the same product and model. They also found evidence of “symbolism” – that is, many websites had a model that had considerably darker skin than the others, as “a sort of stand-in for a wide range of diversity” , said Butkowski, leader of the study. main author.

To conduct the research, they developed a method for quantitatively measuring skin lightness and darkness to capture inconsistencies in different representations of the same model across platforms. To accurately characterize this difference, they enlisted Utkarsh Mall, Computer Science PhD studentand together they developed a visual analysis procedure.

“We wanted to take what we were seeing and back it up with a more quantifiable method, beyond what we were seeing,” Butkowski said. “We expected to find that there would be a difference, and that it would be statistically significant.”

In August 2019 Product Photos, they sampled the first photo and video of 10 women’s clothing listings on three retailer websites – Banana Republic, Gap and Old Navy, all subsidiaries of Gap Inc.

Their method was to analyze two regions of the images: the chin, which was chosen because it most often faces forward, and because the still images used by one of the retailers are cropped above the nose ; and all visible skin, to compare skin tones in photo and video modes.

Grayscale histograms – characterizations of pixel distribution – were created for all 30 images, to visualize pixel intensity and clustering over the possible tonal range.

What also stands out for Humphreys is the diversity of images and the ambiguity when it comes to ethnicity.

“We took that as generally a good thing,” she said, “that we were seeing more ethnic diversity, to some degree. But when it comes to skin tones, they were still relatively fair.

The difference between skin tones was obvious, but what is less obvious, according to the researchers, is which tones most closely reflect reality.

“We don’t know if the videos were darker but closer to what the models actually look like, or if the lighter photos accurately represent the models,” Humphreys said. “Maybe the videos just aren’t well lit, so they end up being darker than the model actually is.”

“We don’t know what we don’t know,” she said, “but what gets really interesting is that the discrepancy itself becomes problematic and a potential indicator of photo manipulation or technological biases.

Anne G. Cash