Image for post
Image for post

Against Face Analysis

The use of mugshots in research, and the shift from face recognition to face attribute classification.

In 2018 I read a research paper about estimating body mass index from face photos, written by an insurance company. The paper had a diagram, a mosaic of photos of mostly Black men showing that the training data was built on mugshots taken by the Florida department of corrections. I’ve been making art about face analysis since 2009, and I was generally familiar with the long history connecting it to policing, and that it is deployed in a way that reinforces systemic racism and mass incarceration. I knew mugshots were easily accessible. But I didn’t know that mugshots were used to build these systems, and this seemed like another layer of abuse. I decided to dig deeper, with support from the Eyebeam Rapid Response fellowship.

A grid of blurred mugshots showing five examples of of five body mass index categories from underweight to severely obese.
A grid of blurred mugshots showing five examples of of five body mass index categories from underweight to severely obese.
Marketing diagram showing a progression from “human in the loop” to “human on the loop” to “human out of the loop”.
Marketing diagram showing a progression from “human in the loop” to “human on the loop” to “human out of the loop”.
Source: Veritone
Mostly blurred spreadsheet, with a list of states and totals on left side, ranging from thousands to hundreds of thousands.
Mostly blurred spreadsheet, with a list of states and totals on left side, ranging from thousands to hundreds of thousands.
Screenshot of research into how mugshots are stored and accessed.
Image for post
Image for post
Diagram of training process for a face recognition algorithm, showing data flowing through layers with annotated equations.
Diagram of training process for a face recognition algorithm, showing data flowing through layers with annotated equations.
Diagram of face recognition architecture from insightface

Artist working with code.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store