Algorithmic profiling of the unemployed: A case study and a framework for understanding legitimization processes

Machine learning-based algorithms (MLAs) are often criticized for being biased or unfair when used by public institutions to make predictions or decisions about people's lives, particularly when marginalized or disadvantaged groups are involved. In this context, an important topic is the role of public discourses on the legitimacy of these tools, as these can significantly impact such projects. However, research on this topic is sparse. To add to this knowledge, the present study conducts a longitudinal case study of a public Danish project involving the deployment of an MLA system for profiling unemployed citizens with respect to their risk of long-term unemployment. The system was used for around seven years, but after significant public criticism, it was eventually terminated. Through a grounded theory-based document analysis of the public discourse on the MLA system, a set of legitimization factors and themes is identified. Using these elements, it is shown that public criticism and justification do not only appeal to different principles of legitimacy but may concern different aspects of phenomena. Furthermore, the study demonstrates that legitimization aspects should not be considered in isolation, as these can be used to develop arguments related to other aspects, some of which are inclined to counter each other.