You cannot make the algorithm do something it does not want to do: exploring the agency and power of AI systems in practice

Will AI change society as we know it? Popular accounts paint a picture of AI as inherently powerful, transformative, and inscrutable – sustaining a rather mythical and seductive story of AI. To challenge such panoramas, we offer a situated case study of an AI system in the making where we through several ‘ethnographic episodes’ (re)explore the agency and power of AI. The case study is based on nearly two years of ethnographic enquiry into the development process of a machine learning-based recommender system in a large regional media organisation in Denmark. STS scholars have already begun to demystify the workings of AI, highlighting the fragility of such systems by showing how they depend on heterogeneous others to act in the first place. However, such decentring moves come with the risk of turning away from the AI’s role as an actor. We place emphasis on this, as we describe how data scientists, managers, and editors attempt to negotiate and collaborate with the AI system and how it turns out to be a rather difficult collaborator: An unwilling, stubborn and sometimes even incompetent actor. Due to its inability or unwillingness to negotiate, the AI system induces the need for its surroundings to adapt to it. It seems that the transformative power of AI is not necessarily a result of these systems being inherently powerful or highly advanced. It can also be their quite clumsy and simple-minded nature that provokes a range of transformations in the contexts they enter. Approaching AI systems as awkward and stubborn actors, we find, offers a more interesting way of thinking about their influence on society than the story of AI as inherently advanced and powerful.