Are chatbots designed to make us all ‘the same’? Studies such as ‘Artificial HiveMind’ show that different chatbots produce identical or very similar results, regardless of the language used in the prompt or the country of origin of the prompt. What impact do chatbots have on diversity, innovation, and minority opinions? Does this modern phenomenon evoke the biblical theme of the confusion of languages at the Tower of Babel?
In the latest of her Duet interviews, Dr Caldarola, editor of Data Warehouse as well as co-author of Big Data and Law, and Prof. Dr Simanowski discuss the social impact of chatbots.
To kick off our Duet interviews, we should first explain in more detail for our readers what large language models (LLMs) are, what they can do, how they are used in the interaction between humans and machines, and how and on whose behalf, they are or will be programmed.
Prof. Dr Simanowski: In my book, I use the term ‘Sprachmaschine’ (language machine). Language machines are chatbots such as Gemini, Claude, ChatGPT, LLaMA, DeepSeek, etc. In English, they are called Large Language Models (LLMs). These are machines into which we humans enter prompts such as a question or task. These machines then provide us with an answer or result. They are based on foundation models or large models to which we humans have no access. We humans interact with the chatbots.
These machines standardise languages. Certain aspects of cognitive, non-physical activity are translated into a specific technique which is a specific method for processing human prompts. This technique is the method of how something is done, depending on the data used to feed and train the chatbots. The data consists of words or letters drawn from books, online texts…in other words, communications of all kinds amounting to trillions of words. From this, the chatbots learn how words statistically relate to one another i.e., how often they are used and with what probability they are associated with other words. When producing ‘its own’ texts, the chatbot then knows which word or combination of words is highly likely to follow the next word or combination of words. This means that the words and word combinations that predominate in these texts reflect and determine the way most people speak and write.
Chatbots are used for all kinds of cognitive tasks, information retrieval, and political and moral questions. They are particularly popular as therapists rather than as encyclopaedias. Even moral questions, such as ‘Marriage for all?’ or ‘When should I propose marriage?’ these questions are then answered statistically based on the chatbot’s programmed algorithms. If the chatbot then responds that I should propose marriage when both parties share those feelings and that I shouldn’t let myself be pressured into it, this answer reflects a clear Western perspective rooted in the West’s concept of romantic love. This answer does not reflect arranged marriages by parents, as are common in India, for example. The chatbot’s response is always an export of values. Mostly these are Western values, because the training data of LLMs and their finetuning by programmers predominantly represent Western values.
What impact might the statistical results of the chatbot have? What does this technology mean for people? What opportunities and what risks do they present?
Media such as chatbots, bring both opportunities and risks. Media scholar Marshall McLuhan said that ‘media are both an extension and an amputation of man.’ Of course, chatbots help us humans with research, summarising, and writing text. At the same time, we humans also lose certain abilities when we no longer use or practise them. That’s just how our brains work. As for risks, they lie in mainstream culture. You only get what the majority says or writes. As a result, innovation falls by the wayside because the prompt always delivers a mainstream answer.
Human innovation stems from the fact that people are constantly exposed to different ideas, ways of thinking, and perspectives. Innovation means ‘thinking outside the box.’ And that is, by its very nature, impossible for a chatbot, because the chatbot speaks with a single voice.
People no longer have a personal connection to the texts. With chatbots, we are essentially dealing with a ‘reluctant librarian’ who denies people direct access to the books. The journey to the book is omitted where people often encounter other interesting books and realise just how complex and rich the topic, the question, the thesis is in terms of its various aspects. The reluctant and at the same time overly assisting librarian tells the prompter that she has read all the books and can give him or her the answer directly. That is exactly what the chatbot does. And because this is, of course, extremely convenient, time-saving, and tempting as a service for the prompter, this service is gladly accepted.
Even if the chatbot’s response is balanced because balance has been programmed into it, it still represents only a single voice. Multiple voices would only exist if the various chatbots were fed and trained with different data and if we would consult different chatbots. However, a study from mid-October 2025, ‘Artificial HiveMind,’ notes that 70 chatbots effectively arrive at the same result when asked the same question.
In this context, how important is it to understand what it means to be in the minority when the statistical majority calls the shots? Who will correct the chatbot’s incorrect speech, in which the statistics lead it to produce?
Politically correct speech may be a minority viewpoint. It is usually not the majority view. In this context, people often refer to the ‘perpetuated bias’ of AI or chatbots. When the prompt operator enters the prompt into the chatbot: ‘Give me an image of a successful person / a successful CEO / a successful doctor,’ the prompt operator receives an image of a white man. These majority stereotypes are then eliminated from the chatbot through so-called system prompts during fine-tuning by the programmer, so that minorities become visible.
There was an incident involving Gemini where, in response to a request for images of historical figures, the Gemini chatbot generated an image of a Black George Washington as well as a female pope of Indigenous descent. It was a scandal. The mistake was that a system prompt had been entered into Gemini instructing the AI to prioritise diversity when generating images of people. This system prompt might have worked when asked for an image of a person with a dog, but it simply doesn’t work with historical figures. That system prompt should have been deactivated or excluded in that case.
One could view chatbots as a moral advancement because, during fine-tuning, they are taught to speak in a politically correct manner, enabling them to use language that is non-discriminatory, inclusive, and so on. We could describe this as progress without being ethnocentric. This is exactly what AI is taught to do, because it often uses gendered or gender-neutral pronouns precisely the way many people do not speak and do not want to speak. This is a continuation of the culture war by technical means in the interest of liberals, not the conservative right. Even if the data at the first training level is certainly conservative, it becomes rather progressive at the second training level through fine-tuning. So much for statistics and minorities.
You write that we should not succumb to AI, that we should remain the subjects of our language, our lives, our existence, and our thinking (interpreting, discussing, philosophising…). What must we be mindful of in order to remain subjects?
We become subjects in the moment of writing or speaking. It is there that we find ourselves, as Hannah Arendt once put it. We must bring together the knowledge we hold within us. In the process of writing or speaking, we notice contradictions and uncover various perspectives that we carry with us. So, in the process of writing or speaking, we discover who we truly are. If we delegate this process of self-discovery to a chatbot and a librarian who blocks access to those various sources with various perspectives, then we miss out on the process of becoming a subject. That is a loss.
Of course, we can use chatbots to make things easier, including in the cognitive realm. When we do so, we need to consider how far we want to take it. There’s no set formula for this.
I believe it is dangerous when people no longer read texts themselves, but instead have them summarised by a chatbot and form and express their views of the world based on that summary. If there is a mainstream trend, chatbots will interpret and summarise texts in a specific, statistically driven, selective, and biased manner. Important aspects of texts that are mentioned only once or rarely (less than 30% frequency) will then be overlooked for statistical reasons. So, if we no longer research, read, and write for ourselves, and instead succumb to our own laziness or inner demons, we come to rely entirely on the output of the chatbot, adopt its perspective, and cease to be active participants.
My favourite quote in this context:
Jenny Holzer
‘Protect me for what I want’,
To guard against the convenience of chatbots, as well as the risks associated with it, we need a design that makes it harder to rely on this convenience. Research and experiments on this topic are already underway. The interface of a chatbot would then have to be built in such a way that we have to take detours when using the LLM. The UX (User Experience) design would thus have to be made uncomfortable, non-intuitive, and therefore inefficient, with a lot of friction. It has been found that we would then prefer to think for ourselves rather than take these detours. Since poor UX design is not in the intrinsic interest of AI companies who actually want to sell user-friendly, pleasant, and in-demand products and services, these cumbersome and time-consuming design detours are unlikely to gain traction in reality, unless the executive and legislative branches enact and enforce corresponding regulations.
Chatbots can process information more quickly than humans in terms of both volume and speed. What impact might this tool have on our thinking?
Chatbots reflect the mainstream. If we stop thinking for ourselves and questioning things critically, and instead let the mainstream become our own conviction, then we become succumb to mainstream thinking. Moreover, this failure to think is a threat to our very existence, because thinking is what defines us as human beings.
Language is the means of communication through which we express ourselves to others, describe phenomena, engage in discourse… but also understand ourselves. The world exists within language, and it is only through language that we have access to the world. Without certain terms, certain phenomena in the world do not exist. So, as chatbots use certain (statistically frequent) terms, this also changes our perspectives on the world, our political views, etc., and limits our perspectives in that regard.
Algorithms do not foster a self-deprecating ‘both-and’ perspective, but rather a dogmatic ‘either-or’ model; they do not encourage reflective thinking, but rather reflexive thinking – technically driven by the dualism of ‘Like’ and ‘Dislike’ buttons and socially dictated by the time pressure under which people respond to communication prompts.
It seems as though humans are creating a new god in the form of AI; one that can process data and information far more effectively and efficiently than humans and therefore knows better than humans what is good for them. Technology that increasingly relieves humans of burdens, even to the point of making decisions about their own existence.
We must approach chatbots with a critical eye, use them responsibly, and understand their implications. That way, we can think critically and remain in control. There are many ways to achieve this. Here, I’d like to give an example.
In Germany, there is Telli, a platform for schools. It brings together various chatbots, allowing students to compare the responses provided by different chatbots. User can enter a question on this platform and receive the responses from two selected chatbots, thus being able to see and analyse the subtle differences in their response. It is precisely with such platforms that critical thinking can be trained.
When a chatbot ‘speaks to us,’ when it addresses us informally and speaks in the first person, and when it even has a humanoid appearance, we humans often succumb to the illusion that we have a scholar, a friend, a companion, a colleague, a buddy, and perhaps even a lover standing before us. In reality, it is a machine without emotion, without affection, and with purely statistical statements. How can we avoid succumbing to this temptation?
I find it hard to understand why so many people succumb to this temptation to the extent that they fall in love with their avatars and even want to marry them. When I interact with the chatbot, I notice that it responds to prompts with interesting insights and perspectives, often with ones I hadn’t thought of myself. But I don’t develop any emotion or affection in the process. Perhaps it’s because I know exactly how chatbots work. Even when I prompt it on the topic of life advice, I know that the chatbot has no life experience and merely summarises statistical data. In my view, it’s important for everyone to understand that a chatbot is a machine that operates on statistics.
Chatbots are used in nursing homes as a therapeutic tool. This works well because the illusion works. It is essentially an extended form of self-talk. This illusion especially when it is beautiful and pleasant may work particularly well in difficult times, when people look for confirmation and consolidation.
But there is another aspect to consider: the impatience people have developed due to digital media, especially social networks. This has led to people being reluctant to engage with one another. The number of close friends with whom they can discuss many aspects of life has also declined. Consequently, a chatbot that listens, engages, and flatters becomes a very appealing alternative to a friend. As a result, people are losing the ability to connect with others who disagree or are just ‘complicated’.
You believe that critical thinking, rather than technical knowledge, is what will prepare us for the social consequences of digitalisation. What is needed to foster critical thinking?
The education system needs educational approaches that, on the one hand, teach media literacy such as, equip people to navigate the information superhighway safely. On the other hand, there is also a need for training in media reflection skills so that people stick with the metaphor. To know what is actually happening on the information superhighways and to be able to question the traffic rules and road signs.
Not enough is being done in this area. The reasons are clear. With the Bologna Process, the education system has taken on a neoliberal and pragmatic form. While computer science instruction has been incorporated into the curriculum, no philosophical or sociological reflection on digital media is offered. The latter does not need to become a separate subject. It would suffice to incorporate media reflection as a module within the various existing subjects. Of course, teachers of these subjects would need to be trained in these expanded modules. This is not currently being done.
Chatbots may change that, because they are fostering a new, heightened awareness of the risks among educators and the general public. But here, too, we’ll have to wait and see how things develop. I am afraid, students will simply learn how to ‘prompt correctly,’ rather than asking themselves what they can do if they feel they want to counter this trend.
We need to prepare citizens for what lies ahead. We need engaged individuals. We need people who understand the societal consequences of programming; in other words, it’s not enough to know what I can do with an algorithm. One must also understand what it does to us. We need the ability to discuss how the interface and the rules of interactions on social networks propel hate speech and fake news, or how influencers and trolls affect citizens’ capacity for democracy and society’s democratic standards.
The key lies in understanding the interplay between the economic, cultural, and technical aspects of digitalisation. Simply educating ‘productive and reliable users of digital technologies’ (Homo economicus) is not enough, as this reflects a purely economic understanding of democracy and consequently contributes to its erosion. We need the Homo politicus, or digital citizen, who understands democracy in participatory and activist terms and who approaches seemingly inevitable technology with a critical eye. Technological progress must not lead to social regression. Technological progress does not automatically solve social problems; rather, it sometimes creates new ones.
Prof. Dr Simanowski, thank you for sharing your insights on the social impact of chatbots.
Thank you and all the best for your upcoming interviews.
