Societal forces around AI

#9.1Introduction

In this chapter, we discuss a number of societal aspects of generative AI. These include aspects related to the creation of language models, the way AI can polarize or instead create connections, and the power of companies that develop and offer AI techniques.

#9.2Data sources and copyright issues surrounding LLMs

To build language models, large amounts of data are needed, but it is often unclear how the companies that make LLMs obtain that data. Previously, companies were fairly transparent about the data sources they used to train their models. A large portion of the Dutch texts used to train AI models, such as ChatGPT, for example, comes from the so-called Common Crawl database (mC4 dataset), which functions as a representation of the internet. De Groene Amsterdammer and Data School (Hofman & Veerbeek, 2023) analyzed these sources.

This analysis revealed three striking categories:

• The largest source of information in the dataset is the DocPlayer website. This website was set up by a Russian entrepreneur and automatically collects documents that appear online. The documents are often reused without permission, which makes this source legally and ethically problematic. In addition, the site also contains a great deal of personal and sensitive data. For example, tax returns and job application letters have been found there. That makes the use of this source particularly controversial.
• Other important sources are articles from newspapers and journalistic magazines. These are journalistically high-quality sources, but the publishers never gave permission for this use. The developers of LLMs make free use of data in which publishers have made substantial investments of time, money, and journalistic labor. Publishers are therefore exploring legal avenues to change this situation.
• Many of the sources on which LLMs are trained are in fact low quality, with their reliability seriously in question. For example, the reviews people leave in webshops. In addition, the Nazi conspiracy website Stormfront also turns out to be one of the sources used, and in terms of size in the C4 source list it ranks only one place below RTL Nieuws.

Tech companies have recently become very secretive about the sources they use because the legal basis for doing so is under debate. Writers, for example, are filing lawsuits against the major generative AI companies because they want to be compensated at the very least, if the reuse of their work is not already illegal. Lawsuits about this, for example in the United States, are still being assessed differently by different judges, so there is still no clear answer to this problem (Visser, 2025).

Still, the generative AI companies continue collecting data to improve models, because they are of course competing with each other. That is why they also continue to use controversial data sources such as the illegal book sites Anna’s Archive and LibGen (Pontefract, 2025). Several lawsuits are pending over this against the major language model developers (Spinner, 2025). But data from social media platforms is also being used more and more. For example, Elon Musk is going to use data from X (formerly Twitter) to develop his models (POLITICO, 2025) and Meta chief Mark Zuckerberg wants to use data from WhatsApp, Facebook, and Instagram for this (Heikkiläarchive, 2024). But companies are also increasingly switching to sources that are legally less problematic. OpenAI, for example, is entering into agreements with publishers to use their data (OpenAI, 2023). And Adobe uses only its own stock material for its generative AI, Firefly.

#9.3Humans training AI models

Although generative AI seems above all to be a technical development, people still play an important role in it. To train the models as we described in chapter 5, a great deal of human labor is needed, especially by so-called ‘data labelers.’ These are people with specific expertise in, for example, language or a particular subject area, who are deployed during the supervised and reinforcement learning phase: they assess texts or images and label them.

According to estimates, millions of people worldwide are active in this work. In itself that is not problematic, but journalists such as Martijn (2023) and Van Bergeijk (2025) do express serious concerns about the supervision and pay of these workers. According to them, many data labelers are underpaid and exposed for long periods to mind-numbing and psychologically taxing work, such as assessing large quantities of violent texts and images.

Against that, according to the journalists, there is little for them: no mental health support, no permanent employment, and no appropriate financial compensation. Most data labelers work through an arrangement in which data labeling companies hire them by the hour and by the piece at a low rate. Compensation varies from two euros an hour in African countries to thirty euros an hour in countries such as the Netherlands. The work is very irregular. Data labeling companies temporarily hire large numbers of workers when they receive assignments, but once an assignment is completed, there may be no work for a long time. Anyone who, according to a company, works too little or not quickly or accurately enough often loses their contract immediately without explanation. In short, training LLMs relies heavily on human labor, while the working conditions and rights of these workers are often minimal.

From the European perspective, we regard this pay system and these conditions as exploitation. From the democratic and social values of the Netherlands and the EU, workers' rights and well-being are a primary concern. From the perspectives of other political spheres of influence, apparently different values often apply.

This therefore creates an enormous dilemma for European companies that want to develop language models. European companies are expected to want to train models on the basis of working conditions that are responsible. But that naturally makes training models more expensive and creates a competitive disadvantage relative to countries or companies that do not take this as seriously. The question is how that gap should be bridged.

#9.4Algorithms, AI, social media, discrimination, filter bubbles, extremism

Especially as a citizen, you’ll encounter AI. For example, in the way you’re informed by social media, how government agencies make decisions, or how your privacy is protected. AI thus affects not only your future career, but also your social standing, your rights, and your opportunities. It therefore calls for reflection on multiple levels: as a student, a professional, and a citizen.

One of the most important aspects of this is how algorithms, in combination with AI, influence the information you are shown and the people you come to know. And that, in turn, shapes your opinions and your view of the world. And this isn’t just about relatively harmless information or people who are interested in a particular music genre, video games, or stamp collecting. No, it goes much further because it affects deeply held norms and values such as happiness, well-being, self-image, fear, empathy, equality, friends or foes and democratic values. It is essential to understand how technology companies create algorithms that can maneuver you into an increasingly narrow and extreme straitjacket of information and thought framework. And that can lead to increasingly extreme societal viewpoints by normalizing these extreme viewpoints.

#9.5.1How algorithms hold your attention

Social media platforms are designed to keep you scrolling, watching, and liking for as long as possible. More time on the platform means more opportunities to show ads, which is the main source of revenue for most platforms. The algorithms used by online stores or social media platforms are designed to analyze your behavior and keep showing you content that is increasingly appealing to you. Even without AI, platforms can estimate user preferences, but AI makes this process much more effective and refined. Content that fits your preferences well gets priority in this. But sensational content also turns out to hold attention well, which means that what you are shown can sometimes be extreme. Think of violent content, extreme opinions, or dangerous challenges.

In general, the algorithms work on the basis of collected data and the patterns in it. They analyze your engagement as a user and filter messages. The objective behind this is not necessarily aimed at giving you as much pleasure as possible, but above all at holding your attention for as long as possible.

Data collection (input) - everything you do on the platform is data for the algorithm:

• Interactions: which posts do you like, share, or comment on? Which links do you click?
• Watch time: how long do you look at a video or image? Do you scroll past quickly or stop?
• Connections: who are your friends/followers? What do they like and share?
• Profile information: your age, location, and interests you have provided.
• Search behavior: what do you search for within the platform?
• Sometimes also data from outside the platform, via cookies and trackers.
• Sometimes also data from your phone, such as your contacts, your GPS location, or your photo gallery.

Pattern recognition and profiling - the algorithm continuously analyzes all this data to build a detailed profile of you. It learns:

• Which topics you find interesting (politics, sports, music, cat videos, specific hobbies, specific types of people).
• Which opinions or perspectives you are likely to share or appreciate, or which you react negatively to, so that you keep engaging for longer.
• Which accounts (people, news sources, groups) you follow.
• Which type of content (video, text, image) interests you most.

Prediction of engagement (engagement).

Based on your profile, the algorithm predicts for each piece of potential content (a post from a friend, a news article, an ad, a viral video) how likely it is that you will respond to it (like, comment, share, watch for a long time). Content with a high predicted level of engagement gets priority.

Personalization and filtering (the core of the bubble):

• Prioritization: the algorithm fills your feed with the content it thinks you will like best or interact with the most. This is almost always content that matches your previous behavior, personal characteristics, and preferences. But it can also be an opposing or extreme opinion, so that you respond to it.
• Filtering: content that the algorithm predicts you will do little with, for example because you find the topic boring or because it comes from an unknown source, is placed lower or not shown at all.

The steps you read above are applied everywhere online. The data are not collected to test specific hypotheses as in classical statistics; instead, patterns are sought that emerge from the data themselves. In this context, the terms AI and Big Data are often confused with each other.

Algorithms can also be confronting when they predict preferences or personality traits that you are not aware of or would rather keep hidden. For example, young people in the book Filterworld, by the American journalist and culture critic Kyle Chayka (2024) say that the TikTok algorithm often realized earlier than they themselves did that they might be queer. Some users even wonder whether their preferences in terms of hobbies or interests are really their own, or whether they are partly shaped by the algorithms that constantly steer them. The New Yorker (2022) calls this phenomenon ‘algorithmic anxiety’: the uncomfortable feeling that algorithms influence your identity.

Still, algorithms also regularly get it wrong. When the system has little information about you or you turn off personalized ads, it fills the gap with general assumptions. For example, based on age, gender identity, or location. As a result, as a woman in your thirties you may get ads for baby products, even if you have no interest in them. Or, as a man, you may mainly see car or sports ads, purely because of stereotypes. Such predictive strategies can be not only irritating, but also limiting or alienating.

#9.5.2Echo chambers and filter bubbles

Because almost all platforms use algorithms to keep users active for as long as possible, users ultimately end up being automatically clustered into groups that are very similar. As a result, you end up in so-called echo chambers or filter bubbles.

This process works through a reinforcement loop that keeps growing stronger. If you mainly respond to or look at content and people that confirm your worldview, the algorithm learns that you value this kind of content. It will then show you even more of it. As a result, your existing beliefs are constantly reinforced and repeated. You become trapped in what experts call an echo chamber.

At the same time, you are systematically exposed less and less to diversity or other perspectives. Because the algorithm filters out content that is unlikely to immediately get your like or share, you see fewer and fewer dissenting opinions, other perspectives, or information that challenges your existing ideas. You thus end up in a filter bubble, a personalized information universe that unconsciously shields you from other viewpoints.

The algorithm also reinforces a natural human tendency. People naturally like to connect with like-minded others, which in itself does not have to be harmful. For example, if in your offline life you cannot find fellow sufferers of an illness you have, no roommate who wants to talk endlessly about tectonic plates, or no one who shares your love of an obscure Swedish soccer club, you can find them online. The system cleverly plays into this. On the one hand, it shows you content more often from the friends and connections with whom you interact most, and on the other hand it shows you strangers who are very similar to you. Those are usually exactly the people who already share your opinion. This creates a spiral in which your digital world increasingly resembles a confirmation of what you already thought and felt.

#9.5.3Extremism and polarization

All these mechanisms are not without dangers. Of course, when it comes to exchanging experiences about gardening with like-minded people, there is not much going on. But from a political and societal perspective, it can also lead to polarization, extremism, and radicalization. This is rooted in a combination of psychological and social factors (Barberá, 2020; Lim & Bentley, 2022).

It often begins with what is known as confirmation bias: your natural tendency to seek out information that confirms your existing beliefs. That was also the case with that roommate who no longer wants to use sunscreen. They first saw a single TikTok about it, kept watching it for a long time, responded to comments, and became convinced. They then saw multiple TikToks with the same message and think: see, sunscreen really is bad for you. In an echo chamber, you are presented with almost nothing but this kind of confirming information. Every like you give, every post you share, every article you read, seems to prove you right. As a result, your conviction feels stronger and more certain all the time.

At the same time, constantly seeing and hearing the same opinions makes these ideas seem increasingly normal and credible. This is also known as the ‘illusory truth effect’: through repetition, what may at first have been a questionable thought becomes an accepted truth for you. Because dissenting opinions or critical voices are systematically filtered out or ignored, the dominant ideas within the echo chamber are never really challenged. You feel no need to refine your own arguments or take nuances or complexity into account. This leads to simplification and hardening of positions.

A particularly powerful effect is group polarization. When you and like-minded people discuss things with one another, you tend afterward to adopt a more extreme version of your original position. This happens because you hear new arguments that support your position, because you want to conform to what you see as the group norm (which is often slightly more extreme than the individual average), and because you want to distinguish yourself as a good group member by expressing a somewhat stronger opinion.

Shared beliefs also create a sense of solidarity and social validation. Your group identity becomes strongly tied to these beliefs, making it harder to change your mind without feeling that you are betraying the group and a part of yourself. To strengthen their own group cohesion, people or groups with dissenting views are often portrayed negatively, simplistically, or even as hostile. This us-versus-them thinking reduces empathy, sometimes dehumanizes the other person, and makes it easier to take more extreme positions toward them.

But it is not only the users themselves who come up with increasingly extreme positions. It is precisely the algorithms designed by the platforms that reinforce this. Platforms such as YouTube or TikTok can lead users toward increasingly extreme content through their recommendation algorithms. This process is called algorithmic radicalization (O’Callaghan et al., 2015), in which algorithms steer users toward ever more radical content to retain engagement. This does not necessarily happen with malicious intent, but as a byproduct of algorithms designed to maximize user engagement.

Although the mechanisms described above do not coincide one-to-one with AI, and certainly not with generative AI, it is still very important to be able to recognize them in your professional and personal life.

We conclude this chapter with the following. You should be aware that the way the personalization and filtering step (the core of the bubble) is determined and designed lies in the hands of companies, and therefore of people. Those people are the owners and managers of companies that provide the software and make money from it. It is their deliberate choice to design these algorithms in such a way that, as an end user, you keep coming back as much as possible, so that they can then show you advertising tailored to you. In effect, they decide to a significant extent how you think and behave. These companies therefore have a very great responsibility. Especially in a time of major polarization, they should take that into account in their design. Technically, in any case, it is unquestionably easy to do. So ask yourself: are these companies taking their responsibility? Will the rules of the AI Act help (see section 9.5)?

#9.5.4Algorithms, AI, and discrimination - concepts are often confused

By now you already know quite a lot about bias and possible discrimination in light of the development and application of generative AI. But another important aspect of algorithms is discrimination. We use the scholarship fraud scandal at the Dutch Education Executive Agency (DUO) as an example (NOS, 2023).

At DUO, students were checked between 2012 and 2023 using an algorithm with self-devised risk criteria such as age and education level. Although there was no direct discrimination (directly selecting on migration background), there was indirect discrimination through selection based on characteristics such as age, type of education (mbo received a higher risk score than hbo or university) and distance from parent(s), which led to more students with a non-European migration background being selected for review than other students.

The main problem is the disproportionality in who was checked in advance on the basis of ‘non-objective justification’ for the risk criteria. The criteria were devised on the basis of ‘impressions.’ Even if some of the checked students had in fact committed fraud, it is discriminatory if certain groups are checked disproportionately often on the basis of wrongly assumed risks. That is contrary to principles of equality and anti-discrimination legislation. The second problem is that - if fraud were established within the selected group - a self-reinforcing effect would occur: it would then be established that a disproportionately large number of students with a migration background had committed fraud within DUO’s total file. In that way, their self-devised criteria would be validated and a vicious circle would arise.

The interesting thing about this case is that in the many articles written about it, the words ‘discrimination,’ ‘AI,’ and ‘algorithms’ were mentioned in the same breath. That could lead you to think that they all mean the same thing. Whereas in DUO’s case, discrimination had nothing to do with AI or generative AI.

#9.6Privacy and data protection - understanding the laws

The European Union has created two important legal frameworks with the AI Act and the General Data Protection Regulation (GDPR) to protect you as a student and citizen. The GDPR contains strict rules on the collection and processing of your personal data, with transparency, purpose limitation, and data minimization at its core. The AI Act adds specific rules for AI systems, divided into risk categories with corresponding obligations. High-risk applications must meet strict requirements in the areas of transparency, human oversight, and accountability. Certain AI practices that threaten your fundamental rights are completely prohibited, such as facial recognition in public spaces. The institution where you study or the company where you work is responsible for handling this lawfully, so that you can work with systems that meet the requirements.

Despite this legal protection, as a user you also have your own responsibility in protecting your privacy and that of others. It naturally begins with securing your own computer, using good passwords, and encrypting your data where possible. When it comes to generative AI, you start by critically assessing which AI services you use and which data you share with them. For example, it is very easy to create a free account with a generative AI service, but do you know what happens to your data then? For instance, you can regularly check permissions and use alternatives that collect less of your data where possible. Be aware of the value of your personal data and remain critical of ‘free’ services that are in fact paid for with your data. Therefore, never share special-category data or sensitive data in those kinds of systems. By making use of both legal protection and your own personal vigilance, you can take advantage of the benefits of AI without sacrificing too much privacy.

#9.7Power of Big Tech

Generative AI is a technology that is causing a fundamental shift in power in the digital world. At the helm of this shift are the major technology companies - often referred to as Big Tech - such as OpenAI (with investments from Microsoft), Google, Amazon, Meta, and Apple. These companies not only have enormous amounts of data and computing power, but also the financial resources, networks, and infrastructure to develop and roll out generative AI on a global scale. The development of large language models such as GPT-4, Gemini, and Claude requires investments of hundreds of millions of dollars, which makes the threshold very high for smaller companies or public institutions. And this concentration of power is accelerating (Verhagen, 2025).

Only a handful of companies determine which social media platforms exist, which algorithms they deploy, and which AI models are used worldwide, what infrastructure they run on, under what conditions, and with what ethical implications. This one-sided concentration of resources leads to a centralization of power. An important aspect of this, for example, is diverting visitors away from websites, especially journalistic websites. Because people are increasingly searching the internet with the help of generative AI (if young people are not already doing so via YouTube, TikTok, or Instagram), people visit less and less the websites from which the large companies get the data. When searching, people stop at the AI’s answer without still visiting the original website, causing those websites to lose a great deal of revenue. According to The Atlantic (Reisner, 2025), this could soon mean that many journalistic websites disappear, even though they are crucial to democracy because they inform citizens, exercise oversight over the government, and provide a platform for public debate.

The European Commission and other policymaking bodies have long warned that this technological dominance carries risks for democratic oversight, transparency, and fair access to information and innovation (Bietti, 2020; Crawford, 2021c). For example, big tech companies often unilaterally determine which data are included in the models, which languages and cultural contexts are dominant, and who gets access to advanced models. This has direct consequences for, for example, education, media, and public services, which are increasingly becoming dependent on AI systems supplied by companies.

The global power dynamics around AI are therefore increasingly being determined by economic and geopolitical interests. While the United States has a lead in private AI innovation, China wants to catch up through state-driven projects. Europe is trying to position itself as the guardian of ethical and transparent AI, but in doing so it remains heavily dependent on foreign technology and platforms. This asymmetry makes it difficult for European governments and institutions to make truly sovereign choices about how AI is used in, for example, education, health care, or the justice system (Mazzucato, 2021). Concerns about digital colonization - whereby technologies are not only introduced but also bring with them the values and economic relations of their producers - are therefore increasing (Couldry & Mejias, 2019).

The dominance of big tech in generative AI also raises ethical questions about inclusion, fairness, and the role of the public sector. Over the past year, the owners of Facebook and Instagram (Meta, Mark Zuckerberg), Tesla, X, and Grok (Elon Musk), and others (TikTok, Google), have openly opposed diversity policies. And that translates directly worldwide. Who gets involved in developing these technologies? Which voices remain unheard? And which domains of knowledge are reinforced or instead ignored by the datasets on which the models are trained? These questions touch on the core of digital sovereignty and the right to technology that serves society, not merely shareholders.

To counterbalance the power position of big tech, the call for public investment in open and transparent AI models is growing, for example through universities, libraries, or international collaborations such as the European AI Alliance (European Commission, 2018). Such initiatives can contribute to a more balanced and democratically legitimate AI future, provided they are accompanied by structural funding, clear regulations, and broad social involvement (Brundage et al., 2020; Eubanks, 2018). Initiatives such as Hugging Face, EleutherAI, and BLOOM advocate open access to AI models, datasets, and training procedures. In this way, they try to promote technological independence and inclusion. Yet these projects remain dependent on funding from governments or, once again, big tech itself (Kreps & Kriner, 2023).

#9.8Open-source models versus closed models

Open-source LLMs and closed LLMs differ in their approach to accessibility and use. Open-source models, such as Llama, Falcon, and Mistral, offer transparency because their code, training process, and often their weights are publicly accessible. This makes it possible for researchers and developers to study, adapt, improve, and collaborate on the models. It improves insight into how the systems work. In addition, open-source models can be run locally (on a user's computer), which can make them more privacy-friendly and less dependent on external services.

But open-source models often have more limited resources for training and refinement, which can result in less advanced capabilities. Closed models such as GPT-4/5 and Claude therefore usually offer better performance and reliability. But their closed nature limits scientific progress because researchers do not gain insight into how the models work or make decisions. The choice between open-source and closed LLMs therefore comes down to a trade-off between transparency, control, and innovation through collaboration on the one hand, and performance and professional support on the other.

#9.9.1Review questions

Which three notable categories of data sources were found in the mC4 dataset used to train language models such as ChatGPT?

What is the difference between echo chambers and filter bubbles, and how can algorithms contribute to polarization and extremism?

Name three important differences between open-source LLMs (such as Llama and Mistral) and closed models (such as GPT-5 and Claude).