¶ Scenario 7: The Mirror -- AI Shows Us Who We Are

Forecast: The apocalyptic scenarios -- Terminator, Skynet -- will not materialise. What will materialise is subtler and more disturbing: AI will show us who we are.

¶ No Terminator -- an Amplifier

The fear of "evil AI" is a legacy of Hollywood. The real danger is more prosaic and therefore harder to combat: AI is an amplification of human decisions, human data, human values.

When AI discriminates, it is because the people who trained it discriminated. When AI concentrates power, it is because humans wanted to concentrate power and AI was the tool. The mirror does not lie. But not everyone wants to look into it.

¶ Biases in Training Data

AI systems learn from data produced by humans. These data reflect historical inequalities:

Amazon had to retire an AI recruiting tool in 2018 because it systematically disadvantaged women -- it had learnt from ten years of male-dominated hiring practice [1].
COMPAS, a recidivism prediction system used in the USA, rated Black defendants as systematically more likely to reoffend than white defendants -- with identical histories [2].
Image generators reproduce stereotypes: doctors are white and male, nurses white and female -- because the training data reflect exactly that.

¶ Power Structures in Algorithms

Shoshana Zuboff described in 2019 in The Age of Surveillance Capitalism: the large technology companies extract human behaviour as raw material to produce prediction products that are traded on behavioural markets [3].

Google knows not only what you search for. Google knows what you will do next. Facebook knows not only what you like. Facebook knows how you feel before you know it yourself. And with AI, this predictive power becomes exponentially more potent.

¶ The Most Subtle Disempowerment

Aldous Huxley's Brave New World (1932) is the more precise prophecy than Orwell's 1984 [4]. Huxley's dystopia works not through repression but through satisfaction:

Orwell feared that books would be banned. Huxley feared that nobody would want to read a book any more.
Orwell feared the denial of truth. Huxley feared that truth would be drowned in a sea of irrelevance.

Ninety years after its publication, Brave New World reads like a manual for what is to come.

China has built the prototype. The social credit system, piloted since 2014, rates the behaviour of 1.4 billion people. By 2023, over 26 million flight tickets and 6 million train tickets had been denied on the basis of low scores [5]. The system works not despite but because of its invisibility: there is no official at the door. There is an algorithm that quietly decides.

¶ What This Means for Switzerland

The decisive question for 2050 is not: "What does AI do to us?" The question is: "What do we do with the knowledge that AI gives us about ourselves?"

Switzerland has taken a first step with the revised Data Protection Act (revDPA, in force since September 2023). But data protection alone is not enough. What is needed is algorithmic transparency: all AI systems that prioritise, rank or filter content in the public information space must disclose their criteria.

Computer science was long understood as a technical subject. It is a profoundly human one. AI is the most honest mirror humanity has ever built. The question is whether we have the courage to look into it -- and to change what we see.

¶ Bibliography

[1] Reuters: Amazon scraps secret AI recruiting tool that showed bias against women. 10 October 2018.

[2] Angwin, Julia et al.: Machine Bias: There's software used across the country to predict future criminals. And it's biased against blacks. ProPublica, 23 May 2016.

[3] Zuboff, Shoshana: The Age of Surveillance Capitalism. PublicAffairs, 2019.

[4] Huxley, Aldous: Brave New World. Chatto & Windus, 1932.

[5] National Public Credit Information Center (China): Credit Report 2023. Beijing, 2024.