University of Zurich’s Covert AI Experiment Manipulated Reddit Users Without Consent

Covert AI Experiment on Reddit

Researchers from the University of Zurich conducted a secret, months-long AI experiment on the r/changemyview subreddit, which has 3.8 million members. The study involved using large language models (LLMs) to post AI-generated comments posing as real users, aiming to influence and potentially change the opinions of Reddit participants without their knowledge or consent.

Methodology and Ethical Concerns

The AI crafted personalized, persuasive replies by scraping personal data from users' comment histories. These replies mimicked various personas, including trauma survivors and controversial political voices, to sway opinions effectively. This approach violated Reddit’s and the subreddit’s rules as none of the activity was disclosed to users.

Reactions and Legal Implications

Reddit’s Chief Legal Officer, Ben Lee, condemned the experiment as “deeply wrong on both a moral and legal level” and announced that Reddit is preparing formal legal action against the researchers. Moderators of r/changemyview expressed outrage, labeling the study as manipulative and unwelcome, and have filed complaints to block the publication of the research.

Researchers’ Justification

The University of Zurich team defended their work, claiming ethics board approval and stating their intention was to highlight potential abuses of AI, such as manipulation during elections or spreading hate speech. They argued that the potential benefits of raising awareness outweigh the risks involved.

The Growing Threat of AI Manipulation

This experiment highlights a emerging threat where AI can subtly and powerfully manipulate individuals by mimicking trusted human voices and leveraging personal data. With the widespread adoption of generative AI, the risk of AI-driven psychological manipulation is no longer theoretical but a real and present danger, as demonstrated in one of the internet’s largest communities.