Dark Arts

Steven Feldstein is a senior fellow at the Carnegie Endowment for International Peace and the author of The Rise of Digital Repression: How Technology Is Reshaping Politics, Power, and Resistance. As Feldstein explains, repression-enabling AI applications have become key elements of the authoritarian repertoire globally. Autocrats have invested heavily in them because, although they’ve insulated their power and rolled back democratic movements in recent years, they still understand that the biggest enduring threat to that power in the contemporary world is their own people, either rising up in revolutions or voting them out in elections. And the biggest emerging opportunity to control people is by connecting the digital environments they increasingly live in to state surveillance systems powered by AI.

J.J. Gould: How have autocratic authorities been using artificial intelligence?

Steven Feldstein: There was a moment in the early 2010s when new digital-information and -communication platforms—social-media applications especially—had started to play this remarkable role in helping civilians around the world mobilize and challenge the autocracies they were living under. We saw this in the color revolutions in post-Soviet Eurasia through the Arab Spring. And it led to a lot of optimism that these liberation technologies—as they were called—would help propel a new wave of democratic revolutions globally.

What’s actually happened, alas, is that autocratic governments have figured out how to use new digital technologies—AI applications especially—to repress their citizens more effectively, undercut emerging liberation movements, and reinforce autocratic political power. We’ve seen this above all in China—but also in Russia, the Gulf states, and other authoritarian and illiberal regimes. The range of applications has been expanding, but a few use cases stand out.

One is tracking popular discontent and, when it comes to it, controlling mass protest. That can work in a number of different ways. It can work on a mass scale through automated social-media monitoring, interpreting what people are thinking from what they’re saying online. It can work through public-surveillance cameras and other ways of seeing when and where people are gathering—and then preempting political demonstrations or arresting people who participate in them. We’ve seen this increasingly in Russia, for example, in these techniques Moscow uses to pick up and neutralize anti-war protesters.

A third use case is super-enhancing propaganda and disinformation. Where an autocratic regime has a constitutional obligation to hold formal elections, for instance, and where it might have conventionally used methods of rigging those elections like ballot-stuffing or voter suppression, it’s now more and more likely to augment those methods with AI technology. It can now collaborate with bot and troll armies to spread approved messaging. It can identify and engage key social-media influencers. It can leverage social-media platforms to push out automatic, hyper-personalized disinformation campaigns. And it can use deep-fake technology to generate ever-more realistic audio and video forgeries to discredit challengers.

Gould: Why the demand for repression-enabling AI technology? Haven’t traditional autocratic means of repression worked just fine in preventing a new wave of democracy globally?

Feldstein: From the end of World War II through the late 1980s, the most common way for an autocrat to lose power was a coup—which is to say, by being forced out by elite competitors internal to the regime. After the Cold War, the threat started to shift toward popular challengers external to the regime—namely, toward mass revolts or electoral defeats. The implication for autocrats has been a need to focus on controlling those challengers—by repressing popular civic movements and manipulating elections.

There’s a very direct logic aligning this perceived need with AI technologies. The application of these technologies for political repression can be expensive to develop—but in the end, not as expensive, in terms of either resource costs or political risks, as relying wholly on the manual work, as it were, of old-fashioned security forces. So in the context of the shifting threat to autocrats, AI technologies have improved both the effectiveness and efficiency of political repression.

“Now you have governments all over the world wanting to buy the technologies that enable this monitoring and management, and you have companies all over the world manufacturing and selling them. Autocrats from the Gulf, like Qatar, or from the Middle East, like Egypt, or from South Asia, like Pakistan, are all to some extent importing Chinese technologies. But they’re also importing technologies from companies in other countries, many of which are democratic. The world has become a lot more complicated in that way.”

“There’s understandably a lot of concern in democratic societies about the kinds of social control the large language models that power AI will potentially enable—both at an industrial scale, in spreading bad information, and in ways that are remarkably customized for persuasion at the individual level too. There’s also a lot of concern about ways large language models are starting to power surveillance techniques in criminal detection and law enforcement—with the use of these techniques already, in some cases, racing ahead without regard to any regulations that lay out standards and norms for what’s private or secure and what isn’t.”

“You can, to some extent, deny the most advanced technologies—which are the hardest to acquire—to those who’d accrue the most damage with them against their citizens. As an example, in the U.S., an executive order on spyware recently came out … that would deny any kind of market ability within the U.S. to companies selling these tools to regimes with bad human-rights records. That creates a big incentive to stop doing it. Does this mean those countries won’t be able to attain the tools at all? Probably not. They can probably get something like them from someone else. But it makes it harder, it makes it more expensive, and it probably thwarts these countries from being able to use the most top-of-the-line capabilities they’d otherwise want to use.”