You could easily mistake this particular Twitter user as a European Union citizen with a wide range of interests. He posts up to 60 times a day, in Dutch, German, English and Spanish. And he comments on rulings from the European Court of Human Rights as well as the Crimea crisis and the issue of Catalan. But there's still something a bit fishy about his account. He's constantly seeking to sow discord. He is sometimes belligerent, frequently twists facts and also targets Muslims and the EU while supporting Brexit and Trump. "The EU court is the new Nazi court," he recently wrote.
Even stranger than all this is the fact that, over the last four weeks, more than 800 accounts temporarily shared the same name as this single Twitter user – all very briefly. What was going on?
This game of tag jibes neatly with the strategies of Russian trolls. The supposed EU citizen appears to be a kind of virtual name tag that is simply passed along on the internet. Ultimately, though, the trail leads to St. Petersburg, to a troll factory known as the Internet Research Agency (IRA). A few years ago, a number of accounts held by the IRA were exposed and blocked – and this user was linked conspicuously closely to some of them.
Nobody knows how many trolls there are or how many people they are able to deceive on the internet. But there's no denying their existence. And the upcoming European Parliament elections, to be held later this month, are on their radar.
For countries like Russia, the internet is like a gift that keeps on giving: It enables opinions and elections to be manipulated in other countries without having to fire a single shot, spend a lot of money or unnecessarily endanger any of your own people. It makes it possible to undermine and destabilize democracies from the comfort of a desk.
The Kremlin recognized that potential early on. Specialists with the Russian secret service agencies GRU and SWR even form policies on the basis of the data they capture. And then there are the trolls on platforms like Facebook and Twitter, controlled by opinion factories like the IRA in St. Petersburg. They foment conflict and fuel campaigns in Germany, Great Britain and the United States. The influence Moscow had on Donald Trump's election to the U.S. presidency in fall 2016 underscored just how successful these efforts can be.
In the grand game of geopolitics, the Russian government exploits this gray area between war and peace as an elegant way to expand its influence. But how did it all get started – and where? It is a search that must also look at the software the trolls have at their disposal. And on that search, the journey starts in a rather unexpected place.
The Perfect Curve
In the Spanish capital of Madrid, Javier Perez Dolset, a tall, soft-spoken 49-year-old, is sitting at a conference table and smiling as he recalls those two bars that could be moved up and down with a controller to hit a pixel ball back and forth. He was 6 years old, he says, when his father gave him a Pong console – and he's been obsessed with video games and programming ever since. "I was the biggest nerd in school," he says.
In 1998, when he was in his late 20s, he developed the successful video game "Commandos." Today, his empire includes the private U-tad technical university on the outskirts of Madrid, with a modern campus, complete with roads that are named after European cities, where students learn to program. His animation studio, which produces films for U.S. studios, is also located on campus. His passion for all things digital has been a constant throughout his life, even though it has also got Perez Dolset into trouble, including legal disputes and the loss of control over his most important company. It has also led to his entanglement in the digital conflict zone of troll wars.
The story begins around 2010, a time when Perez Dolset's company ZED had already been making a mint for years with mobile communications products like ring tones, music and games. But as the importance of social networks grew, Perez Dolset quickly recognized what it could mean for his clients – mobile operators who in turn had hundreds of thousands of customers of their own – if dissatisfied consumers were to join forces there to complain about products or services.
Perez Dolset decided to develop software that could serve as an early warning system to detect such brewing storms and neutralize the bad PR. He pumped millions of euros into the project – including funds from government loans, leading to subsequent claims that Perez Dolset misused those funds. By 2013, the project resulted in a program called the Social Networks Analysis Platform, or SNAP.
A fake attention curve that can outwit Twitter
SNAP is made up of three components. Perez Dolset and Juan Carlos Gonzalvez, the program's architect, explained how SNAP works over the course of several meetings in Madrid. ZEIT and its partners were also able to view elements of the source code.
The first component is a data rake that collects as much of the current data stream on Twitter as possible. The second component analyzes the data and converts it into graphics that help show such information as: What discussions are starting to form around what hashtags? Who are the most important participants, based on how much they are writing, how many followers they have and how often their posts are shared? SNAP can visualize links that might otherwise remain hidden. Who's connected to whom, for example. Or when very different hashtags attract the interest of a surprisingly large mix of users. Depending on the question asked, points and connecting lines are arranged on the monitor and form complex webs of relationships. But without the third component, the attack tool, it would all be little more than a gimmick. Perez Dolset named that tool the Social Baton because, just as a conductor conducts an orchestra using a baton, the customer should also be able to steer Twitter accounts using SNAP. "Orchestrated campaigns through tweets and retweets," an advertising brochure promises.
Twitter is a so-called micro-blogging platform – a real-world analogy might be a crowded pedestrian zone with everyone babbling their thoughts to themselves. At times, though, some pedestrians pick up what others are saying, and suddenly you have a choir of people shouting things like, "Company X makes bad products." The choir grows into a cacophony, and the hashtag becomes a trending topic, and the whole world begins talking about the content in question.
When a hashtag goes viral in this way, the attention it generates grows in the form of a particular curve. And this is where SNAP goes on the attack: The program is designed to simulate this curve – all it needs is control over a certain number of Twitter accounts. SNAP's algorithm then calculates how to behave to create a fake attention curve that can outwit Twitter. "In this way, you can generate millions of content views out of 20 tweets," claims Perez Dolset.
The idea behind SNAP isn't all that original, but it did take time and money to understand Twitter and to learn how to get around its mechanisms for combating manipulation. Gonzalvez, the head of development, says one of the challenges was storing the large quantities of tweets needed for real-time analysis in a powerful database. The source code indicates that an initial version of SNAP existed in 2012 – and it is likely that it was ahead of the game from a technical standpoint.
Perez Dolset initially pitched the program to a telecommunications company in Panama. But a strange thing happened before he managed to land a commercial client. A man named Pedro Arriola, an adviser to Spain's conservative party Partido Popular (PP), which led the country's government at the time, also happened to be a member of the supervisory board of Perez Dolset's university. And, says Perez Dolset, Arriola recognized that SNAP could also be useful in the political arena.
The conservatives deployed SNAP in the next general election, in December 2015, and again when Spain held new elections in June 2016. Perez Dolset's developers even programmed a special app, which hundreds of activists handpicked by the party loaded onto their smartphones. To make something go viral, all party leaders had to do was to enter the desired hashtag into an interface. SNAP's algorithm would then calculate the perfect curve – and send instructions to the mobile phones of the party activists, who basically just had to push a button for SNAP to generate what seemed like a natural flood of interest.
"It was the first tool that tried to do something like that," says Juan Corro, who was the head of communications at Partido Popular at the time. "The good thing about it was ... it would maximize the impact of the network." Partido Popular paid a symbolic price of 6,000 euros to use it. The Andalusian regional chapter of the Spanish Socialist Workers' Party, the country's other major party, also had access to SNAP.
Looking back, Juan Corro of Partido Popular claims that SNAP wasn't as effective as had been hoped and the party didn't deploy it again in 2019 general election campaign. Nevertheless, it doesn't change the fact that the first targeted manipulation on a social network for the purposes of an election campaign appears to have happened in the EU. "In these elections," says Perez Dolset, "we ran a troll factory."
The Trolls of St. Petersburg
By then, though, what seems to be the largest internet manipulator of all had already been active for some time: a troll factory in St. Petersburg. The Russian media had been writing for years about the "Trolls of Olgino," named after the suburb from which they flooded the internet. The organization officially registered itself as a company in 2013 under the name Internet Research Agency.
Its hundreds of workers tended to be young and earned good money by Russian standards, around $1,000 a month. But people who left the company early on described the work they did there as simplistic and repetitive. They said they were required to meet a certain quota of government-friendly posts on blogs, in forums and on websites – and that the tenor of those posts was dictated to them. At times, they would target Ukraine, at others they would paint a more flattering picture of the Russian economy than the reality. The former trolls also described being active on Facebook, Instagram, YouTube, Tumblr and the Russian social media network Vkontakte. "It was an idiotic job," the Economist quoted one ex-troll as saying. "Copy and paste."
There is no doubt the IRA is towing the government line, but is it an instrument of the Kremlin? A classified report from Germany's foreign intelligence service, the BND, and the Federal Office for the Protection of the Constitution (BfV), the country's domestic intelligence agency – which ZEIT has seen – states that the troll factory is financed "by Yevgeny Prigozhin," a confidant of Russian President Vladimir Putin. The intelligence services prepared the report for Chancellor Angela Merkel's Chancellery in 2016.
The extremely wealthy entrepreneur is occasionally called the "Kremlin chef" because his catering companies receive highly paid government contracts. Prigozhin publicly denies any connection to the trolls, and didn’t answer a number of detailed questions submitted by ZEIT. The Kremlin has also consistently denied it exerts control over the IRA. But the classified analysis from the BND and BfV explicitly names Prigozhin as one of the oligarchs who provide support for the Kremlin and "are utilized for the open and covert financing of influence activities."
Did the Trolls get an upgrade in the form of the Spanish software?
In the early years, the IRA trolls' messages were in Russian, their task that of portraying the country as a bastion of stability that was standing up to the West. Their approach tended to emphasize quantity over quality and the trolls still retweeted their tweets by hand. But the IRA began to transform in early 2014 and the organization established an English-language department. The first posts from the unit were still clumsily formulated and sometimes contained errors, but they had already tightened their focus on issues highly sensitive to Americans, like gun laws and immigration.
And the closer the fall 2016 U.S. presidential elections drew, the more active the trolls became. They began impersonating Americans, inciting against Hillary Clinton and praising Donald Trump – and they intentionally sought out every controversial issue in the U.S. to deepen societal divisions.
By September 2017, when Russian meddling became public knowledge, Trump had already long since won the election. And more and more information began to emerge. Such as the fact that the cyber-offensive had been boosted by Russian women who traveled throughout the U.S. to gather information for the IRA. They took photos in states like Louisiana and Georgia to make their subsequent tweets look as convincing as possible. In January 2018, Twitter warned up to 677,000 users that they had come into contact with accounts it believed were associated with the IRA. An indictment from a U.S. District Court spoke of "information warfare."
In Madrid, Javier Perez Dolset also read the news about the IRA's activities on Twitter. "Whoa, this looks very familiar to me," he thought to himself, he says. He began to wonder if the Russians may have got their hands on SNAP.
The Russian Connection
There's a reason Perez Dolset believes his software may have fallen into IRA hands: He has been doing business in Russia since 2007. He recalls a visit to a restaurant where the hosts served wine for 60,000 euros; and a dinner where the fish was flown in by private jet from Tunisia. They were golden years – initially, at least. In 2009, his company ZED acquired a 50.1 percent stake in the Moscow communications company Temafon. With the move, Perez Dolset had entered into an opaque tangle of companies where oligarchs close to the Kremlin seem to have been among those pulling the strings. One of the companies had closed a corrupt deal in Uzbekistan. Perez Dolset claims he didn't know anything about it.
At first, he even deepened the relationship further. In 2013, his company ZED joined forces with a number of companies, including the Russian communications firm TEMA. The new company, called ZED+, was based in the Netherlands. But just as everything was supposed to really take off, the whole thing suddenly collapsed. According to Spanish police files, some of his partners had allegedly funneled money into hidden subsidiaries. Perez Dolset claims he didn't know anything about it. ZED+ fell into difficulties, and investigations were also opened into the Uzbekistan deal. ZED+'s bank, ING, the largest in the Netherlands, also got caught up in the scandal and was eventually forced to pay hundreds of millions of euros in fines for not properly vetting its customers' accounts. Perez Dolset found himself in the crosshairs too for failing to live up to his supervisory duties. There was even talk of falsified documents, a claim he denies. ZED+ was then placed under external control in a bitter dispute that is still ongoing today. Perez Dolset is demanding 750 million euros in damages from ING.
But there's another interesting aspect of this story: Perez Dolset claims that his Russian partners blocked him from accessing Temafon during the dispute – a version of events that is corroborated by the Spanish files. Perez Dolset says the Russian partners were dismissive when he tried to phone them, and they failed to report on their activities to the Spaniards. From that point on, he says he had no control over what happened with the software.
So, did the Trolls of Olgino get a significant upgrade in the form of the Spanish software? ZEIT and its partners have taken a closer look at Perez Dolset's allegation. The fact that Temafon had access to the software was contractually established and ZEIT is in possession of a copy of the shareholder agreement. On page 24, it states: "ZED will ensure that all members of the ZED Group offer all new products and all new versions and updates, etc., of any existing or new product to the companies within the group." Perez Dolset adds that, "The code was on our server in Madrid, the partners had access."
It is also likely that SNAP would have been of interest to the IRA. Even if the software was less powerful than Perez Dolset and Gonzalvez claimed, it probably would have given the trolls a significant technological leap. "We always assumed the IRA software came from outside of Russia," says Clint Watts, senior fellow at the Foreign Policy Research Institute and a former FBI investigator. "Around that time, the IRA was significantly expanding their capacities, and it would make sense to look for more advanced."
Watts, who provided expert testimony in March 2017 before the U.S. Senate about Russian interference, also says that when he analyzed the behavior of IRA Twitter accounts together with colleagues, they realized that even though they were run by real persons, they behaved almost as coordinated as bots (which Twitter frequently discovers and deletes). According to Perez Dolset and Gonzalvez, the programmers behind SNAP, this pattern of behaviour is what SNAP’s algorithm would direct the accounts to do. Clint Watts says: "It does seem very consistent."
Germany and Europe are also targets
How, though, did the connection between Temafon and the IRA come to pass? This is the point where a critical meeting comes into play that Perez Dolset claims took place on Dec. 16, 2013: a working meal at a posh Moscow restaurant at which, according to Perez Dolset, Yevgeny Prigozhin, the Kremlin "chef," was also present. Perez Dolset says he presented SNAP during this meeting – in Prigozhin's presence.
A presentation about SNAP for the alleged lord of the trolls? That, Western intelligence analysts believe, would further substantiate the suspicion. Perez Dolset has documentation of his Moscow trip, but nothing to verify Prigozhin's presence. He says his meetings had been organized by his local partners and that he often didn't know who he was going to meet with or the reason for the meeting. "That's why I don't have records of every meeting I took," he says. "But I know I met Prigozhin. He has a face you don't forget."
In response to a request for comment from ZEIT, Prigozhin's company responded as one might expect from a troll: Of course, he founded the Irish Republican Army (which shares the abbreviation IRA). And he lived for years on a Thai island together with Perez Dolset – and the ZEIT reporter also ought to seek psychiatric care.
From the U.S. to Europe?
Today, the IRA has long since moved on. And although we may have little knowledge of the Russian trolls' software of choice, it is obvious that they continue to learn and evolve. The times when you could recognize trolls by the fact that they stopped tweeting during lunchtime in St. Petersburg have passed. And as a result of the U.S. investigations, the world now knows what it looks like when the IRA deploys all its fire power for months on end. Although it will never be possible to determine with certainty whether the IRA helped to sway the U.S. election, we do know with certainty that it promoted divisions within the country.
But what have the IRA accounts been doing in the time since then?
To find out, ZEIT has examined the activities of as many IRA-controlled Twitter accounts as possible. In October 2018, Twitter released around 9 million Russian troll tweets from 3,841 accounts that Twitter believes were controlled by the IRA.
Our analysis shows that Germany and Europe are also targets. Of those messages, Twitter identified around 100,000 tweets that were in German. The largest number, just under 15,000, were published by the troll army in September 2017, the same month as Germany's last general election. Surely no coincidence.
The most common topics are revealed by the hashtags used by the trolls: #Merkel (2,243 times), #Erdogan (1,185), #refugees ("#Flüchtlinge" in the German original; 1,182) and #stopTerror ("#stopptTerror"; 971). The hashtag #AfD, standing for Germany's right-wing populist Alternative for Germany party, is also among the Top 10 (627). Interestingly, the tweets from fall 2017 aren't exclusively directed against the government: The hashtag #merkelmuststay ("#Merkelmussbleiben) also was used 508 times.
It's not possible to determine how many people read these tweets or were influenced by them. By German standards, though, these are relevant orders of magnitude. The trolls were determined to foment discord. Often, they used opinionated tweets to feed both sides of a conflict. In addition, the IRA tweeters often chose elections and terrorist attacks as the occasions for their meddling. They used anti-Islam hashtags following terrorist attacks and pro-Brexit hashtags on the day of the British EU referendum as well as against French President Emmanuel Macron before his election.
The most successful German tweet in our sample, with around 1,000 likes and retweets, provides a good example because it brings together two issues that tend to be quite controversial: cars and Muslims. The tweet went out on July 28, 2017, the same day an asylum-seeker in Hamburg's Barmbek neighborhood stabbed a customer in a supermarket. The tweet read: "#Diesel cars are #banned to protect the population. But the borders are kept open for Islamic threats. #Barmbek."
Is there anything that can be done to counter these attempts at influence? A degree of defeatism seems to be gaining traction in expert circles. The very existence of trolls is accepted by many as irreversible.
Even prosecutors and investigators seem helpless. U.S. prosecutors are seeking to prosecute 14 people from Russia, including Prigozhin, but the chances are essentially zero that they will ever succeed in getting hold of them.
There's a similar situation in the Netherlands, where prosecutors are reportedly mulling whether to extend their investigation into the case of the MH17 passenger flight, which is believed to have been shot down by a Russian missile system in 2014, to include the IRA. Of the 298 casualties, 190 were Dutch citizens, and Russian trolls launched an extremely effective disinformation campaign aimed at muddling the origins of the strike that brought down the jet. It's unclear if there will ever be any charges.
Garvan Walshe, head of election analysis at the London Institute for Strategic Dialogue, says, "We can no longer view social media as a mirror of society." Twitter, for example, is by no means an authentic reflection of what the populace is thinking, he says. The opposite tends to be true.
And that can also be seen as evidence for the effectiveness of the propaganda distributed by the trolls. Social media platforms were once regarded as places of lively exchange, where everyone could participate in the major debates of the day. Do we now have to bury that hope?
Javier Perez Dolset, of all people, claims that his latest project is the development of software that can detect attempts to exert influence.
Translated by Daryl Lindsey