On the hunt for bots and disinformation

How can we know if the account we see on Facebook or X (formerly Twitter) is powered by AI? It is in fact not that easy to determine, explains Matteo Magnani, Professor of Data Science.

“Identifying bots can be difficult, but there are some behaviours that may point to automation. For example, when they post a very large number of messages,” notes Magnani.

Can detect spread of disinformation

He develops algorithms that can be used to identify bots and “malicious accounts”, for example, separate Facebook accounts that appear to have nothing to do with each other but are actually run by the same operator. Thanks to the algorithms, it is possible to track their posts, compare them with each other and identify that they come from the same source.

“These accounts can spread both true and false information, but what they have in common is that they want you to hear what they say. An operator with multiple accounts and millions of followers can actually influence our opinions, which is why it’s important to identify them.”

The algorithms are good at identifying suspicious accounts, but determining whether or not they share disinformation requires a different type of analysis. In previous research projects, Magnani has therefore collaborated with fact-checkers who work to identify and remove harmful information from social media.

With the development of AI, the bots have become “smarter” and more difficult to identify. For example, they can use ChatGPT and similar solutions to reformulate their posts so they are not identical. Magnani is therefore developing algorithms that consider more variables than just the exact content of the text.

Studying climate change

The same methods used to detect bots can also help to map other events on the web. Magnani is working together with Alexandra Segerberg at the Department of Government to study how different actors communicate about climate change. They do this by examining what people write on social media and what images they share in connection with climate conferences.

“Today, we can ask an algorithm to summarise the content of millions of images shared online during an event. It can also summarise what millions of people are talking about in relation to that same event. This was something that previously required a manual review, making the amount of data that could be analysed very limited.”

Methods like these pave the way for entirely new ways of studying society. Magnani is currently in the early phase of writing a textbook on how AI can be used in social sciences. He will discuss its opportunities: from using ChatGPT to perform simpler tasks such as coding interviews, to more advanced uses such as allowing AI to summarise the content of images shared for propaganda purposes. A large part of the book will also be devoted to research ethics.

“I want to show how to use these data and methods so that we get valid research results, but also to discuss how the use of AI can change the way we do research. There are major risks in allowing AI to make decisions that we would normally make ourselves.”

Wants to map the entire population of Sweden

Another research project, still in its infancy, is to build a social network detailing the entire adult population of Sweden. To achieve this, Magnani will work with researchers from different disciplines to develop a computer-based model of society and the relationships between people.

“In Sweden, there is good access to data beyond social media. For example, we can get information about our family relationships, we can see when people worked at the same place, we can find out what and when they studied together and see where they live and have lived previously. By mapping these relationships, we can study society in an entirely new way.”

Once the network is in place, the idea is to use it to study the emergence of segregation and the polarisation of opinion. For example, it will be possible to investigate whether a person’s attitude towards refugees is affected by where they live, who they are married to or what other relationships they have had throughout their lives.

“If you use a survey, you can ask a person what they think. Here we can see how what they think is influenced by the social relationships they have.”

A computer-based model of society can also be used to expose the “population” to different scenarios. What would happen if housing prices in a particular neighbourhood were reduced? Would segregation decrease?

“You can experiment in a way that you can’t do with real people.”

Aren't there risks involved in developing such models? Could it not inspire companies that want to map the population in order to influence them on various issues, as in the case of Cambridge Analytica?

“When developing methods, we must always weigh the advantages against the disadvantages. Our methods can be used to solve problems and make things better, but there is always a risk that they will be used in the wrong way,” replies Magnani.

“Similar methods can already be used today by companies and governments. Within research, our methods are published and made available to the public; that’s better than keeping them hidden within states. When the methods are made public, there is also the opportunity to legislate against aspects deemed to be harmful to the population.”

Sandra Gunnarsson