Assessing the Ethics and Politics of Policing the Internet for Extremist Material

The Internet serves not only as a breeding ground for extremism, but also offers myriad data streams which potentially hold great value to law enforcement. The report by the OII’s Ian Brown and Josh Cowls for the VOX-Pol project: Check the Web: Assessing the Ethics and Politics of Policing the Internet for Extremist Material explores the complexities of policing the web for extremist material, and its implications for security, privacy and human rights. Josh Cowls discusses the report with blog editor Bertie Vidgen.*

*please note that the views given here do not necessarily reflect the content of the report, or those of the lead author, Ian Brown.

In terms of counter-speech there are different roles for government, civil society, and industry. Image by Miguel Discart (Flickr).


Ed: Josh, could you let us know the purpose of the report, outline some of the key findings, and tell us how you went about researching the topic?

Josh: Sure. In the report we take a step back from the ground-level question of ‘what are the police doing?’ and instead ask, ‘what are the ethical and political boundaries, rationale and justifications for policing the web for these kinds of activity?’ We used an international human rights framework as an ethical and legal basis to understand what is being done. We also tried to further the debate by clarifying a few things: what has already been done by law enforcement, and, really crucially, what the perspectives are of all those involved, including lawmakers, law enforcers, technology companies, academia and many others.

We derived the insights in the report from a series of workshops, one of which was held as part of the EU-funded VOX-Pol network. The workshops involved participants who were quite high up in law enforcement, the intelligence agencies, the tech industry civil society, and academia. We followed these up with interviews with other individuals in similar positions and conducted background policy research.

Ed: You highlight that many extremist groups (such as Isis) are making really significant use of online platforms to organize, radicalize people, and communicate their messages.

Josh: Absolutely. A large part of our initial interest when writing the report lay in finding out more about the role of the Internet in facilitating the organization, coordination, recruitment and inspiration of violent extremism. The impact of this has been felt very recently in Paris and Beirut, and many other places worldwide. This report pre-dates these most recent developments, but was written in the context of these sorts of events.

Given the Internet is so embedded in our social lives, I think it would have been surprising if political extremist activity hadn’t gone online as well. Of course, the Internet is a very powerful tool and in the wrong hands it can be a very destructive force. But other research, separate from this report, has found that the Internet is not usually people’s first point of contact with extremism: more often than not that actually happens offline through people you know in the wider world. Nonetheless it can definitely serve as an incubator of extremism and can serve to inspire further attacks.

Ed: In the report you identify different groups in society that are affected by, and affecting, issues of extremism, privacy, and governance – including civil society, academics, large corporations and governments

Josh: Yes, in the later stages of the report we do divide society into these groups, and offer some perspectives on what they do, and what they think about counter-extremism. For example, in terms of counter-speech there are different roles for government, civil society, and industry. There is this idea that ISIS are really good at social media, and that that is how they are powering a lot of their support; but one of the people that we spoke to said that it is not the case that ISIS are really good, it is just that governments are really bad!

We shouldn’t ask government to participate in the social network: bureaucracies often struggle to be really flexible and nimble players on social media. In contrast, civil society groups tend to be more engaged with communities and know how to “speak the language” of those who might be vulnerable to radicalization. As such they can enter that dialogue in a much more informed and effective way.

The other tension, or paradigm, that we offer in this report is the distinction between whether people are ‘at risk’ or ‘a risk’. What we try to point to is that people can go from one to the other. They start by being ‘at risk’ of radicalization, but if they do get radicalized and become a violent threat to society, which only happens in the minority of cases, then they become ‘a risk’. Engaging with people who are ‘at risk’ highlights the importance of having respect and dialogue with communities that are often the first to be lambasted when things go wrong, but which seldom get all the help they need, or the credit when they get it right. We argue that civil society is particularly suited for being part of this process.

Ed: It seems like the things that people do or say online can only really be understood in terms of the context. But often we don’t have enough information, and it can be very hard to just look at something and say ‘This is definitely extremist material that is going to incite someone to commit terrorist or violent acts’.

Josh: Yes, I think you’re right. In the report we try to take what is a very complicated concept – extremist material – and divide it into more manageable chunks of meaning. We talk about three hierarchical levels. The degree of legal consensus over whether content should be banned decreases as it gets less extreme. The first level we identified was straight up provocation and hate speech. Hate speech legislation has been part of the law for a long time. You can’t incite racial hatred, you can’t incite people to crimes, and you can’t promote terrorism. Most countries in Europe have laws against these things.

The second level is the glorification and justification of terrorism. This is usually more post-hoc as by definition if you are glorifying something it has already happened. You may well be inspiring future actions, but that relationship between the act of violence and the speech act is different than with provocation. Nevertheless, some countries, such as Spain and France, have pushed hard on criminalising this. The third level is non-violent extremist material. This is the most contentious level, as there is very little consensus about what types of material should be called ‘extremist’ even though they are non-violent. One of the interviewees that we spoke to said that often it is hard to distinguish between someone who is just being friendly and someone who is really trying to persuade or groom someone to go to Syria. It is really hard to put this into a legal framework with the level of clarity that the law demands.

There is a proportionality question here. When should something be considered specifically illegal? And, then, if an illegal act has been committed what should the appropriate response be? This is bound to be very different in different situations.

Ed: Do you think that there are any immediate or practical steps that governments can take to improve the current situation? And do you think that there any ethical concerns which are not being paid sufficient attention?

Josh: In the report we raised a few concerns about existing government responses. There are lots of things beside privacy that could be seen as fundamental human rights and that are being encroached upon. Freedom of association and assembly is a really interesting one. We might not have the same reverence for a Facebook event plan or discussion group as we would a protest in a town hall, but of course they are fundamentally pretty similar.

The wider danger here is the issue of mission creep. Once you have systems in place that can do potentially very powerful analytical investigatory things then there is a risk that we could just keep extending them. If something can help us fight terrorism then should we use it to fight drug trafficking and violent crime more generally? It feels to me like there is a technical-military-industrial complex mentality in government where if you build the systems then you just want to use them. In the same way that CCTV cameras record you irrespective of whether or not you commit a violent crime or shoplift, we need to ask whether the same panoptical systems of surveillance should be extended to the Internet. Now, to a large extent they are already there. But what should we train the torchlight on next?

This takes us back to the importance of having necessary, proportionate, and independently authorized processes. When you drill down into how rights privacy should be balanced with security then it gets really complicated. But the basic process-driven things that we identified in the report are far simpler: if we accept that governments have the right to take certain actions in the name of security, then, no matter how important or life-saving those actions are, there are still protocols that governments must follow. We really wanted to infuse these issues into the debate through the report.

Read the full report: Brown, I., and Cowls, J., (2015) Check the Web: Assessing the Ethics and Politics of Policing the Internet for Extremist Material. VOX-Pol Publications.

Josh Cowls is a a student and researcher based at MIT, working to understand the impact of technology on politics, communication and the media.

Josh Cowls was talking to Blog Editor Bertie Vidgen.

Does crowdsourcing citizen initiatives affect attitudes towards democracy?

Crowdsourcing legislation is an example of a democratic innovation that gives citizens a say in the legislative process. In their Policy and Internet journal article ‘Does Crowdsourcing Legislation Increase Political Legitimacy? The Case of Avoin Ministeriö in Finland’, Henrik Serup Christensen, Maija Karjalainen and Laura Nurminen explore how involvement in the citizen initiatives affects attitudes towards democracy. They find that crowdsourcing citizen initiatives can potentially strengthen political legitimacy, but both outcomes and procedures matter for the effects.

Crowdsourcing is a recent buzzword that describes efforts to use the Internet to mobilize online communities to achieve specific organizational goals. While crowdsourcing serves several purposes, the most interesting potential from a democratic perspective is the ability to crowdsource legislation. By giving citizens the means to affect the legislative process more directly, crowdsourcing legislation is an example of a democratic innovation that gives citizens a say in the legislative process. Recent years have witnessed a scholarly debate on whether such new forms of participatory governance can help cure democratic deficits such as a declining political legitimacy of the political system in the eyes of the citizenry. However, it is still not clear how taking part in crowdsourcing affects the political attitudes of the participants, and the potential impact of such democratic innovations therefore remain unclear.

In our study, we contribute to this research agenda by exploring how crowdsourcing citizens’ initiatives affected political attitudes in Finland. The non-binding Citizens’ Initiative instrument in Finland was introduced in spring 2012 to give citizens the chance to influence the agenda of the political decision making. In particular, we zoom in on people active on the Internet website Avoin Ministeriö (Open Ministry), which is a site based on the idea of crowdsourcing where users can draft citizens’ initiatives and deliberate on their contents. As is frequently the case for studies of crowdsourcing, we find that only a small portion of the users are actively involved in the crowdsourcing process. The option to deliberate on the website was used by about 7% of the users; the rest were only passive readers or supported initiatives made by others. Nevertheless, Avoin Ministeriö has been instrumental in creating support for several of the most successful initiatives during the period, showing that the website has been a key actor during the introductory phase of the Citizens’ initiative in Finland.

We study how developments in political attitudes were affected by outcome satisfaction and process satisfaction. Outcome satisfaction concerns whether the participants get their preferred outcome through their involvement, and this has been emphasized by proponents of direct democracy. Since citizens get involved to achieve a specific outcome, their evaluation of the experience hinges on whether or not they achieve this outcome. Process satisfaction, on the other hand, is more concerned with the perceived quality of decision making. According to this perspective, what matters is that participants find that their concerns are given due consideration. When people find the decision making to be fair and balanced, they may even accept not getting their preferred outcome. The relative importance of these two perspectives remains disputed in the literature.

The research design consisted of two surveys administered to the users of Avoin Ministeriö before and after the decision of the Finnish Parliament on the first citizens’ initiative in concerning a ban on the fur-farming industry in Finland. This allowed us to observe how involvement in the crowdsourcing process shaped developments in central political attitudes among the users of Avoin Ministeriö and what factors determined the developments in subjective political legitimacy. The first survey was conducted in fall 2012, when the initiators were gathering signatures in support of the initiative to ban fur-farming, while the second survey was conducted in summer 2013 when Parliament rejected the initiative. Altogether 421 persons filled in both surveys, and thus comprised the sample for the analyses.

The study yielded a number of interesting findings. First of all, those who were dissatisfied with Parliament rejecting the initiative experienced a significantly more negative development in political trust compared to those who did not explicitly support the initiative. This shows that the crowdsourcing process had a negative impact on political legitimacy among the initiative’s supporters, which is in line with previous contributions emphasizing the importance of outcome legitimacy. It is worth noting that this also affected trust in the Finnish President, even if he has no formal powers in relation to the Citizens’ initiative in Finland. This shows that negative effects on political legitimacy could be more severe than just a temporary dissatisfaction with the political actors responsible for the decision.

Nevertheless, the outcome may not be the most important factor for determining developments in political legitimacy. Our second major finding indicated that those who were dissatisfied with the way Parliament handled the initiative also experienced more negative developments in political legitimacy compared to those who were satisfied. Furthermore, this effect was more pervasive than the effect for outcome satisfaction. This implies that the procedures for handling non-binding initiatives may play a strong role in citizens’ perceptions of representative institutions, which is in line with previous findings emphasising the importance of procedural aspects and evaluations for judging political authorities.

We conclude that there is a beneficial impact on political legitimacy if crowdsourced citizens’ initiatives have broad appeal so they can be passed in Parliament. However, it is important to note that positive effects on political legitimacy do not hinge on Parliament approving citizens’ initiatives. If the MPs invest time and resources in the careful, transparent and publicly justified handling of initiatives, possible negative effects of rejecting initiatives can be diminished. Citizens and activists may accept an unfavourable decision if the procedure by which it was reached seems fair and just. Finally, the results give reason to be hopeful about the role of crowdsourcing in restoring political legitimacy, since a majority of our respondents felt that the possibility of crowdsourcing citizens’ initiatives clearly improved Finnish democracy.

While all hopes may not have been fulfilled so far, crowdsourcing legislation therefore still has potential to help rebuild political legitimacy.

Read the full article: Christensen, H., Karjalainen, M., and Nurminen, L., (2015) Does Crowdsourcing Legislation Increase Political Legitimacy? The Case of Avoin Ministeriö in Finland. Policy and Internet 7 (1) 25–45.

Henrik Serup Christensen is Academy Research Fellow at SAMFORSK, Åbo Akademi University.

Maija Karjalainen is a PhD Candidate at the Department of Political Science and Contemporary History in the University of Turku, Finland.

Laura Nurminen is a Doctoral Candidate at the Department of Political and Economic Studies at Helsinki University, Finland.