IGF 2017 WS #68 Fake News, AI Trolls & Disinformation: How Can the Internet Community Deal with Poison in the System

Short Title: 

Poison in the System: Fighting Propaganda Bots & Hate Online

Proposer's Name: Ms. Sarah Moulton
Proposer's Organization: National Democratic Institute
Co-Proposer's Name: Mr. Chris Doten
Co-Proposer's Organization: National Democratic Institute
Mr.,Chris,DOTEN,Civil Society,National Democratic Institute
Ms.,Sarah,MOULTON,Civil Society,National Democratic Institute

Additional Speakers: 

Nighat Dad

Nighat Dad is the Executive Director of Digital Rights Foundation, Pakistan, and an accomplished lawyer and a human rights activist. Nighat Dad is a pioneering campaigner around access to open internet in Pakistan and around the globe. She has been actively engaged at a policy level on issues focusing on Internet Freedom, women and technology, Digital Security and women’s empowerment. Ms. Dad was recently included in Time Magazine’s Next Generation Leaders List for her work on helping women fight online harassment.



Introduction to the topic by Moderator (5 mins)

Introduction of Topic Leaders (10 mins)

Open Discussion with Session Participants (40 mins)

The Birds of a Feather session will be driven primarily by interests of in-person and online participants, but guiding questions would include:

  • What is the role of internet policy makers in responding to nation-state engagement in online conversations?
  • What are the economic incentives for creating disinformation flows? Should one remove troll networks?
  • Does fact checking work? Should communications platforms adopt it? Who checks the fact checkers?
  • How can we address the disproportionate impact of trolls on women’s online participation?
  • With accelerating AI behind sock-puppet accounts, what is the future of civic dialogue online?

Concluding remarks (5 mins)



- Session Title: Fake News, AI Trolls & Disinformation: How Can the Internet Community Deal with Poison in the System

- Date: December 19, 2017                 

- Time: 15:00 - 16:00    

- Session Organizer: National Democratic Institute            

- Chair/Moderator:  Chris Doten, National Democratic Institute (onsite), Sarah Moulton, National Democratic Institute (online)            

- Rapporteur/Notetaker: Priyal Bhatt, National Democratic Institute    

- List of Speakers and their institutional affiliations:                

Matt Chessen, U.S. Department of State
Donatien Niyongendako, DefendDefenders
Alina Polyakova, The Brookings Institution
Samuel Woolley, Oxford Internet Institute-University of Oxford

- Key Issues raised (1 sentence per issue):                 

  • People are looking for infotainment and consume fake news because it is emotionally pleasing.
  • Governments need to strike a balance between the panic surrounding disinformation during elections and short-term solutions that aren’t censorship or criminalizing online discourse.
  • Understanding the economic incentives that support dissemination of disinformation can illustrate the motivations and roles of various platforms and explain why concerted action against disinformation is difficult.
  • With respect to emerging technologies, it is important to consider not just disinformation in the present, but possible disinformation practices of the future. For example, it will soon be possible to alter audio and video just as one can fake text.
  • While it is necessary to think individually about the responsibility of the digital platforms for their content and the role of government and civil society, the approach to combat disinformation must be a multi-stakeholder, democratic process.

- If there were presentations during the session, please provide a 1-paragraph summary for each presentation: N/A

- Please describe the Discussions that took place during the workshop session (3 paragraphs):                

The discussions during this workshop session were focused primarily on understanding the factors that support and perpetuate disinformation practices, as well as sharing possible solutions to combat misinformation. From the onset of the conversation, both participants and panelists noted the often disregarded, emotional aspect of disinformation; people consume disinformation--irrespective of their belief in it--for infotainment and because it is emotionally pleasing.

The economic incentives that support disinformation were also considered. There is a market for disinformation--with consumers interested in infotainment and producers interested in the monetary gains from disseminating disinformation. A participant shared the example of Macedonia where youth shared disinformation primarily for the opportunity to make money. Given these monetary benefits, the implications of taxing disinformation or changing the revenue structure for media organizations were debated, and broader questions of who controls and categorizes information were raised.

Both participants and panelists also shared ideas on short-term and long-term solutions that could make disinformation less prevalent. Long-term solutions primarily focused on educating consumers of information to differentiate between legitimate and illegitimate news, as well as civic and ethics education for those who build the platforms. Questions were also raised on reconciling these long-term solutions with the short-term needs of government to eliminate disinformation during particularly critical periods, such as elections. For example, in East Africa, impersonation of human rights defenders accounts is common in the election period or during demonstrations. In an effort to address the fake news, states look for quick solutions that may lead to censorship and criminalizing online discourse. In light of this, it is important for civil society to work with governments and the platforms themselves to ensure that any regulation does not drown out digital activism.

- Please describe any Participant suggestions regarding the way forward/ potential next steps /key takeaways (3 paragraphs):    

Both panelists and participants shared a variety of ideas and solutions considering next steps for the IGF community. From civic education programs to community-driven verification of news, a number of practical ideas were shared within the group. An overarching theme of the conversation was the need to involve multiple actors. Panelists noted the need for collaboration between government, civil society, and owners of the digital platforms in setting the narrative rather than simply reacting to past events. Disinformation poses a challenge to democracy, but it also requires a democratic solution.

Consideration was also given to emerging technologies, particularly with respect to artificial intelligence, and how stakeholders can better prepare for disinformation of the future. The possibility of manipulation of audio and video information is expected to grow in the coming years and must be addressed, perhaps through digital signatures. However, many of these technologies are dual-use which makes effective regulation more challenging. It may be more worthwhile to focus on malicious actors rather than the tools themselves.

A number of individuals who participated in the conversation formed new relationships that they reported would be useful partnerships in the future.

Gender Reporting

- Estimate the overall number of the participants present at the session: 125-150 participants

- Estimate the overall number of women present at the session: ~60 women

- To what extent did the session discuss gender equality and/or women’s empowerment? The workshop session did not discuss these issues.

- If the session addressed issues related to gender equality and/or women’s empowerment, please provide a brief summary of the discussion: N/A

Session Format: Birds of a Feather - 60 Min

Country: United States
Stakeholder Group: Civil Society

Country: United States
Stakeholder Group: Civil Society

Speaker: Matt Chessen
Speaker: Alina Polyakova
Speaker: Samuel Woolley
Speaker: Donatien Niyongendako

Content of the Session:
While this is designed to be a free-flowing conversation, the discussion leader will ensure that the group touches on three core points about shaping a better internet for the future: how to deal with trolling and online harassment, particularly via AI bots, without infringing on free speech; how social platforms and content algorithms can be manipulated to surface hatred or false information, and what, if anything, can be done about it; and the role of governmental policymakers in writing legal frameworks to support an open but inclusive online dialogue. Given the range of experiences of our participants, as well as the other diverse voices we expect will participate in the conversation, we expect a vibrant discussion.

Relevance of the Session:
At the same time that the internet is becoming the indispensable communications medium of a majority of citizens around the world, the online space increasingly seems to be warping into a hostile environment for civil dialogue. The openness that makes the internet the greatest communication platform in human history is being misshapen by forces who attempt to drown out speech with hatred, misinform with biased propaganda, and craft realities grounded in alternative sets of facts. Critical questions will be answered - or ignored - in the next years that will shape the role of the internet in our lives: the obligations of policymakers to reign in online hatred vs. protecting free speech; of social platforms to build content algorithms that do not reflect unconscious bias or can be easily manipulated; and the role of civil society in pushing back against the internet deluge of falsehood and fake news, sometimes pushed by nation-states. How can decisions or policies enacted by the wider internet governance community help or hinder the outcomes take by these different stakeholders?

Tag 1: Freedom of Expression Online
Tag 2: Disinformation
Tag 3:

While the Birds of a Feather session will not have formal presentations, key topic leaders have been identified who will further guide conversation around key discussion questions posed to the session participants, based on their diverse expertise. Alina Polyakova of the Atlantic Council (confirmed) focuses on the uses of the internet by nation-states to manipulate public opinion, while Matt Chessen (confirmed) as a Digital Diplomacy team member from the US State Department will articulate a governmental perspective on mass communication. Oxford’s Samuel Woolley (confirmed) will share a perspective advanced research on the ways in which computational propaganda is being distributed by AI-powered bots, while Donatien Niyongendako of DefendingDefenders (confirmed) can discuss the impacts his civic organization has seen on the suppression of public communication in the Horn of Africa.

This Birds of a Feather session will have lead speakers representing a variety of stakeholder groups, including government, academia, civil society, and technology specialists. No suggested members have been participants in an IGF event in the past, and each of the confirmed speakers represents differing areas of subject matter expertise and geographic focus. Additional speakers to be confirmed will include representatives from the Global South, as well as those who have direct experience encountering online harassment and trolling - particularly women. We intend to use the online discussion capabilities to further diversify the types of participants in the discussion.

Onsite Moderator: Chris Doten
Online Moderator: Sarah Moulton
Rapporteur: Amanda Domingues

Online Participation:
The livestream for this event will be promoted in advance through the social networks of the organizer, as well as by discussants. Responses to the session’s key discussion topics will be solicited in advance from community members who will not be able to participate in the live discussion, such as those located in time zones not conducive to viewing the livestream. Questions and comments received will be shared with the Onsite Moderator for incorporation into the live discussion. During the live session, the Online Moderator will coordinate with remote participants to facilitate live audio participation in the discussion if technical media permit. In addition, non-audio based comments and questions received through social media or chat discussions in the virtual meeting space will be integrated into the discussion directly by the Online Moderator.

Discussion facilitation:
As a birds of a feather session, we hope for a lively, perhaps confrontational discussion as our speakers engage as peers with the other participants around the table. The facilitator will sharpen contrasts between points of view, request examples from everyone in the room, and note points of disagreement. The goal of this session is not necessarily to reach consensus on the nature of the problem or potential mitigations, but rather to elucidate a variety of frank points of view, and perceptions of what key internet stakeholders ought to be doing in response. This is not a panel, and as such all speakers will be looking to engage substantively with the others in the room. Specific conversational prompts may include:
What is the role of internet policy makers in responding to nation-state engagement in online conversations?
What are the economic incentives for creating disinformation flows? Should one remove troll networks?
Does fact checking work? Should communications platforms adopt it? Who checks the checkers?”
With accelerating AI behind sock-puppet bot accounts, what is the future of civic dialogue online?

Conducted a Workshop in IGF before?: No
Link to Report:

Background Paper