A Field Guide To Fake News And Other Information Disorders Single Pages

User Manual: Pdf

Open the PDF directly: View PDF .
Page Count: 216

Download
Open PDF In Browser	View PDF

ELD
I
F
A
TO
E
D
GUI

INF

O

her
t
o
and
ERS
D
R
ISO
D
ION
T
A
RM

A COLLECTION OF RECIPES
FOR THOSE WHO LOVE TO COOK
WITH DIGITAL METHODS
COMPILED BY

Liliana Bounegru
Jonathan Gray
Tommaso Venturini
Michele Mauri

1

A FIELD GUIDE TO "FAKE NEWS" AND OTHER
INFORMATION DISORDERS

Compiled by Liliana Bounegru, Jonathan Gray, Tommaso Venturini
and Michele Mauri.
This guide explores the use of digital methods to study false viral news,
political memes, trolling practices and their social life online. It is a
project of the Public Data Lab with support from First Draft.
The Public Data Lab (publicdatalab.org) is an interdisciplinary network
seeking to facilitate research, democratic engagement and public
debate around the future of the data society.
First Draft (firstdraftnews.com) is dedicated to improving skills and
standards in the reporting and sharing of information that emerges
online.

© 2017 Public Data Lab. Amsterdam.
The guide is released under the Creative Commons Attribution License
(creativecommons.org/licenses/by/4.0/). This means you can freely
share, adapt and draw on this work as long as you give credit, as per
the terms of the license.
If you reproduce or draw on material from this guide, we’d be grateful
if you could credit and link back as per the following statement:
“This article draws on A Field Guide to "Fake News" and Other
Information Disorders, a collaboration of the Public Data Lab and
First Draft. For further details see: http://fakenews.publicdatalab.
org”
The copyright for the research that this guide draws upon remains with
its respective contributors.
This project would not have been possible without the contributions of
researchers from the following institutions:
Centre for Journalism Studies, University of Ghent (NL)
Citizen Data Lab, Amsterdam University of Applied Sciences (NL
DensityDesign Lab, Politecnico di Milano (IT)
Digital Methods Initiative, Media Studies,
University of Amsterdam (NL)
Govcom.org Foundation, Amsterdam (NL)
Institut National de Recherche en Informatique et en
Automatique (INRIA) (FR)
King's College London (UK)
Laboratoire d’Étude des Sciences et des Techniques (STS-Lab),
Université de Lausanne (CH)
Laboratoire Interdisciplinaire Sciences Innovations Sociétés
(LISIS), Université Paris-Est Marne-la-Vallée (FR)
Médialab, Sciences Po, Paris (FR)
Techno-Anthropology Lab, Aalborg University Copenhagen (DK)
University of Siegen (DE)
Published by Public Data Lab.
Version 1.0.0
Released in January 2018.

ELD
I
F
A
TO
E
D
GUI

INF

O

her
t
o
and
ERS
D
R
ISO
D
ION
T
A
RM

CONTENTS
6		

Introduction

14

Conventions used in the book

		

CHAPTER 1

17

MAPPING FAKE NEWS HOTSPOTS
ON FACEBOOK
1. What publics does fake news animate on
Facebook?
2. How may the trajectory of a fake news story be
traced on Facebook?
3. Do fact-checking initiatives reach the publics of
fake news on Facebook?

20
40
52
		

CHAPTER 2

61

TRACING THE CIRCULATION
OF FAKE NEWS ON THE WEB
1. Where do fake news originate? By what sites are
they first retransmitted?
2. Which are the most visible sources related to a fake
story? When and by whom are they mentioned?

64
78
		

CHAPTER 3

95

USING TRACKER SIGNATURES TO MAP THE
TECHNO-COMMERCIAL UNDERPINNINGS
OF FAKE NEWS SITES
1. Do fake news sites use different kinds of trackers
from mainstream media sites?
2. How can fake news and mainstream media sites be
profiled based on their tracker usage?
3. How do tracker ecologies on fake news sites change
over time?
4. Which other websites share the same tracker IDs as
fake news sites?
5. Do trackers associated with hyper-partisan, and
misleading information sites vary across language
spheres?

97
103
109
115
121

		

CHAPTER 4

127
130
139
148

STUDYING POLITICAL MEMES
ON FACEBOOK
1. How can meme spaces on Facebook be traced?
2. How do memes frame political and media events?
3. How may the content of memes be studied?

		

CHAPTER 5

161

184

MAPPING TROLL-LIKE PRACTICES
ON TWITTER
1. How may we detect Twitter accounts which
negatively target political representatives?
2. How may we characterise the sources of
troll-like activity?
3. How may troll-like practices be characterised?

199

Conclusion

203

Glossaries

210

Contributors and acknowledgements

165
172

A FIELD GUIDE TO “FAKE NEWS”
AND OTHER INFORMATION
DISORDERS
INTRODUCTION
[1] See, for example, Robert
Darnton, “The True History
of Fake News”, The New
York Review of Books,
February 2017, available at:
http://www.nybooks.com/
daily/2017/02/13/the-truehistory-of-fake-news/
[2] See, for example, “Sky
Views: Facebook's fake
news threatens democracy”, Sky News: http://
news.sky.com/story/skyviews-democracy-burns-asfacebook-lets-fake-newsthrive-10652711
[3] See Edson Tandoc, Zheng
Wei Lim & Richard Ling
"Defining ‘Fake News’",
Digital Journalism, 811, August 2017. pp. 1–17. 2017.
Available at: https://www.
tandfonline.com/doi/full/10
.1080/21670811.2017.136
0143
[4] See Claire Wardle & Derakhshan Hossein, Information Disorder: Toward an
interdisciplinary framework
for research and policymaking (Report to the
Council of Europe), 2017.

What is “fake news”? And what can be done about it?
Depending on who you ask, fake news is said to represent
a step-change in information warfare; an emerging form of
cynical profiteering; an engine for energising “alt-right” and
other digitally mediated grassroots political mobilisations
around the world; a partisan battle cry for a new liberal
“ministry of truth”; an unwanted byproduct of the online
platforms which organise our digital societies; or a canary
call signalling a collapse of consensus around established
institutions and processes of knowledge production,
heralding a new “post-truth” era in politics and public life.
According to some commentators fake news is just old wine
in new bottles – and similar misinformation phenomena
have existed for at least as long as the printing press and
other communication technologies through which they
circulate.[1] Others suggest that new online platforms
accelerate and “supercharge” their circulation in a way
which introduces hitherto unprecedented challenges and
dynamics.[2] Others even claim that the term "fake news"
should be avoided altogether because it is too vague[3],
politically dangerous[4]; indistinguishable from past forms
of disinformation[5]; charged with an over-simplistic idea of
truth as direct correspondence to reality[6], and missing the

[5] See Caroline Jack, What’s
Propaganda Got To Do With
It? Available at: https://
points.datasociety.net/
whats-propaganda-got-todo-with-it-5b88d78c3282
[6] See Michael Lynch, Posttruth, alt-facts, and asymmetric controversies, 2017,
First 100 Days, Available at:
http://first100days.stsprogram.org/2017/02/06/
post-truth-alt-facts-andasymmetric-controversiespart-i/
[7] See Henry Jenkins, Sam
Ford, & Joshua Benjamin Green, Spreadable media. New York:
New York University
Press, 2017, Available at:
http://doi.org/10.1017/
CBO9781107415324.004
[8] See, John Naughton, “Facebook and Twitter could pay
the price for hate speech”,
The Guardian, March 2017:
https://www.theguardian.
com/commentisfree/2017/
mar/19/john-naughtongermany-fine-social-mediasites-facebook-twitter-hatespeech
[9] See, for example, Full Fact,
“The State of Automated
Factchecking”, https://fullfact.org/automated and Lori
Hawkins, “Austin startup
wants to end fake news
- and fake everything else on the internet”, 512Tech,
February 2017: http://www.512tech.com/technology/
austin-startup-wants-endfake-news-and-fake-everything-else-the-internet/EcchWFgrl4PQmjPvmkzycJ/
[10] Bruno Latour, Reassembling
the Social: An Introduction
to Actor-Network-Theory,
Oxford: Oxford University
Press, 2009

INTRODUCTION

most important and dangerous features of the phenomenon
it describes which is not deceptiveness, but "spreadability".[7]
Proposed responses include new media literacy, educational
and fact-checking initiatives; new laws, policies and fines for
technology companies who fail to remove offending content [8]; and a host of new startups and technical fixes – from
authenticated content to automated fact-checking projects.[9]
Across these different kinds of responses, observers agree
that the term “fake news” is deceptive and that these
problematic fabrications cannot be straightforwardly defined.
And while we “follow the actors” [10] and retain the main
name by which these activities have been originally turned
into an issue of public concern to indicate the controversy
that prompted us to undertake this empirical investigation,
we recognise that fabrications gathered under the label
“fake news” come in many different shades. This need
not be taken as proof of the futility of investigating this
phenomenon. On the contrary: their different shades are
what is at stake in our investigation and accepting that there
is no easy way to demarcate between “fake” and “non-fake”
across all cases opens interesting research opportunities. It
is precisely because its forms and contents are designed to
mimic those of mainstream media – and precisely because
it travels through similar circuits – that fake news offers us
the occasion to study not just the strategies and formats of
fakeness, but the politics and composition of the media and
information environments of the digital age more generally.
This guide aims to enrich public debate and catalyse
collective inquiry around this rapidly evolving and highly
contested issue – by suggesting different ways in which it
can be empirically studied, mapped and investigated online.
Ultimately our hope is not just to provide better accounts of
the issue of fake news and phenomena associated with it,
but also to contribute to more substantive forms of public
engagement around it. We hope this guide will contribute
to facilitating broader public debate and involvement
7

around processes of reshaping platforms and policies,
laws and infrastructures, technologies and standards that
are implicated in the circulation of fake news and other
fabrications. This includes remaining attentive to possible
unintended consequences of these different responses, as well
as other interests and concerns.
The guide explores the notion that fake news is not just
another type of content that circulates online, but that it is
precisely the character of this online circulation and reception
that makes something into fake news. In this sense fake
news may be considered not just in terms of the form or
content of the message, but also in terms of the mediating
infrastructures, platforms and participatory cultures which
facilitate its circulation. In this sense, the significance of fake
news cannot be fully understood apart from its circulation online.
It is the register of this circulation that also enables us to
trace how material that starts its life as niche satire can be
repackaged as hyper-partisan clickbait to generate advertising
money and then continue life as an illustration of dangerous
political misinformation.
As a consequence this field guide encourages a shift from
focusing on the formal content of fabrications in isolation to
understanding the contexts in which they circulate online.
This shift points to the limits of a “deficit model” approach
– which might imply that fabrications thrive only because
of a deficit of factual information. In the guide we suggest
new ways of mapping and responding to fake news beyond
identifying and fact-checking suspect claims – including
“thicker” accounts of circulation as a way to develop a richer
understanding of how fake news moves and mobilises
people, more nuanced accounts of “fakeness” and responses
which are better attuned to the phenomenon.
[11] See Theodore Porter,
“Thin Description: Surface
and Depth in Science and
Science Studies.” Osiris,
2012: www.jstor.org/
stable/10.1086/667828
8

While online and platform metrics often serve to take
measure of engagement by means of what Theodore Porter
calls “thin descriptions”[11] – i.e. aggregated quantities such
as total likes, shares, posts – we suggest different ways of
A FIELD GUIDE TO FAKE NEWS

exploring how different publics engage with and ascribe
meaning to fake news and how this moves and mobilises
different actors in the process. In doing so while we start
our inquiry with fake news, we end up surfacing a wide
range of grassroots political, media and participatory
cultures online and the social and political issues around
which they assemble. Some of these may challenge and
prompt a rethinking of our ideas of the forms and formats of
grassroots political action online.

[12] For a recent overview see
Shannon Mattern’s “Cloud
and Field”, Places Journal,
August 2016: https://
placesjournal.org/article/
cloud-and-field/

We have adopted the metaphor of the “field guide” in the
tradition of a number of recent guides which transpose the
language and imagery of mapping places, flora and fauna
onto the cloud, digital infrastructures and life online.[12]
However this metaphor stands in need of some qualification.
Many classical natural historical “field guides” aspire to
provide systematic taxonomies of natural phenomena by
taking them out of their contexts in order to abstract and
compare their features. By contrast with our guide we aim
to do precisely the opposite – not to decontextualise, but to
recontextualise fake news phenomena by suggesting ways
to follow them “in the wild”: as they travel across the web,
search engines, digital platforms, fact-checking initiatives
and news websites.
We do not set out to provide a definitive single set of
watercolour portraits, anatomical illustrations, cartographic
charts, satellite imagery or infrastructural diagrams of the
phenomenon in question – or even lists of characteristic
features which may be used for the purposes of identification.
Instead we illustrate a range of methods and procedures
which readers may use in order to explore fake news
phenomena online for themselves. As part of this process we
wish to extend the repertoire of mapping practices which are
publicly available to make sense of fake news online and in
this sense the graphics that we provide can be understood as
temporary placeholders to encourage further exploration.
We also draw attention to different ways of examining how

INTRODUCTION

9

[13] See Philip E. Agre, “Toward
a Critical Technical Practice:
Lessons Learned in Trying
to Reform AI”, in Geof
Bowker, Les Gasser, Leigh
Star and Bill Turner (eds),
“Bridging the Great Divide:
Social Science, Technical
Systems and Cooperative
Work", NJ: Erlbaum, 1997.
Available at: http://polaris.
gseis.ucla.edu/pagre/critical.html

[14] Shannon Mattern, “Cloud
and Field”, Places Journal,
August 2016: https://
placesjournal.org/article/
cloud-and-field/

[15] See, for example, the work
of Pamela Smith and her
colleagues on the “Making
and Knowing” project
at Columbia University:
http://recipes.hypotheses.
org/7430, February 2016,
and http://www.makingandknowing.org/

things are categorised and labelled as fake news and the politics
of these practices of classification. In this sense we hope to
cultivate what has been called “critical technical practice”[13]
– which in this case would include reflection on the use of
digital methods and digital data and how these not only serve
to designate phenomena which can be straightforwardly and
independently picked out, but how these very methods may
also be involved in the process of articulating what fake news
is. As Shannon Mattern puts it, in undertaking to investigate
fake news online we should be aware of “the shadows cast
by our presence as explorers in the field.”[14] And rather than
producing maps for the sake of producing maps, we should
consider what maps do, who and what they are for and
the effects that they produce as social, cultural and political
devices.
Insofar as we focus on providing procedures for inquiry
rather than pictures of the phenomena, this guide may
also be considered a kind of “recipe book.” Recent research
suggests that there is an interesting relation between the
documentation of recipes and the emergence of procedural
knowledge in the early modern period – such that practices
of writing down processes for cooking and craft are
entangled with the history of the emergence of scientific
method.[15] Over the past few decades the metaphors of the
“recipe” and the “cookbook” have also become popular in
relation to software programming. In our guide, we illustrate
different approaches to mapping and investigating fake news
online through a series of methodological “recipes.” As with
many cookery books, our aim is not just to support readers in
following the specific recipes that we present, but rather to
use these recipes to illustrate a certain approach to cooking –
with the hope that readers are inspired to adapt, modify and
venture beyond them. We also include a number of “serving
suggestions” about how they may be put to work.
We hope that the recipes in this guide will enrich
investigations of fake news and other fabrications in a way
which has affinities with a common narrative approach

10

A FIELD GUIDE TO FAKE NEWS

in mystery fiction – namely the scenario that in pursuit
of solving an apparently simple crime, the plot thickens,
the cast grows, the questions multiply and there are
unexpected twists or changes of perspective. By following the
production, circulation and responses to fake news online –
we may end up being drawn into things that we do not set
out to investigate: whether the media strategies of fake news
publishers, propagandists, trolls or bots; the commercial and
technical architectures of online content; the politics and
dynamics of viral content; and how social life adapts, evolves
and innovates in response to some of the world’s biggest
online platforms and websites. In this sense, it will be clear
that fake news involves more than a few rogue producers
or state conspiracies – and raises important and difficult
questions about the role of digital technologies in society and
how we mutually shape and are shaped by them.

[16] Edgar Allan Poe and Jacob
Schwartz, “The Purloined
Letter”, London: Ulysses
bookshop, 1931.

[17] See, Noortje Marres, “The
Redistribution of Methods:
On Intervention in Digital
Social Research, Broadly
Conceived”, The Sociological Review, June 2012;
and Mike Ananny and Kate
Crawford, “Seeing Without
Knowing: Limitations of the
Transparency Ideal and its
Application to Algorithmic
Accountability”, New Media
and Society, December
2016.
[18] Liliana Bounegru, Mette
Simonsen Abildgaard,
Andreas Birkbak, Jonathan
Gray, Mathieu Jacomy,
Torben Elgaard Jensen,
Anders Koed Madsen and
Anders Kristian Munk, "Five
Provocations about Fake
News" (under review).
INTRODUCTION

In Edgar Allan Poe’s classic mystery story “The Purloined
Letter”[16], the prefect of police – “G” – and his colleagues
search for a letter said to contain scandalous information
behind wallpaper, under carpets, in the legs of furniture and
in cushions, only to eventually find the letter “hiding in plain
sight”. In a similar vein, we may consider the algorithmically
mediated circulation of fake news on digital platforms in
terms of what Noortje Marres characterises as “distributed
accomplishment” or what Mike Ananny and Kate Crawford
describe as “relational achievement”.[17] This entails a shift
from “seeing in” systems as a kind of looking “under the
hood”, to “seeing across” a diverse range elements which are
implicated in the patterning of collective life online.
Many of the researchers who have contributed to the guide
share a background in a field called Science and Technology
Studies (STS). Some of the lines of inquiry pursued in the
guide are informed by a forthcoming paper exploring what
STS can bring to the study of fake news.[18] The recipes are
also informed by a “digital methods” research approach that
has developed through an engagement with this field and
which many of us have contributed to through our teaching
11

[19] See, for example, Richard
Rogers, “Digital Methods”,
2013, Cambridge, MA: MIT
Press.
[20] See, for example, Noortje
Marres, “Material Participation: Technology, the
Environment and Everyday
Publics”, London: Palgrave
Macmillan, 2012.

[21] See Tommaso Venturini,
Anders Munk and Axel
Meunier, “Data-Sprint: A
Public Approach to Digital
Research” in C. Lury, P.
Clough, M. Michael, R.
Fensham, S. Lammes, A.
Last, & E. Uprichard (Eds.)
“Interdisciplinary Research
Methods”, London: Routledge, 2017.
[22] Jonathan Gray, Liliana
Bounegru and Lucy Chambers (Eds.) “The Data Journalism Handbook”, Sebastopol, CA: O’Reilly Media,
2012, available at: http://
datajournalismhandbook.
org/
[23] Upon being invited to
become Director of the Argentine National Library at
a moment which coincided
with the deterioration of his
eyesight, Borges famously
wrote: “No one should
read self-pity or reproach
/ into this statement of
the majesty / of God; who
with such splendid irony
/ Granted me books and
night at one touch”. See
J. L. Borges, “Poem of the
Gifts” in “Selected Poems:
Volume 2”, London: Penguin Books, 2000, p. 95.
12

and research.[19] We also draw upon our field’s interest
in public engagement and participation around digital
technologies and data infrastructures.[20] As such our focus is
less on advancing particular legal or technical fixes, than on
facilitating processes of public engagement and democratic
deliberation – including provoking curiosity about different
ways of seeing the issue and imagination about the different
ways in which we might respond.
The material in this guide has been produced through
a series of “data sprints” and research workshops in
Amsterdam, Copenhagen and Milan, hosted by members
of the Public Data Lab. The “data sprint” is a short form
working format that has emerged at the intersection
between Science and Technology Studies and New Media
Studies, drawing on approaches associated with open-source
software development, open data and civic hacking in order
to convene a range of actors to collaborate around the coproduction of data and research projects – including between
fields of practice with different outlooks.[21]
Two of us have a background in data journalism, having
co-edited The Data Journalism Handbook and undertaken
various initiatives in this field.[22] This guide builds on a
long-standing interest in supporting productive encounters
between data journalists and digital researchers. While fake
news seems like a remarkably ripe area for experimentation
between these two fields, just as the writer Jorge Luis Borges
lamented being granted “books and night at one touch”[23] it
is not without a sense of irony that we note that as public
attention around this issue grows, fake news websites are
beginning to vanish – leading to proposals for a “fake news
archive” amongst our contributing researchers. Happily the
approaches and analytical techniques in this guide may be
used to inform collaborations between data journalists and
digital researchers around the study of other contentious
issues and controversies as they unfold on digital media, as
well as of the mediating capacities of platforms, algorithms
and infrastructures which shape life online.
A FIELD GUIDE TO FAKE NEWS

The data sprint format has also helped us to catalyse new
experimentation and empirical work in a comparatively
short period of time – a distinct advantage given the
pace of developments around fake news. For this we are
immensely grateful to researchers, graduates and students
at DensityDesign Lab (Politecnico di Milano, Italy), the
Digital Methods Initiative (University of Amsterdam,
Netherlands), the European Journalism Centre, the
Laboratoire Interdisciplinaire Sciences Innovations Sociétés
(Université Paris-Est, France), the médialab (Sciences Po,
Paris, France), the Media of Cooperation research group
(University of Siegen, Germany), the STS-Lab (University
of Lausanne, Switzerland) and the Techno-Anthropology
Lab (Aalborg University Copenhagen, Denmark) – without
whose energy, creativity and dedication this project would
not have been possible.

Jonathan Gray (@jwyg), Liliana Bounegru (@bb_liliana),
Tommaso Venturini (@TommasoVenturin)
London, March 2017

INTRODUCTION

13

CONVENTIONS
USED IN THE BOOK
In this book we use the  (eye) symbol to indicate visual
results, the  (wrench) to point to the tools glossary and
the → (arrow) to point to the concepts glossary. To avoid
distracting our readers we only use the glossaries icons to
mark the first occurrence per recipe of the term or tool
explained in glossaries.
Furthermore, each recipe in our chapters is introduced by a
diagram, or a method map, representing the key analytical
steps taken to arrive at our results. In each method map,
arrows represent actions and icons represents their results.
You can see the steps in the method maps as possible
ingredients for your own recipes.
Some recipes lead to multiple outcomes. When this is the
case you will find at the beginning of the recipe a complete
method map for the entire recipe (on a blue background),
and the parts relevant to each individual step in the recipe
highlighted on a white background at the beginning of each
recipe alongside the description of the relevant step.
Below you can find a list of all the icons we use for the
methods maps.

A dataset in the
form of a table.

14

Any kind of visualization,
such as a bubble graph or
a network diagram. See
the Concepts Glossary for
the full list of visual models
used in this guide.

A list. This could be,
for example, a list of
websites, or a list of
Facebook pages.

A FIELD GUIDE TO FAKE NEWS

A screenshoot. Usually
taken from a web browser
with the aim of preserving
a snapshot of a web page.

A corpus of images. A
set of images captured
with the same method.

Temporal information.
Could be, for example,
the creation time of a
Facebook post.

Hashtag. Used in
many social networks,
for example in Twitter
and Facebook.

User profile. It represents
all the information
related to a user in a
social network. For
example in Twitter the
user profile contains the
@name, the description,
the profile picture.

ACTIONS
Sometimes, relevant actions have their own icons. Below you
can find the full list of them.

Automatic operations.
Used to highlight when
an action (e.g. dividing
items into categories)
is performed by a
machine.

Manual operations.
Used to highlight when
an action (e.g. dividing
items into categories) is
performed by a human
being.

Image comparison.
It is used to highlight
when the analyst must
visually compare a
corpus of images.

+

Union of lists. When
two or more lists are
merged into one.

INTRODUCTION

15

16

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

Chapter 1

MAPPING
FAKE NEWS HOTSPOTS
ON FACEBOOK
What publics does fake news
animate on Facebook?
How may the trajectory of a
fake news story be traced on
Facebook?
Do fact-checking initiatives reach
the publics of fake news on
Facebook?

Introduction - This section provides a set
of recipes for tracing the circulation of fake
news on Facebook. The focus is on circulation
because false and misleading knowledge claims
are not born “fake news”. To become fake
news they need to mobilise a large number of
publics – including witnesses, allies, likes and
shares, as well as opponents to contest, flag
and debunk them. Facebook’s architecture
poses challenges to the study of circulation
of content due to the nature of its access and
permissions system. Hence we focus on tracing
the publics of fake news through its most
publicly accessible entities: pages and groups,
which may be considered to constitute already
assembled publics.
Around the 2016 US presidential elections
commentators have noted the emergence of
a Facebook-native, hyper-partisan “political
media machine” that was highly effective in
gathering large numbers of → followers and
18

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

generating → engagement [1].
This fake news dissemination
machine and responses to it, is
what the recipes in this section
enable to explore. The first two
recipes focus on mapping the publics that are
energised by fake news on Facebook, as well as
the trajectories through which fake news stories
travel on Facebook. The third recipe provides
an approach to address the effectiveness of factchecking initiatives in reaching the publics of
fake news on Facebook. Through these recipes
we aim to gesture towards different ways of
providing “thicker” accounts of circulation and
engagement around fake news on social media
beyond the “thin descriptions” of aggregated
counts and metrics.
[1] See, John Herrman, “Inside
Facebook’s (Totally Insane,
Unintentionally Gigantic,
Hyperpartisan) Political-Media Machine”, Agust
2016, The New York Times:
https://www.nytimes.
com/2016/08/28/magazine/inside-facebooks-totally-insane-unintentionally-gigantic-hyperpartisan-political-media-machine.html

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

19

CHAPTER 1 → RECIPE 1

WHAT PUBLICS
DOES FAKE NEWS
ANIMATE ON FACEBOOK?

BEFORE STARTING

The starting point for this recipe is a list of
fake news stories. There are different ways of
obtaining these lists – including starting with
existing lists as well as creating your own. To
illustrate this recipe we use an already existing
list of 22 fake news stories about various political
issues pertaining to the 2016 presidential
elections in the US that generated most
→ engagement on Facebook. These were
identified by BuzzFeed News.
The recipe comprises of four steps. We start by
identifying the themes that are exploited in our
set of stories as well as the key political events
which they editorialise (a ). Next we identify
the most prominent public Facebook pages and
groups that share these stories (b ). We also
explore whether certain publics have preferred
story themes (c) and profile the publics that
are energised by fake news stories about the US
elections (d ).

20

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

START
input URLs in

list of 22 URLs of political fake
news stories
Source: BuzzFeed News

CrowdTangle

output data
> Fake news story URLs
> Facebook pages and groups
that share the URLs
> Number of interactions
per each page or group
> Date of sharing
of the story
identify time intervals with
highest frequency of publication
of fake news stories

a
Google News

visualise

identify key related
events with

WHICH MEDIA AND POLITICAL
EVENTS ARE SUCCESSFUL IN
SETTING THE FAKE NEWS
AGENDA?

visualise

RAWGraphs

WHICH FACEBOOK PAGES
AND GROUPS PROMOTED THE
HIGHEST NUMBER OF FAKE
NEWS STORIES?
c

Gephi

visualise

import data in

import data in

b

DO FACEBOOK PUBLICS HAVE
PREFERRED STORY THEMES?

manually categorise
Facebook pages

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

visualise

d

WHAT KINDS OF PUBLICS ARE
ENERGISED BY FAKE NEWS?
21

START
list of 22 URLs of political fake
news stories
Source: BuzzFeed News

input URLs in

CrowdTangle

output data
> Fake news story URLs
> Facebook pages and groups
that share the URLs
> Number of interactions
per each page or group
> Date of sharing
of the story
identify time intervals with
highest frequency of publication
of fake news stories

a
Google News

22

visualise

identify key related
events with

WHICH MEDIA AND POLITICAL
EVENTS ARE SUCCESSFUL IN
SETTING THE FAKE NEWS
AGENDA?

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 1 → RECIPE 1

a. EXAMINE THE THEMES EXPLOITED
IN FAKE NEWS STORIES AND
IDENTIFY THE EVENTS WHICH THEY
EDITORIALISE

This analysis may be done by qualitatively
examining the content of each article and
identifying key political or media events
related to the issues exploited in the articles,
which occurred around the publication date
of each story. This is done to enable a better
understanding of the issues that animate the
publics that circulate fake news.
◊◊ If the content of the fake news article is no
longer available on its original URL you
may use the Internet Archive’s  WayBack
Machine to check whether an archived
version of the URL is available.
◊◊ To identify key events occurring around
the dates of publication of the stories which
are related to the themes exploited in the
stories you may use a news aggregator such
as  Google News Search as well as news
article archives.
◊◊ An annotated timeline of stories and
relevant events occurring around the same
dates might provide a starting point for
reflection about the relationship between
political and media events and fake news
stories.

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

23

WHICH MEDIA AND
POLITICAL EVENTS ARE
SUCCESSFUL IN SETTING
THE FAKE NEWS AGENDA?

Afric
Any
D
3 October
ABCNEWS.COM.CO

Actor Bill Murray Announces 2016
Presidential Run

11 Octo

PACIFISM

WORLDNEWSDAILY

Timeline of best performing fake
news stories about the US elections
on Facebook in 2016 and events they
editorialise. Successful fake news stories
appear to exploit populist themes such as
anti-establishment sentiment, nationalist
and anti-immigration sentiment as well
as perceived or projected weaknesses
of political candidates such as misogyny
and corruption. A number of events at
the end of July, mid-October and early
November are successful in setting the
fake news “agenda”.

ISIS Leader Calls fo
Muslim Voters to Su
Clinto

ANTI-MUSLIM SE

7 September
BURRARDSTREETJOURNAL.COM

R

President Obama Confirms He Will
Refuse To Leave Office If Trump Is
Elected
PRESIDENTIAL RACE

18 September
POLITICOPS.COM

Mike Pence: “Sarah Palin Is My Role
Model For Beautiful, Smart American
Women” - Newslo

6 July

GENDER DISCRIMINATION

CHRISTIAN TIMES

ANTI-CLINTON

26 Septem

ENDING THE

BREAKING Romanian Hacker With
Access To Clinton Emails Found
Dead In Jail Cell

Pope Francis Shoc
Endorses Donald T
President, Releases

23 July

CORRUPTION OF THE POLITIC

BIZSTANDARDNEWS.COM

8 March

Graham Says Christians Must
Support Trump or Face Death Camps

REACT365.COM

Obama passed law for grandparents
to get all their grandchildren every
weekend

ANTI-DEMOCRAT

25 July
KYPO6.COM

Pope Francis Shocks World,
Endorses Hillary Clinton for
President, Releases Statement

N/A

11 March
HEAVIERMETAL.NET

RAGE AGAINST THE MACHINE To
Reunite And Release Anti Donald
Trump Album

ANTI-TRUMP

T
H

ANTI-TRUMP

January

February

Donald Trump accepts nomination

March

23 Jul

Wikileaks reveals Democratic Party 25 Jul
has a bias against Bernie Sanders
Hillary Clinton accepts nomination

April

May

Trump attacks "
states Clinton
and media

The M
Clin

28 Jul

Trump Says Se

Mich
Trum

Nin
Inappropriatel

24

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

3 November

EMPIREHERALD.COM

can Billionaire Will Give $1 Million To
yone Who Wants To Leave America if
Donald Trump is Elected President
DISCRIMINATION AGAINST MINORITIES

5 November
THELASTLINEOFDEFENSE.ORG

Van Full Of Illegals Shows Up To Vote
Clinton At SIX Polling Places, Still
Think Voter Fraud Is A Myth?

ober

YREPORT.COM

11 December

VOTER FRAUD, ANTI-IMMIGRATION SENTIMENT

or American
upport Hillary
on

ABCNEWS.COM.CO

Obama Signs Executive Order
Banning The Pledge Of Allegiance In
Schools Nationwide

5 November
THELASTLINEOFDEFENSE.ORG

ENTIMENT

WHOA! Hillary Caught On Hot Mic
Trashing Beyonce’ With RACIAL SLURS!

14 October
WORLDNEWSDAILYREPORT.COM

ANTI-CLINTON

RUPAUL CLAIMS TRUMP TOUCHED
HIM INAPPROPRIATELY IN THE
1990S

NATIONALIST SENTIMENT

12 December
5 November

ABCNEWS.COM.CO

DENVER GUARDIAN

Obama Signs Executive Order Declaring
Investigation Into Election Results;
Revote Planned For Dec. 19th

MISOGYNY

15 October

CORRUPTION OF THE POLITICAL ESTABLISHMENT

USANEWSFLASH.COM

FBI Agent Suspected in Hillary Email
Leaks Found Dead in Apparent
Murder-Suicide

11 November
TMZHIPHOP.COM

CORRUPTION OF THE POLITICAL ESTABLISHMENT

mber

17 October

E FED

CONSERVATIVESTATE.COM

cks World,
Trump for
s Statement

Trump Offering Free One-Way
Tickets to Africa & Mexico for Those
Who Wanna Leave America
ANTI-IMMIGRANTION SENTIMENT

Hillary Clinton In 2013: “I Would Like To See
People Like Donald Trump Run For Office;
They’re Honest And Can’t Be Bought”

CAL ESTABLISHMENT

CORRUPTION OF POLITICAL ESTABLISHMENT

Fake news stories

FBI Agent Suspected in Hillary Email
Leaks Found Dead in Apparent
Murder-Suicide

22 November
ABCNEWS.COM.CO

Donald Trump Protester Speaks Out: I
Was Paid $3,500 To Protest Trumps Rally

PRO-ENTREPRENEURIALIST SENTIMENT

ANTI-CLINTON

29 July
BURRARDSTREETJOURNAL.COM

Trump Claims America Should Never
Have Given Canada Its Independence
NATIONALIST SENTIMENT

June

July

August

"corrupt establishment",
n should be "locked up" 13 Oct
a is in "war against him"
–Vox

Most Explosive WikiLeaks
nton Revelations (So Far)
–Breitbart

helle Obama denounces
mp's comment on women
–NY Times

October

November

December

Beyonce, Jay Z & Clinton share a stage 5 Nov
–NY Times
No charge to Hillary Cinton and no further investigation over email scandal
6 Nov
–FBI Director Comey
Arizona Anti-Immigrant Sheriff to Deploy Deputies at Polling Places
–The Guardian

Events

exual Assault Allegations
14 Oct
are "Made-Up Stories"
–IB Times

September

Donald Trump continues to accuse Democrats of voter fraud.
On Saturday 5th November, Trump claimed the late opening of 7 Nov
a voting site in a Latino neighorhood of Las Vegas due
to long lines pointed to a "rigged system."
–Democracy Now

nth Woman Says Trump
15 Oct
ly Groped or Kissed Her
–The Guardian

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

Arizona Anti-Immigrant Sheriff
11 Nov
to Deploy Deputies at Polling Places
–The Guardian

25

START
list of 22 URLs of political fake
news stories
Source: BuzzFeed News

input URLs in

CrowdTangle

output data
> Fake news story URLs
> Facebook pages and groups
that share the URLs
> Number of interactions
per each page or group
> Date of sharing
of the story

26

RAWGraphs

visualise

import data in

b

WHICH FACEBOOK PAGES
AND GROUPS PROMOTED THE
HIGHEST NUMBER OF FAKE
NEWS STORIES?

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 1 → RECIPE 1

b. IDENTIFY THE FACEBOOK PAGES AND
GROUPS THAT SHARE THESE STORIES

This may be done with a social media
monitoring tool such as the browser extension
of  CrowdTangle. The number of followers per
page or group as well as the number of
→ interactions per posts should be recorded in
a spreadsheet alongside the names of pages and
groups that share fake news stories.
◊◊ Please note that a fake news story may be
reposted on a number of different websites.
For this reason a methodological decision
needs to be taken from the outset as to
whether only the pages and groups that
share the original URL of the story will be
recorded or whether all pages and groups
that share all versions of the fake news
story will be collected.
◊◊ You may want to take note of the pages or
groups which shared the highest number
of fake news stories as well as the total
number of interactions generated by each
group or page.
◊◊ If you use  CrowdTangle please note that
for Facebook the tool returns the top 500
most popular public posts to verified pages
as well as to pages with more than 125.000
fans.[1]
◊◊ You may use a → circle packing
visualisation to represent the pages and
groups that share fake news items as well
as the number of stories which they share
and the number of interactions which they
generate. You may use  RAWGraphs for
this operation.

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

[1]
See, CrowdTangle’s “Frequently
Asked Questions”, available
at: https://apps.
crowdtangle.
com/chrome-extension/faq

27

WHICH FACEBOOK
PAGES AND GROUPS
PROMOTED THE HIGHEST
NUMBER OF FAKE NEWS
STORIES? WHICH ONES
CREATED THE HIGHEST
ENGAGEMENT?
Public Facebook pages and groups that
share fake news items, sized according
to the number of items they share and
coloured according to their number of
followers. Each page can share the same
item more than once. The pages and
groups that share the highest number
of stories are primarily pro-Trump
supporters and anti-Hillary groups. The
page that generates the highest number
of interactions with fake news stories is
the fan page dedicated to republican TV
commentator, Jeanine Pirro.

Women SCOUT

THE TRU

TRUMP - SP
AGAINST ISL
OF AM

TRUMP FORCE ONE
TRUMP OF CHAMPIONS

Newslo

The Australian Tea Party
F@CK YOU
HILLARY CLINTONDemocratic
WE HATE YOU!
News
and Events

The Burrard Street Journal

Indies 4 Trump
The Church
of PUNK

All Thing
#trumpvict
#neverhill
Deplorables for TRUMP
Chonda Pierce

The Empire Herald

FU Trump

One Nation Under God

AMOUNT OF INTERACTIONS
1

Citizens
For Trump

Judge Jeanine
Pirro has Fans

0

Don P

73,732

FAKE NEWS POSTED
1

Females
for TRUMP

30

The Original Wake
Up America

Freedom Don't
Come Free
& Devildoc

PATRIOT
AMERICANS
FOR DONALD TRUMP
The Resistance:
The Last Line
Of Defense
Patriots for a
Free Republic Trump Squad

Friends
Like Donald

Politicked
The Trump
American Party

28

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

TS for TRUMP (c)
We Endorse
Donald J. Trump
Washington State for
THE TRUMPERS!!!
Donald Trump 2016

UMPS STORM GROUP

WOMEN4TRUMP

PEAK OUT
LAMIZATION
MERICA

NO Hillary for 2016

NEVER
HILLARY
CLINTON
Hillary Clinton
vote blue

Sicilians For
Donald Trump
Veterans for Donald Trump

NEVER HILLARY

Hillary Clinton in 2020

DONALD TRUMP
COMMANDER IN CHIEF !!!!

Hillary Clinton Revolution

VOTE TRUMP ONLY THE AMERICAN PARTY RISING

Donald Trump for President!
D.L. Hughley
Rock Feed

Donald Trump for
President Connecticut
Anonymous The
World is Ours

gs
tory
lary

Resist Donald J. Trumpf

Donald Trump
The Political Movement

1 Million Strong For
Hillary Clinton in 2016

Conservative
Veterans
for America

Being Liberal
Means Being
a Hypocrite 2 Colorado

VETERANS FOR PRESIDENT
DONALD J TRUMP 2016

Donald Trump
Criminalize VS. Hillary
Conservatism Clinton

Donald
Trump
President

Donald Trump For President

Make America
Great Again

UNITE TO FIGHT
BRING AMERICA
BACK !!! TRUMP 2016

for Donald
Trump 2016

Donald Trump
Donald J. Trump Donald Trump Commander
for PRESIDENT America's in Chief 2020
Facebook Group President (c)

HILLARY IN
2016 & 2020

Republican
Patriotic
Party

MY VOTE'S FOR
Reclaim
TRUMP
The Deplorables America

Who
J. Trump
TrumpNation
REPUBLICANS - CONSERVATIVES TEA PARTY PATRIOTSRIGHT WING AMERICANS

d
Tomorrow's
News Today.

Trump America

Trump
Train

Trump USA's CEO

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

29

START
input URLs in

list of 22 URLs of political fake
news stories
Source: BuzzFeed News

CrowdTangle

output data

30

c
Gephi

visualise

import data in

> Fake news story URLs
> Facebook pages and groups
that share the URLs
> Number of interactions
per each page or group
> Date of sharing
of the story

DO FACEBOOK PUBLICS HAVE
PREFERRED STORY THEMES?

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 1 → RECIPE 1

c. IDENTIFY WHETHER FACEBOOK
PUBLICS HAVE PREFERRED STORY
THEMES

To explore whether particular story themes
assemble publics and to qualitatively profile
those publics based on the stories that animate
them you may conduct a network analysis of
public Facebook pages and groups connected by
the stories which they share.
◊◊ Starting from the dataset extracted with 
CrowdTangle’s browser extension, you may
create a network file where each time a
Facebook group or page posts a fake news
story a link is established between that page
or group and that story.
◊◊ You may use  Table2Net to convert your
CSV (comma-separated values) file into a
network file and  Gephi to explore the
network. A force-directed layout algorithm
such as ForceAtlas2[2] can help you visualise
the outcomes.
◊◊ Identify which stories are most successful
in energising publics as well as whether
publics have preferred story themes.

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

[2]
See, Mathieu
Jacomy, Tommaso
Venturini, Sebastien Heymann and
Mathieu Bastian,
“ForceAtlas2, a
Continuous Graph
Layout Algorithm
for Handy Network
Visualization Designed for the Gephi Software”, June
2014, PLoS ONE:
http://journals.
plos.org/plosone/
article?id=10.1371/
journal.
pone.0098679
31

DO FACEBOOK PUBLICS
HAVE PREFERRED
STORY THEMES?
Network of public Facebook pages and
groups connected by the fake news
stories which they share. Notable is the
core of the network which consists of a
series of pages and groups associated
with Trump supporters which are
animated by anti-Hillary stories.

T
H

Hillary Clin
Revolutio

J

Reclaim America

M
One Nation
Under God

America's Veterans
are Loved

I
FAKE
NEWS URL

AMOUNT OF
INTERACTIONS
0

98,032

D

NO H
for 2

N° OF PAGES SHARING
A NEWS ITEM

1

125

The Trump America

FACEBOOK
PAGE OR
GROUP
0

32

VOTE T
THEA
PART

AMOUNT OF
INTERACTIONS

65,649

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

K

Fake News Headlines

N

A FBI Agent Suspected in Hillary Email
Leaks Found Dead in Apparent
Murder-Suicide
B Hillary Clinton In 2013: “I Would Like
To See People Like Donald Trump Run
For Office; They’re Honest And Can’t
Be Bought”

L
D.L. Hughley

C ISIS Leader Calls for American Muslim
Voters to Support Hillary Clinton

P

D Donald Trump Protester Speaks Out:
"I Was Paid $3,500 To Protest
Trump's Rally"

O

E Obama Signs Executive Order
Declaring Investigation Into Election
Results; Revote Planned For Dec. 19th

R

The Burrard
Street Journal

F WHOA! Hillary Caught On Hot Mic
Trashing Beyonce’ With RACIAL
SLURS!
G Van Full Of Illegals Shows Up To Vote
Clinton At SIX Polling Places, Still
Think Voter Fraud Is A Myth?

nton
on
Colorado for Donald
Trump 2016

E
U

Donald Trump
Commander
in Chief 2020

F

G
F@CK YOU HILLARY
CLINTON WE HATE YOU!

Hillary
2016

an Party

TRUMP ONLY
AMERICAN
TY RISING

I Obama Signs Executive Order Banning
The Pledge Of Allegiance In Schools
Nationwide
J BREAKING Romanian Hacker With
Access To Clinton Emails Found Dead
In Jail Cell

C

Q

S

The Resistance: The Last
Line Of Defense

H RAGE AGAINST THE MACHINE To
Reunite And Release Anti Donald
Trump Album

L Pope Francis Shocks World, Endorses
Hillary Clinton for President, Releases
Statement
M Pope Francis Shocks World, Endorses
Donald Trump for President, Releases
Statement

Stop the Mosque
in Bendigo

TrumpNation

K Actor Bill Murray Announces 2016
Presidential Run

N Trump Claims America Should Never
Have Given Canada Its Independence

Donald Trump
For President

O Mike Pence: “Sarah Palin Is My Role
Model For Beautiful, Smart American
Women” - Newslo
P RUPAUL CLAIMS TRUMP TOUCHED
HIM INAPPROPRIATELY IN THE
1990S

B

Q President Obama Confirms He Will
Refuse To Leave Office If Trump Is
Elected

A

TRUMP
FORCE ONE

Make America
Great Again

TRUMP - SPEAK OUT AGAINST
ISLAMIZATION OF AMERICA

R Graham Says Christians Must Support
Trump or Face Death Camps

Donald Trump
for President

S African Billionaire Will Give $1 Million
To Anyone Who Wants To Leave
America if Donald Trump is Elected
President
T Trump Offering Free One-Way Tickets
to Africa & Mexico for Those
Who Wanna Leave America
U Obama passed law for grandparents to
get all their grandchildren every
weekend

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

33

START
list of 22 URLs of political fake
news stories
Source: BuzzFeed News

input URLs in

CrowdTangle

output data
> Fake news story URLs
> Facebook pages and groups
that share the URLs
> Number of interactions
per each page or group
> Date of sharing
of the story

manually categorise
Facebook pages

34

visualise

d

WHAT KINDS OF PUBLICS ARE
ENERGISED BY FAKE NEWS?
A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 1 → RECIPE 1

d. PROFILE THE PUBLICS ANIMATED BY
FAKE NEWS

This may be done by conducting a qualitative
analysis of all public Facebook pages that share
fake news items based on self-descriptions
available on their “About” pages.
◊◊ You may take an → emergent coding
approach to identify the themes that
emerge from the description of pages.
You may take note of a more generic
category (e.g. “grass-roots activism”) as
well as a more specific one (e.g. “antiestablishment”).
◊◊ Sum up the amount of followers across all
pages belonging to the same category.
◊◊ A → treemap visualisation may be used to
represent the weight and hierarchy of each
category. You may use  RAWGraphs for
this operation.

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

35

WHAT KINDS
OF PUBLICS ARE
ENERGISED BY FAKE
NEWS?
Types of Facebook publics animated
by fake news, according to a manual
classification of pages that share fake
news items. Notable are grassroots
activists for a variety of issues, political
candidate loyalists as well as entertainers.

NEWS

WORKERS
Truckers

Small
Bussiness

Steelworkers

News

Trending Liberal
News
News

Page
Dedicated
to the
Philipines

Critical
News

Liberal
news

Con
New

OTHER
Italian
Left

Climate
Skeptics

SOCIAL CHANGE
Pro Latino

Human
rights

Progessives

News

News / Repu
News
New
Aggregators

News
Aggregator

Alternativ
News

Against Gun
Violence
Non-PC News

EARLY CAMPAIGNERS
Early 2020
Early 2020 Early 2020
Bernie
Hillary
Trump
Campaigners Campaigners Campaigners
Hillary Loyalists
Bernie Loyalists

DISCUSSION SPACE
Political
Commentary

Anti
establishment

Politcal
Commentary
Anti-Democrat

BUSINESS
Punk Tattoo

Fashion

HIERARCY
Bussiness

CATEGORY
Sub-category

Motivational
Speaker

Dating app

FACEBOOK PAGES
PER CATEGORY

Spritual
Community
Progessive
Community
Survivalist

2
3

36

Republic

Pro-Black

OCCUPY MOVEMENT
1

Democr

WOMEN-FOCUSED

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

Religiou

GRASSROOTS ACTIVISTS/ CITIZEN VIGILANTES

nvervative
ws

Againt
Pro
Ted Cruz Bernie

For
Against
Disaster
Islam
Preparness

Artist

Against Pro Latino
Financial
Terrorism

Actor

ublican
ws

ve

For Free
Society

PUBLIC FIGURE

For Liberty
and Property

White For
Lives Information
Matter Freedom

Against Trump Against
Republicans

Againt
Gun
Owners
Against
Mainstream
Media
Against Gun
Control
Advocates

ENTERTAINERS
Anti-conservatism

Porn
Actress

Republican
Politician

Journalist

US Patriots

Anonymous

Against
Corruption
Pro-Trump

LOYALISTS
Trump Loyalists

MUSIC/ENTERTAIMENT
COMMUNITY

rat Community

Pro Military

can Community

us

CLICKBAIT

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

37

3 November

EMPIREHERALD.COM

African Billionaire Will Give $1 Million To
Anyone Who Wants To Leave America if
Donald Trump is Elected President

Women SCOUTS for TRUMP (c)

DISCRIMINATION AGAINST MINORITIES

3 October

5 November

ABCNEWS.COM.CO

Van Full Of Illegals Shows Up To Vote
Clinton At SIX Polling Places, Still
Think Voter Fraud Is A Myth?

11 October

PACIFISM

WORLDNEWSDAILYREPORT.COM

7 September

WORLDNEWSDAILYREPORT.COM

FBI Agent Suspected in Hillary Email
Leaks Found Dead in Apparent
Murder-Suicide

GENDER DISCRIMINATION

CHRISTIAN TIMES

T

CORRUPTION OF THE POLITICAL ESTABLISHMENT

PRO-ENTREPRENEURIALIST SENTIMENTThe Burrard Street Journal

11 March
HEAVIERMETAL.NET

N

NATIONALIST SENTIMENT

March

April

May

1

June

0

23 Jul

P

July

August

D.L. Hughley
September

1

28 Jul

October

Females
for TRUMP

November
Judge
Jeanine
Pirro has Fans

Beyonce, Jay Z & Clinton share a stage 5 Nov
The Original Wake
–NY Times

O

No charge to Hillary Cinton and no further investigation over email scandal
6 Nov
–FBI Director Comey

On Saturday 5th November, Trump claimed the late opening of
R
The Burrard
a voting site in a Latino neighorhood of Las Vegas due
to long lines pointed to a "rigged system."
Street Journal
Donald Trump continues to accuse Democrats of voter fraud.

7 Nov

Politicked

Hillary Clinton
Revolution

Reclaim America

I

The Resistance: The Last
Line Of Defense

Page
Dedicated
to the
Philipines

Critical
News

Human
rights

Liberal
news

News / Republican
News
News
Aggregators

S

SOCIAL CHANGE
Pro Latino

News News
Aggregator

Progessives

Alternative
News

Donald Trump
Commander
in Chief 2020

U

Donald Trump
Commentary
For President

B

OF
TIONS

A

CATEGORY
Sub-category

FACEBOOK PAGES
PER CATEGORY
TRUMP - SPEAK OUT AGAINST
ISLAMIZATION OF AMERICA

65,649

1

Punk Tattoo

Fashion

Bussiness

Motivational
Speaker

38

TRUMP
FORCE ONE

Make America
Great Again

Dating app

Against Trump Against

HandRAGE
THE MACHINE ToRepublicans
Property AGAINST
Lives Information
Matter Freedom
Reunite And
Release Anti Donald
Againt
Trump Album
Gun
I Owners
Obama Signs Executive Order Banning
Against
The Pledge Of Allegiance In Schools
Mainstream
US Patriots
NationwideAnti-conservatism
Media

P RUPAUL CLAIMS TRUMP TOUCHED
HIM INAPPROPRIATELY IN THE
1990S

ENTERTAINERS

Porn
Actress

Republican
Politician

Journalist

Q President Obama Confirms He Will

Donald Trump
for President
OCCUPY MOVEMENT

WOMEN-FOCUSED

MUSIC/ENTERTAIMENT

COMMUNITY
Refuse To Leave Office If Trump Is
Spritual
Community

Democrat Community

Elected

Pro Military

R Graham Says Christians Must Support
Trump or Face Death Camps

Progessive
Community
Survivalist

2
3

PUBLIC FIGURE

Financial

O Mike Pence: “Sarah Palin Is My Role
Model For Beautiful, Smart American
Women” - Newslo

BUSINESS

HIERARCY

VOTE TRUMP ONLY
THEAMERICAN
PARTY RISING

White For

Islam

N Trump Claims America Should Never
Have Given Canada Its Independence

Anti
establishment

Politcal
Commentary
Anti-Democrat

The Trump American Party

For Liberty

Disaster

Trump Loyalists
M Pope
Francis Shocks World, Endorses
Donald Trump for President, Releases
Statement

Bernie Loyalists

DISCUSSION SPACE

TrumpNation

Ted Cruz Bernie

L Pope Francis Shocks World, Endorses
Hillary Clinton for President, Releases
LOYALISTS
Statement

Hillary Loyalists

Political

NO Hillary
for 2016

Actor

Society

K Actor Bill Murray Announces 2016
Presidential Run
Pro-Trump

Non-PC News

EARLY CAMPAIGNERS

Stop the Mosque
in Bendigo

D

Artist

G Van Full Of IllegalsPreparness
Shows Up ToTerrorism
Vote
Clinton At SIX Polling Places, Still
Think Voter Fraud Is A Myth?

Gun
JAgainst
BREAKING
Romanian Hacker With
Control
Access To Anonymous
Clinton Emails Found Dead
Advocates
In Jail Cell
Against

Early 2020 Early 2020 Early 2020
Bernie
Hillary
Trump
Campaigners Campaigners Campaigners

F

G

F WHOA! Hillary Caught On Hot Mic
GRASSROOTS ACTIVISTS/ CITIZEN VIGILANTES
Trashing
Beyonce’
Trump
USA's CEOWith RACIAL
Trump
Train Againt
ForSLURS!
Free
For
Against Against Pro Latino
Pro

Corruption

E

F@CK YOU HILLARY
CLINTON WE HATE YOU!

98,032

Climate
Skeptics

Against Gun
Violence

C

Q

OF
TIONS

NEWS

Trump America
News
Trending Liberal Convervative
Tomorrow's
News
News
News Today. News

OTHER

M
One Nation
Under God

WORKERS

Italian
Left

Colorado for Donald
Trump 2016

America's Veterans
are Loved

REPUBLICANS - CONSERVATIVES TEA PARTY PATRIOTSRIGHT WING AMERICANS

Trump
TruckersThe Small
Steelworkers
American
Party
Bussiness

–Democracy Now

Arizona Anti-Immigrant Sheriff
11 Nov
to Deploy Deputies at Polling Places
–The Guardian

Ninth Woman Says Trump
15 Oct
Inappropriately Groped or Kissed Her
–The Guardian

MY VOTE'S
FOR
D Donald Trump
Protester
Speaks Out:
Reclaim
TRUMP
HILLARY IN
"I2016
Was
Paid $3,500 To America
Protest
& 2020 The Deplorables
Trump's Rally"

E Obama Signs Executive Order
Declaring Investigation Into
Election
TrumpNation
Results; Revote Planned For Dec. 19th

Events

Michelle Obama denounces
Trump's comment on women
–NY Times

UNITE TO FIGHT
C ISIS Leader Calls for American
Muslim
BRING AMERICA
BACK !!! TRUMP 2016
Voters to Support Hillary Clinton

Friends Who
Like Donald J. Trump

The Resistance:
The Last Line
Of Defense
Patriots for a
Free Republic Trump Squad

Arizona Anti-Immigrant Sheriff to Deploy Deputies at Polling Places
–The Guardian

Donald Trump For President

Donald Trump
Donald J. Trump Donald Trump Commander
in Chief 2020
for PRESIDENT America's
Facebook Group President (c)

Freedom Don't
Come Free
& Devildoc

Trump

President
B Hillary
Clinton In 2013: “I Would Like
To See People Like Donald Trump Run
Republican
For Office; They’re Honest
And Can’t
Patriotic
America
Party
Be Bought” Make
Great Again

for Donald
Trump 2016

Don P

PATRIOT
AMERICANS
FOR DONALD TRUMP

Up America

The Most Explosive WikiLeaks
Clinton Revelations30
(So Far)
–Breitbart

Conservative
Veterans
for America

Being Liberal
Means Being
a Hypocrite 2 Colorado

December

Resist Donald J. Trumpf

A FBI Agent Suspected in Hillary Email
Leaks Found Dead in Apparent
Donald
Murder-Suicide

1 Million Strong For
Hillary Clinton in 2016

Citizens
For Trump

VETERANS FOR PRESIDENT
DONALD J TRUMP 2016

Fake News Headlines

Donald Trump
The Political Movement

Chonda Pierce

L

VOTE TRUMP ONLY THE AMERICAN PARTY RISING

Rock Feed

Donald Trump
Criminalize VS. Hillary
Conservatism Clinton

FU Trump

73,732

Trump Says Sexual Assault Allegations
14 Oct
are "Made-Up Stories"
–IB Times

J

Anonymous The
World is Ours
All Things
#trumpvictory
#neverhillary

One Nation Under God

Trump attacks "corrupt establishment",
states Clinton should be "locked up" 13 Oct
and media is in "war against him"
FAKE NEWS POSTED
–Vox

Wikileaks reveals Democratic Party 25 Jul
has a bias against Bernie Sanders
Hillary Clinton accepts nomination

Hillary Clinton Revolution
Donald Trump for President!

Donald Trump for
President Connecticut

Deplorables for TRUMP

AMOUNT OF INTERACTIONS
February

DONALD TRUMP
COMMANDER IN CHIEF !!!!

Indies 4 Trump

The Empire Herald

Veterans for Donald Trump

D.L. Hughley

The Church
of PUNK

BURRARDSTREETJOURNAL.COM

Trump Claims America Should Never
Have Given Canada Its Independence

Sicilians For
Donald Trump

NEVER HILLARY

Hillary Clinton in 2020

22 November
F@CK
YOU
HILLARY CLINTONDemocratic
ABCNEWS.COM.CO
WE HATE YOU!
News
Donald Trump Protester Speaks Out:
I
and Events

29 July

ANTI-TRUMP

ANTI-TRUMP

Donald Trump accepts nomination

K

ANTI-CLINTON

25 July
KYPO6.COM

Pope Francis Shocks World,
Endorses Hillary Clinton for
President, Releases Statement

RAGE AGAINST THE MACHINE To
Reunite And Release Anti Donald
Trump Album

Hillary Clinton
vote blue

Was Paid $3,500 To Protest Trumps Rally

ANTI-DEMOCRAT

N/A

NEVER
HILLARY
CLINTON

ANTI-IMMIGRANTION
The Australian
Tea Party SENTIMENT

Graham Says Christians Must
Support Trump or Face Death Camps

REACT365.COM

Obama passed law for grandparents
to get all their grandchildren every
weekend

January

Trump Offering Free One-Way
Tickets to Africa & Mexico for Those
Who Wanna Leave America

17 October

Hillary Clinton In 2013: “I Would Like To See
People Like Donald Trump Run For Office;
They’re Honest And Can’t Be Bought”

BIZSTANDARDNEWS.COM

8 March

H

23 July

11 November

CONSERVATIVESTATE.COM

Pope Francis Shocks World,
Endorses Donald Trump for
President, Releases Statement

TRUMP OF CHAMPIONS

TMZHIPHOP.COM Newslo

CORRUPTION OF THE POLITICAL ESTABLISHMENT

26 September
ENDING THE FED

BREAKING Romanian Hacker With
Access To Clinton Emails Found
Dead In Jail Cell

CORRUPTION OF POLITICAL ESTABLISHMENT

CORRUPTION OF THE POLITICAL ESTABLISHMENT

Mike Pence: “Sarah Palin Is My Role
Model For Beautiful, Smart American
Women” - Newslo

NO Hillary for 2016

Obama Signs Executive Order Declaring
Investigation Into Election Results;
TRUMP
FORCE ONE
Revote Planned For
Dec. 19th

DENVER GUARDIAN

15 October
USANEWSFLASH.COM

TRUMP - SPEAK OUT
AGAINST ISLAMIZATION
OF AMERICA

12 December
ABCNEWS.COM.CO

5 November

FBI Agent Suspected in Hillary Email
Leaks Found Dead in Apparent
Murder-Suicide

MISOGYNY

18 September
POLITICOPS.COM

WOMEN4TRUMP

NATIONALIST SENTIMENT

Fake news stories

6 July

ANTI-CLINTON

RUPAUL CLAIMS TRUMP TOUCHED
HIM INAPPROPRIATELY IN THE
1990S

President Obama Confirms He Will
Refuse To Leave Office If Trump Is
Elected

ANTI-CLINTON

ABCNEWS.COM.CO

WHOA! Hillary Caught On Hot Mic
Trashing Beyonce’ With RACIAL SLURS!

14 October

THE TRUMPS STORM GROUP

Obama Signs Executive Order
Banning The Pledge Of Allegiance In
Schools Nationwide

5 November
THELASTLINEOFDEFENSE.ORG

ANTI-MUSLIM SENTIMENT

BURRARDSTREETJOURNAL.COM

THE TRUMPERS!!!
11 December

VOTER FRAUD, ANTI-IMMIGRATION SENTIMENT

ISIS Leader Calls for American
Muslim Voters to Support Hillary
Clinton

PRESIDENTIAL RACE

We Endorse
Donald J. Trump
Washington State for
Donald Trump 2016

THELASTLINEOFDEFENSE.ORG

Actor Bill Murray Announces 2016
Presidential Run

S African Billionaire Will Give $1 Million
To Anyone Who Wants To Leave
America if Donald Trump is Elected
President

Republican Community

Pro-Black
Religious

T Trump Offering Free One-Way Tickets
to Africa & Mexico for Those
Who Wanna Leave America
U Obama passed law for grandparents to
get all their grandchildren every
weekend

CLICKBAIT

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 1 → RECIPE 1

SERVING SUGGESTIONS
This recipe may be used to better
understand the publics that are
animated by fake news and the
meaning making activities that they
engage in around fake news, i.e.
how they enroll fake news in the
service of their own issue work.
This approach may inform a thicker
description of the impact of fake
news that moves away from its viral
character (the single engagement
number or metric) to understanding
who it mobilises and how.

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

39

CHAPTER 1 → RECIPE 2

HOW MAY THE TRAJECTORY
OF A FAKE NEWS STORY
BE TRACED ON FACEBOOK?

BEFORE STARTING

For this recipe it is recommended that a fake
news story is taken as a starting point and the
URL or URLs on which it is published are
identified. To illustrate this recipe we have
selected as case studies two prominent stories
about the 2016 US presidential elections, namely
"Trump Offering Free One-Way Tickets to
Africa & Mexico for Those Who Wanna Leave
America," a story that exploits anti-immigrant
sentiment and "Rage Against the Machine
to Reunite and Release Anti Donald Trump
Album," which exploits anti-Trump sentiment.
This recipe comes in two flavours. In step one
you will learn to trace public Facebook pages
and groups in which the original story URL is
posted and plot them on a timeline (a). In step
two this analysis will be extended to all URLs on
which a story has been republished (b).

40

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

START
select a fake news story
you want to trace

design queries to identify
URLs where story is published

Google Search

compile list of URLs
where story is published

input URL in

CrowdTangle

input each URL in

CrowdTangle
output data
> Fake news story URLs
> Facebook pages and
groups
that share the URLs
> Number of followers
per page or group
> Date of sharing
of the story

output data for each URL
> Fake news story URLs
> Facebook pages and
groups
that share the URLs
> Number of followers
per page or group
> Date of sharing
of the story

input data to

RAWGraphs

visualise

a

HOW DOES THE STORY
"RAGE AGAINST THE
MACHINE TO REUNITE
AND RELEASE ANTI
DONALD TRUMP ALBUM"
TRAVEL ON FACEBOOK?
CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

input data to

RAWGraphs

visualise

b

HOW DOES THE STORY
"TRUMP OFFERING FREE
ONE-WAY TICKETS TO AFRICA
& MEXICO FOR THOSE WHO
WANNA LEAVE AMERICA" AND
ITS DEBUNKED VERSIONS
TRAVEL ON FACEBOOK?
41

START
select a fake news story
you want to trace

input URL in

CrowdTangle

output data
> Fake news story URLs
> Facebook pages and
groups
that share the URLs
> Number of followers
per page or group
> Date of sharing
of the story
input data to

RAWGraphs

visualise

a

HOW DOES THE STORY
"RAGE AGAINST THE
MACHINE TO REUNITE
AND RELEASE ANTI
DONALD TRUMP ALBUM"
TRAVEL ON FACEBOOK?
42

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 1 → RECIPE 2

a. IDENTIFY FACEBOOK PAGES AND GROUPS
THAT SHARE A FAKE NEWS STORY VIA
THE ORIGINAL URL

Public Facebook pages and groups that share a fake
news story may be detected through a social media
monitoring tool such as  CrowdTangle’s browser
extension.
◊◊ The names of these pages and groups, their
→ followers’ count, the → interactions that
they generate as well as the date of sharing of
the story may be recorded in a spreadsheet per
story URL.
◊◊ To explore the temporal dynamics of
the circulation of the fake news story on
Facebook, you may plot its trajectory across
pages and groups on a timeline.  RAWGraphs
can be used to create the base layer of the
visualisation. Note which publics engage with
the story as well as whether the moment of
debunking of a story affects its circulation.[1]
◊◊ To take the analysis one step further, a
qualitative analysis of how fake news is
enrolled by each of these pages to support
issue work may be undertaken. This may be
done by examining the context in which the
stories are shared, i.e. whether they are shared
uncritically or called out as fake as well as
how they are framed in relation to the issues
represented by the pages that share them. It is
to be noted that such analysis might at times
be difficult due to the fact that Facebook posts
that share the most prominent fake news
stories may be removed from the Facebook
interface and API.

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

[1]
Please note that this
analysis will not account for all instances
of sharing of a fake
news story URL on
Facebook but only for
the top 500 instances
(per URL) of prominent sharing to public
Facebook pages which
CrowdTangle monitors.
For more information
see the note on CrowdTangle data on p.27.

43

HOW DOES THE
STORY "RAGE AGAINST
THE MACHINE TO
REUNITE AND RELEASE
ANTI DONALD TRUMP
ALBUM" TRAVEL ON
FACEBOOK?
Trajectory of “Rage Against the Machine
to Reunite and Release Anti Donald
Trump Album" story on Facebook pages
and groups retrieved with CrowdTangle.
The story circulates best between March
and June 2016 as satire amongst English
language music and entertainment
groups. It is revived in November after
the US elections, when it is also picked up
by Italian music and political pages.

Published the 11th of March 2016 on heaviermetal.net

RAGE AGAINST THE MACHINE
To Reunite And Release Anti Donald Trump Album
The Church of PUNK
Justice Through Music
Rock Feed
Rock.It Boy Entertainment
音地大帝 Indie DaaDee
Magic 89.3
98 Rock
Alabama Citizens for Bernie Sanders 2020
Anonymous The World is Ours
This page is very offensive
Bernie Sanders for President 2020
Occupy Orange County CA
Wunderkynd
Ballin' On A Budget
No Music No Life.
RAGE AGAINST THE MACHINE CHRISTMAS NO.1 2009
Circo Voador
FM 949
Bernie Sanders - Support from abroad
Boogat
FM 949
107.9 The Mix
Occupy London
Occupy movement
United Steelworkers Activists for Political Revolution
United People for Latinos in Film TV and Theater
Missoula for our Green Party Revolution
Our Revolution Continues- Activist Resources

STORIES SHARED ON FB
Post

Dj Mike Style!!!
Heavier Metal

Page that shares it

METAL ARMADA

PAKISTAN RISING

Rock Feed

Interactions

Heavier Metal
Rock Feed

Music and/or Radio
Activism
Politics
Others

“Rock Feed” is sharing the story
both at the earliest as well as
at the end of the first wave of posting.

POSTS INTERACTIONS
0

1588

POSTS ON THE SAME PAGE
11/03/2016
44

APRIL

MAY

JUNE

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

The story reappears on Facebook on
the 11th of November, shared by the
page “Apartment Khunpa

Replace with "No prominent sharing of the story URL on public Facebook pages
and groups from late May to early November according to CrowdTangle data
Apartment Khunpa
The Guitar Mag
Banda Bassotti
Marcello Belotti - Sinistra Italiana EU
ROCK Mundial
The Church of PUNK
Radioactivo 98.5
Los Angeles Punk Museum
Veterans Against Rep. Ignorance
FU Trump
Occupy Orange County CA
JULY

AUGUST

SEPTEMBER

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

OCTOBER

NOVEMBER

DECEMBER
45

START
select a fake news story
you want to trace

design queries to identify
URLs where story is published

Google Search

compile list of URLs
where story is published

input each URL in

CrowdTangle

output data for each URL
> Fake news story URLs
> Facebook pages and
groups
that share the URLs
> Number of followers
per page or group
> Date of sharing
of the story
input data to

RAWGraphs

visualise

b

HOW DOES THE STORY
"TRUMP OFFERING FREE
ONE-WAY TICKETS TO AFRICA
& MEXICO FOR THOSE WHO
WANNA LEAVE AMERICA" AND
ITS DEBUNKED VERSIONS
TRAVEL ON FACEBOOK?

46

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 1 → RECIPE 2

b. IDENTIFY FACEBOOK PAGES AND
GROUPS THAT SHARE ALL INSTANCES
OF A FAKE NEWS STORY

As fake news stories may be republished by a
number of sources, the previous analysis may
be enriched by tracing the circulation not only
of the original URL on which the chosen story
is posted but all instances of story republication
across a number of different sites.
◊◊ To identify the websites which republish
a story as well as those which debunk it,
you may query the title of the fake news
in a search engine of choice (e.g.  Google
Web Search) using a research browser [2]
and extract the URLs corresponding to
instances of republication and debunking
of the story from the returned list of
results.
◊◊ Query the resulting URLs in a social
monitoring tool (such as  CrowdTangle)
to get the list of Facebook groups and
pages that prominently share the URLs
corresponding to both the fake story and
its debunked versions.[3]
◊◊ You may plot these pages on a timeline to
see whether different fake news sources
spawn different story trajectories on
Facebook and whether debunked versions
are being acknowledged.

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

[2]
See instructions
on how to set up
a research browser in this video
tutorial: https://
www.youtube.com/
watch?v=bj65Xr9GkJM

[3]
Please see the note
on p.43 about the
limitations of using
CrowdTangle data.

47

HOW DOES THE
STORY "TRUMP
OFFERING FREE ONEWAY TICKETS TO
AFRICA & MEXICO FOR
THOSE WHO WANNA
LEAVE AMERICA"
AND ITS DEBUNKED
VERSIONS TRAVEL ON
FACEBOOK?
Timeline of "Trump Offering Free
One-Way Tickets to Africa & Mexico
for Those Who Wanna Leave America”
story and its sites of publication on
the web and Facebook. The story is
republished without critical context
on multiple → clickbait sites in the
week following its original publication.
This gives the story multiple lives on
Facebook. Its sharing on a fake news site
animates political publics while its sharing
on clickbait sites sees the story being
recycled as clickbait by viral pages. The
publics sparked into being by the fake
news story and the debunked version
thereof do not overlap.

Published the 11th of November 2016 on viralmugshot.com

Trump Offering Free One-Way Tickets to Africa & Mexico
for Those Who Wanna Leave America
VIRALMUGSHOT.COM

The first instance of the story appears o

The first repost has been
published by myhiphop.org on
the 11th of November

MYHIPHOP.ORG

n viralmugshot.com in

early Novem

NYMAG.COM/SELECTALL
New York Magazine

Daily Media Buzz

Select All
YEPSEE.COM
Amazing and Crazy videos
Crazy Accidents 18+
Damn
Embarrassing Party Photos
EpicFail.com
I post disturbing things.
New York Meta
The Craziest fight videos

The story is published
on Yepsee.com and has
been shared on eight
different Facebook
pages/groups during the
same day

VANITYFAIR.COM
Pakistan Rising
vanityfairmagazine
VFHive

THUGIFY.COM
ThugLifeClips

DAILYNEWSPOSTS.INFO
DONALD TRUMP
COMMANDER IN CHIEF !!!!
NO Hillary for 2016
TRUMP - SPEAK OUT AGAINST ISLAMIZATION
OF AMERICA - FIM DA COLONIZAÇÃO ISLÃ
Trump USA's CEO
TrumpNation
VOTE TRUMP ONLY - THE AMERICAN PARTY RISING

ATRENDING.NET
Exposing The Truth √

QATARDAY.COM
Qatar Day

POLITICO.CO

DIYHILFE.COM

JimHeathT
LionPublishe
mnsucollegedemocra
modernliber
PolPrepostero

Mobbing Hilfe

politi

HOPEFORNIGERIAONLINE.COM

pg/politicom

Hope for Nigeria

politicomedia

PENam

LUXURYANDGLAMOR.COM

DavidWilliam

Daily Video

No sharing on Facebook from mid November to late December
NOV 13

48

NOV 20

NOV 27

DEC 04

DEC 11

DEC 18

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

DEC 25

mber, but CrowdTan
gle only indicates insta

nces of Facebook sharing in late

February

Prince Sam
TheCelebtricity
Don P

The first debunk has been
published by nymag.com
on the 30th of December

FAKE STORY INSTANCES
THEVERGE.COM

Earliest version (original)

TechKnowledgeIt
verge

Reposted Version
Debunked version

NEWSER.COM
newser

POSTS ON FACEBOOK
Post

BUZZFEED.COM
BuzzFeed News
BuzzFeed Politics
BuzzFeed UK News
BuzzFeed World
Isaac Saul
StyleHaul
The European Journalism Observatory | EJO
BuzzFeed
Coloradans For Bernie
Ethical Journalism Network

Page that shares it
PAKISTAN RISING
Interactions

Fake Story
Debunk
Hillary Clinton The People's Choice

POSTS INTERACTIONS
CIVILIZED.LIFE

0

civilized.life

1588

POSTS ON A SAME PAGE

INQUISITR.COM
theinquisitr

OM

TV
ers
ats
rals
ous

INEWS.CO.UK
theipaper

ico

TRUEAMERICANS.ME

mag

President Donald J. Trump
TRUMP - SPEAK OUT AGAINST [...]

adesk

merican

EDUBLOG.SCHOLASTIC.COM

mBateman

EmmaClarkLibrary

NOWTORONTO.COM
nowmagazine

2017
JAN 01

JAN 08

JAN 15

JAN 22

JAN 29

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

FEB 05

FEB 12

FEB 19

FEB 26

MAR 05

MAR 12

49

The story reappears on Facebook on
the 11th of November, shared by the
page “Apartment Khunpa

k pages
e data

Published the 11th of November 2016 on viralmugshot.com

Trump Offering Free One-Way Tickets to Africa & Mexico
for Those Who Wanna Leave America
VIRALMUGSHOT.COM

Apartment Khunpa

The first instance of the story appears o
n viralmugshot.com in

The first repost has been
published by myhiphop.org on
the 11th of November

MYHIPHOP.ORG

early November, but
C

rowdTangle only ind

Select All

The Guitar Mag

Amazing and Crazy videos
Crazy Accidents 18+
Damn
Embarrassing Party Photos
EpicFail.com
I post disturbing things.
New York Meta
The Craziest fight videos

Banda Bassotti

sharing in late February

Prince Sam
TheCelebtricity

The first debunk has been
published by nymag.com
on the 30th of December

New York Magazine

Daily Media Buzz

YEPSEE.COM

icates instances of Facebo
ok

Don P
NYMAG.COM/SELECTALL

The story is published
on Yepsee.com and has
been shared on eight
different Facebook
pages/groups during the
same day

VANITYFAIR.COM
Pakistan Rising
vanityfairmagazine
VFHive

FAKE STORY INSTANCES
THEVERGE.COM

Earliest version (original)

TechKnowledgeIt
verge

THUGIFY.COM
ThugLifeClips

Reposted Version
Debunked version

NEWSER.COM

Marcello Belotti - Sinistra Italiana EU

newser

POSTS ON FACEBOOK

DAILYNEWSPOSTS.INFO

BuzzFeed News
BuzzFeed Politics
BuzzFeed UK News
BuzzFeed World
Isaac Saul
StyleHaul
The European Journalism Observatory | EJO
BuzzFeed
Coloradans For Bernie
Ethical Journalism Network

TRUMP - SPEAK OUT AGAINST ISLAMIZATION
OF AMERICA - FIM DA COLONIZAÇÃO ISLÃ

ROCK Mundial

Post

BUZZFEED.COM

DONALD TRUMP
COMMANDER IN CHIEF !!!!
NO Hillary for 2016

Trump USA's CEO
TrumpNation
VOTE TRUMP ONLY - THE AMERICAN PARTY RISING

Page that shares it
PAKISTAN RISING
Interactions

Fake Story
Debunk
Hillary Clinton The People's Choice

POSTS INTERACTIONS

The Church of PUNK

CIVILIZED.LIFE

ATRENDING.NET

1588

POSTS ON A SAME PAGE

INQUISITR.COM

QATARDAY.COM

theinquisitr

Qatar Day

Radioactivo 98.5

0

civilized.life

Exposing The Truth √

POLITICO.COM

DIYHILFE.COM

JimHeathTV
LionPublishers
mnsucollegedemocrats
modernliberals
PolPreposterous

Mobbing Hilfe

INEWS.CO.UK
theipaper

politico

HOPEFORNIGERIAONLINE.COM

Los Angeles Punk Museum

TRUEAMERICANS.ME

pg/politicomag

Hope for Nigeria

President Donald J. Trump
TRUMP - SPEAK OUT AGAINST [...]

politicomediadesk
PENamerican

LUXURYANDGLAMOR.COM

EDUBLOG.SCHOLASTIC.COM

DavidWilliamBateman

EmmaClarkLibrary

Daily Video

Veterans Against Rep. Ignorance

NOWTORONTO.COM

No sharing on Facebook from mid November to late December

NOV 13

NOV 20

NOV 27

DEC 04

DEC 11

DEC 18

DEC 25

nowmagazine
2017
JAN 01

JAN 08

JAN 15

JAN 22

JAN 29

FEB 05

FEB 12

FEB 19

FEB 26

MAR 05

MAR 12

FU Trump
Occupy Orange County CA

TOBER

NOVEMBER

50

DECEMBER

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 1 → RECIPE 2

SERVING SUGGESTIONS
This recipe may be used to
understand the trajectory of a
fake news story on Facebook,
the different phases of its life
cycle as well as key moments and
intermediaries associated with its
dissemination.

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

51

CHAPTER 1 → RECIPE 3

DO FACT-CHECKING
INITIATIVES REACH THE
PUBLICS OF FAKE NEWS
ON FACEBOOK?

BEFORE STARTING

This recipe takes as a starting point a list of
fake news stories. There are different ways
of obtaining these lists – including starting
with existing lists as well as creating your
own. To illustrate this recipe we use an
already existing list of 22 fake news stories
about various political issues pertaining to
the 2016 presidential elections in the US that
generated most engagement on Facebook.
These were identified by BuzzFeed News.
There are two steps to this recipe. The first
is to identify URLs that circulate corrections
or “debunking web pages” for each fake news
story. The second is to explore how public
Facebook pages engage with both fake news
stories and their corresponding debunking
web pages (b).

52

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

START
List of 22 URLs of political
fake news stories
Source: BuzzFeed News

query
“Fake News Title 1” + Fake
“Fake News Title 2” + Fake
“. . .” in

Google Web Search

Retain the top ranked URL
of a debunk per fake news story
...

merge all URLs in a single list

+

input URLs in

CrowdTangle

output data
> URLs of fake news story
or debunked version thereof
> Facebook pages and groups
that share either the fake
news story or its correction
> number of followers
per page or group
input data to

RAWGraphs

visualise

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

ARE DEBUNKING WEB
PAGES ACKNOWLEDGED
BY THE PUBLICS OF FAKE
NEWS?
53

START
List of 22 URLs of political
fake news stories
Source: BuzzFeed News

query
“Fake News Title 1” + Fake
“Fake News Title 2” + Fake
“. . .” in

Google Web Search

Retain the top ranked URL
of a debunk per fake news story
...

54

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 1 → RECIPE 3

a. IDENTIFY WEB PAGES WHICH AIM TO
DEBUNK FAKE NEWS STORIES

To identify prominent debunking web pages
for a given fake news story you may use the
 Google Web Search engine. In addition to
this, you may also query fact-checking sites for
keywords describing a fake news story.
◊◊ In order to find corrections of fake news
articles queries need to be designed for
each fake news item in your list. One
strategy would be to use the title of the
story in quotation marks followed by the
word “fake” (e.g. “‘Trump Offering Free
One-Way Tickets to Africa & Mexico for
Those Who Wanna Leave America’ fake”).
◊◊ You may use the search engine ranking
as an indication of salience of correction
and select the highest ranked URLs
corresponding to a corrected version of the
fake news story in question.
◊◊ The result of this step is a list of URLs
containing the most highly ranked
debunking web pages per fake news story.

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

55

START
List of 22 URLs of political
fake news stories
Source: BuzzFeed News

query
“Fake News Title 1” + Fake
“Fake News Title 2” + Fake
“. . .” in

Google Web Search

Retain the top ranked URL
of a debunk per fake news story
...

merge all URLs in a single list

+

input URLs in

CrowdTangle

output data
> URLs of fake news story
or debunked version thereof
> Facebook pages and groups
that share either the fake
news story or its correction
> number of followers
per page or group
input data to

RAWGraphs

visualise

56

ARE DEBUNKING WEB
PAGES ACKNOWLEDGED
BY THE PUBLICS OF FAKE
NEWS?
A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 1 → RECIPE 3

b. MAP THE OVERLAP BETWEEN THE
PUBLICS OF FAKE NEWS STORIES AND
WEB PAGES WHICH AIM TO DEBUNK
THEM
Public Facebook pages and groups that
prominently share both fake news stories
as well as web pages which aim to debunk
them may be detected through a social media
monitoring tool such as  CrowdTangle’s
browser extension.

◊◊ To explore whether the debunking web
pages are acknowledged by the publics
which share the fake news stories, identify
whether there is an overlap between
the public Facebook pages and groups
that share fake news stories and those
debunking web pages issued in response.
◊◊ This may be illustrated by means of a
→ circle packing visualisation. You may use
 RawGraphs for this operation.

CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

57

ARE DEBUNKING
WEB PAGES
ACKNOWLEDGED
BY THE PUBLICS OF
FAKE NEWS?
Fake news pages and debunking
web pages have different publics
on Facebook. Only six of the public
pages that share fake news stories have
acknowledged web pages which aim
to debunk them in our CrowdTangle
dataset. While Google looks to prioritise
debunking web pages, on Facebook it is
fake news stories that circulate better.
While both progressive and conservative
pages share fake news stories it is
primarily progressive Facebook pages
and those pertaining to journalists and
fact-checking initiatives that share web
pages which aim to debunk fake news
stories.

Rage Against the Machine To Reunite
And Release Anti Donald Trump Album

Actor Bill Murray Announces
2016 Presidential Run

Pubished the 11th of March 2016
on heaviermetal.net

Pubished the 3th of October 2016
on www.abcnews.com.co

Rupaul claims Trump touched
him inappropriately in the 1990s

African Billionaire Will Give $1
Million To Anyone Who Wants
To Leave America if Donald Trump
is Elected President

Pubished the 14th of October 2016
on worldnewsdailyreport.com

Pubished the 3th of November 2016
on www.empireherald.com

D.L. Hughley

Hillary Clinton In 2013: “I Would Like
To See People Like Donald Trump Run
For Office; They’re Honest And Can’t
Be Bought”
Pubished the 17th of October 2016
on conservativestate.com

Graham Says Christians
Must Support Trump or Face
Death Camps
Pubished the 23th of July 2016
on www.bizstandardnews.com

Page or group which shares both
fake story and its correction

Fake

Obama Signs Executive Order
Declaring Investigation Into Election
Results; Revote Planned For Dec. 19th

Pope Francis Shocks World,
Endorses Hillary Clinton for
President, Releases Statement

Debunk

Pubished the 12th of December 2016
on www.abcnews.com.co

Pubished the 25th of July 2016
on www.kypo6.com

POST INTERACTIONS

100

58

41271

Hillary Clinton
for President in 2016

Connecticut
Progressives

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

Mike Pence: “Sarah Palin Is
My Role Model For Beautiful,
Smart American Women”

President Obama Confirms
He Will Refuse To Leave Office
If Trump Is Elected

Trump Offering Free One-Way
Tickets to Africa; Mexico for Those
Who Wanna Leave America

Pubished the 18th of September 2016
on www.politicops.com

Pubished the 7th of September 2016
on www.burrardstreetjournal.com

Pubished the 11th of November 2016
on www.tmzhiphop.com

Obama passed law for grandparents
to get all their grandchildren
every weekend
Pubished the 8th of March 2016
on www.react365.com

BREAKING Romanian Hacker
With Access To Clinton Emails
Found Dead In Jail Cell

ISIS Leader Calls for American
Muslim Voters to Support
Hillary Clinton

WHOA! Hillary Caught On
Hot Mic Trashing Beyonce’
With RACIAL SLURS!

Pubished the 6th of July 2016
on Christian Times

Pubished the 11th of October 2016
on worldnewsdailyreport.com

Pubished the 5th of November 2016
on www.thelastlineofdefense.org

Trump Claims America
Should Never Have Given
Canada Its Independence

Van Full Of Illegals Shows Up To
Vote Clinton At SIX Polling Places,
Still Think Voter Fraud Is A Myth?

FBI Agent Suspected in Hillary
Email Leaks Found Dead
in Apparent Murder-Suicide

Pubished the 29th of July 2016
on www.burrardstreetjournal.com

Pubished the 5th of November 2016
on www.thelastlineofdefense.org

Pubished the 15th of October 2016
on www.usanewsflash.com

Pope Francis Shocks World,
Endorses Donald Trump for
President, Releases Statement

Obama Signs Executive Order
Banning The Pledge Of Allegiance
In Schools Nationwide

Donald Trump Protester
Speaks Out: I Was Paid $3,500
To Protest Trump's Rally

Pubished the 26th of September 2016
on Ending The Fed

Pubished the 11th of December 2016
on www.abcnews.com.co

Pubished the 22th of November 2016
on www.abcnews.com.co

Bestwebtools
Anonymous Portugal
Internacional

Criminalize
Conservatism

PolitiFact

Read the damn news
CHAPTER 1: MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK

59

Rage Against the Machine To Reunite
And Release Anti Donald Trump Album

Actor Bill Murray Announces
2016 Presidential Run

Pubished the 11th of March 2016
on heaviermetal.net

Pubished the 3th of October 2016
on www.abcnews.com.co

Mike Pence: “Sarah Palin Is
My Role Model For Beautiful,
Smart American Women”

President Obama Confirms
He Will Refuse To Leave Office
If Trump Is Elected

Trump Offering Free One-Way
Tickets to Africa; Mexico for Those
Who Wanna Leave America

Pubished the 18th of September 2016
on www.politicops.com

Pubished the 7th of September 2016
on www.burrardstreetjournal.com

Pubished the 11th of November 2016
on www.tmzhiphop.com

CHAPTER 1 → RECIPE 3

Obama passed law for grandparents
to get all their grandchildren
every weekend
Pubished the 8th of March 2016
on www.react365.com

Rupaul claims Trump touched
him inappropriately in the 1990s
Pubished the 14th of October 2016
on worldnewsdailyreport.com

African Billionaire Will Give $1
Million To Anyone Who Wants
To Leave America if Donald Trump
is Elected President
Pubished the 3th of November 2016
on www.empireherald.com

BREAKING Romanian Hacker
With Access To Clinton Emails
Found Dead In Jail Cell

ISIS Leader Calls for American
Muslim Voters to Support
Hillary Clinton

WHOA! Hillary Caught On
Hot Mic Trashing Beyonce’
With RACIAL SLURS!

Pubished the 6th of July 2016
on Christian Times

Pubished the 11th of October 2016
on worldnewsdailyreport.com

Pubished the 5th of November 2016
on www.thelastlineofdefense.org

Bestwebtools
Anonymous Portugal
Internacional
D.L. Hughley

Hillary Clinton In 2013: “I Would Like
To See People Like Donald Trump Run
For Office; They’re Honest And Can’t
Be Bought”
Pubished the 17th of October 2016
on conservativestate.com

Graham Says Christians
Must Support Trump or Face
Death Camps

Trump Claims America
Should Never Have Given
Canada Its Independence

Van Full Of Illegals Shows Up To
Vote Clinton At SIX Polling Places,
Still Think Voter Fraud Is A Myth?

FBI Agent Suspected in Hillary
Email Leaks Found Dead
in Apparent Murder-Suicide

Pubished the 23th of July 2016
on www.bizstandardnews.com

Pubished the 29th of July 2016
on www.burrardstreetjournal.com

Pubished the 5th of November 2016
on www.thelastlineofdefense.org

Pubished the 15th of October 2016
on www.usanewsflash.com

Page or group which shares both
fake story and its correction

Fake

Obama Signs Executive Order
Declaring Investigation Into Election
Results; Revote Planned For Dec. 19th

Pope Francis Shocks World,
Endorses Hillary Clinton for
President, Releases Statement

Pope Francis Shocks World,
Endorses Donald Trump for
President, Releases Statement

Obama Signs Executive Order
Banning The Pledge Of Allegiance
In Schools Nationwide

Donald Trump Protester
Speaks Out: I Was Paid $3,500
To Protest Trump's Rally

Debunk

Pubished the 12th of December 2016
on www.abcnews.com.co

Pubished the 25th of July 2016
on www.kypo6.com

Pubished the 26th of September 2016
on Ending The Fed

Pubished the 11th of December 2016
on www.abcnews.com.co

Pubished the 22th of November 2016
on www.abcnews.com.co

POST INTERACTIONS

100

41271

Hillary Clinton
for President in 2016

Criminalize
Conservatism

PolitiFact
Connecticut
Progressives
Read the damn news

SERVING SUGGESTIONS
This recipe may be used as
one way to assess the impact
of attempts to debunk fake
news by examining whether
debunking responses to fake
news are acknowledged on the
platform that generates most
engagement with fake news,
Facebook, and by the particular
publics which share and engage
with fake news.

60

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

Chapter 2

TRACING THE
CIRCULATION OF FAKE
NEWS ON THE WEB
Where do fake news stories originate?
By which sites are they first disseminated?
Which are the most visible sources related
to a fake story? When and by whom are
they mentioned?

CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

61

Introduction - Fake news are not just “false
news”. They are interesting not so much
because their content or form are different
from that of “authentic news”, but because they
travel as much as (and sometimes more than)
mainstream news. If a blog claims that Pope
Francis endorses Donald Trump, it's just a lie. If
the story is picked up by dozens of other blogs,
retransmitted by hundreds of websites, crossposted over thousands of social media accounts
and read by hundreds of thousands, then it
becomes fake news.
The following recipes investigate the
circulation of fake news for two reasons. Firstly,
from a political point of view many have
expressed disappointment that techniques and
tactics commonly used to tackle fake news have
not lived up to expectations. Fact-checking and
debunking, in particular, often do not succeed
in preventing the circulation of hoaxes and
rumors. On the contrary: they can inadvertently
62

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

contribute to making them even more visible
on the web. A better understanding of how fake
news travels online can help to inform responses
that are more attuned to the phenomena.
Secondly, from a methodological point of view,
as there is no “ontological” difference between
fake and authentic news, studying fake news
circulation can help us understand more about
how other kinds of news travel.
This recipe comes in two flavours. Firstly,
we propose a manual variant which can be
executed without the need for any particular tool
or technical knowledge. This variant is easy to
execute but also time consuming. It is based on
a search for web pages referring to fake news
stories and on the manual identification of the
dates of their publication and the sources that
they cite. Secondly, we propose a semi-automated
version which allows this approach to be scaled
to more pages, but demands more technical skills
and may require more manual verification.
CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

63

CHAPTER 2 → RECIPE 1

WHERE DO FAKE
NEWS ORIGINATE?
BY WHAT SITES ARE THEY
FIRST RETRANSMITTED?

BEFORE STARTING

For this recipe you will need to choose a fake
news story whose circulation you would like
to trace. The more distinctive your story is,
the easier it will be to follow its circulation. In
the example, we focus on the “Pope Endorses
Trump” story that was widely circulated
around the US Elections.

64

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

START

open Chrome browser
in incognito mode

query
[ Pope AND endors* AND (Trump OR Clinton)
AND NOT (hoax OR “fake news” OR lie OR debunk) ]

Google Web Search

a
compile a list with
the top resulting URLs

input URLs in a spreadsheet
> The page advocating
or debunking the fake
news item
> Rank in the search
engine results
> the date of publication
of each page

record all sources cited in the occurrences
of your story, noting down:

+
+
+

> if they are cited as evidence
or counter-evidence
> if they are cited through
hyperlink, textual reference
or copying/pasting of its
content

Make sure you visit both the results of
your initial search and all the sources
cited by those results

Table2Net

import data in

Gephi

visualise

extract network of instances
and references with

b

Plot URLs on a timeline with

c

visualise

Graph Recipes

CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

HOW DO THE
OCCURRENCES OF THE
“POPE ENDORSES
TRUMP” STORY CITE
EACH OTHER?

WHAT IS THE LIFE OF
THE “POPE ENDORSES
TRUMP” STORY
ACCORDING TO PAGES
IN SEARCH ENGINE
RESULTS?
65

START

open Chrome browser
in incognito mode

query
[ Pope AND endors* AND (Trump OR Clinton)
AND NOT (hoax OR “fake news” OR lie OR debunk) ]

Google Web Search

a
compile a list with
the top resulting URLs

66

input URLs in a spreadsheet
> The page advocating
or debunking the fake
news item
> Rank in the search
engine results
> the date of publication
of each page

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 2 → RECIPE 1

a. IDENTIFY AND ANALYSE THE
OCCURRENCES OF YOUR STORY IN SEARCH
ENGINE RESULTS

Successful fake news stories always appear on
several web pages. In the first step of this recipe,
you will identify and collect information about these
occurrences.
◊◊ Identify the occurrences of your story by
querying one or more search engines (we used
 Google Web Search). Since fake news stories
evolve while circulating, consider keywords that
may capture different variants of the story.
◊◊ Rely on search engines to rank results by
relevance and concentrate on the first results
(under the working assumption that they are the
ones that circulated the most).
◊◊ To avoid "filter bubbles" and personalised
results, consider using a dedicated research
browser.[1]
◊◊ Be aware that search engines may give more
visibility to debunkers than to original sources
– and be careful not to overestimate the
circulation of debunkers based on this. Also
bear in mind that with this approach you see
the phenomena “through the eyes” of the search
engine that you selected – which will become a
part of your story or research.

[1]
See instructions
on how to set up a
research browser in this video
tutorial: https://
www.youtube.com/
watch?v=bj65Xr9GkJM

◊◊ Record all relevant metadata for each relevant
search result. You can collect as many variables
as you like, but make sure to characterise how
the page refers to the story (e.g. as a reliable
news source, or as a problematic claim to be
debunked), as well as some indicator of its
“visibility” (e.g. the rank in the search results).
CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

67

START

open Chrome browser
in incognito mode

query
[ Pope AND endors* AND (Trump OR Clinton)
AND NOT (hoax OR “fake news” OR lie OR debunk) ]

Google Web Search

a
compile a list with
the top resulting URLs

input URLs in a spreadsheet
> The page advocating
or debunking the fake
news item
> Rank in the search
engine results
> the date of publication
of each page

record all sources cited in the occurrences
of your story, noting down:

+
+
+

> if they are cited as evidence
or counter-evidence
> if they are cited through
hyperlink, textual reference
or copying/pasting of its
content

Make sure you visit both the results of
your initial search and all the sources
cited by those results

Table2Net

68

import data in

Gephi

visualise

extract network of instances
and references with

b

HOW DO THE
OCCURRENCES OF THE
“POPE ENDORSES
TRUMP” STORY CITE
EACH OTHER?

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 2 → RECIPE 1

b. EXTRACT THE NETWORK OF
REFERENCES AROUND YOUR STORIES
Fake news is supported or opposed through
a network of references: websites that share
rumours cite other pages to support their
claims, while debunking initiatives flag toxic
websites or refer to sources denying them.
In this step, we will trace the network of
references of a specific fake news story.

◊◊ Record all sources cited in the occurrences
of your story. For each, note down if the
source is cited as evidence or counterevidence and if it is cited through a
hyperlink (e.g. http://snopes.com), a
textual reference (e.g. “the website WTOE
5 News”) or copying and pasting its
content.
◊◊ Make sure you visit not only all the results
of your initial search, but also all the
sources cited by those results.
◊◊ Extract the networks of occurrences and
references (you can use  Table2Net).
◊◊ Visualise the network (using  Gephi, for
instance), applying a force-directed layout;
sizing the nodes according to the number
of citations they receive; and colouring the
nodes according to how they report the
story (advocating or debunking).

CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

69

HOW DO THE
OCCURRENCES OF
THE “POPE ENDORSES
TRUMP” STORY CITE
EACH OTHER?

Newsweek

RedNationRising

redd
Trend Network

EndingtheFe

Network of cross-references between
the pages mentioning the “Pope
Endorses Trump” story. In the image
the nodes represent the different pages
on which the fake story appears. The
comparison of the colour of the nodes
(which indicates whether the page affirm
or debunks the story), their size (which
indicates the number of citations received
by the page) and their number reveal
the great visibility of the debunking and
neutral pages compared to websites that
spread the fake story as authentic.

Bass Barn

Clas
New York Magazine

Business Insider UK

The Guardian

Los A

Every News Here

The New York Times
Rapture Forums

Snopes
Earn The

The New York Times
Nieman Lab

Just Plain P
Morning News USA

The Globe and Mail

Buzzfeed

Hoax-Slaye

TheJournal.ie
Hacking Humanity
NUMBER OF MENTIONS
1

Buzzfeed

18

Reuters

Slate
WTOE 5 News
FactCheck.org
Associated Press News
Fake News

WGAL
CBS News

Debunk
Other

MSN
KYPO6

Y
70

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

Complex

g.us

Complex

dit.com/r/thedonald
The Mercury News

Fed.com

Commonweal Magazine
Southend News Network

Godlike Productions

ss CNBC

Heavy.com

MSN

K

Google Groups

Quora

Angeles Times

Buzzfeed

Catholic News Service

Fox News
Necklace

Green Valley News

Politics!

er

LinkedIn Pulse

WTOE 5 News

The Guardian

Dinar Vets
Newsbreakshere.com

Vox
SCO News
New Statesman
The Independent
TruthOrFiction.com

Buzzfeed
The Daily Dot

ThatsFake.com

NBC News

CATHOLA
The Guardian

WGAL

Buzzfeed
The Daily Beast
Associated Press News

YouTube
CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

71

START

open Chrome browser
in incognito mode

query
[ Pope AND endors* AND (Trump OR Clinton)
AND NOT (hoax OR “fake news” OR lie OR debunk) ]

Google Web Search

a
compile a list with
the top resulting URLs

input URLs in a spreadsheet
> The page advocating
or debunking the fake
news item
> Rank in the search
engine results
> the date of publication
of each page

record all sources cited in the occurrences
of your story, noting down:

+
+
+

> if they are cited as evidence
or counter-evidence
> if they are cited through
hyperlink, textual reference
or copying/pasting of its
content

Make sure you visit both the results of
your initial search and all the sources
cited by those results

extract network of instances
and references with

Table2Net

import data in

Gephi

Plot URLs on a timeline with

c

visualise

Graph Recipes

72

WHAT IS THE LIFE OF
THE “POPE ENDORSES
TRUMP” STORY
ACCORDING TO PAGES
IN SEARCH ENGINE
RESULTS?

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 2 → RECIPE 1

c. VISUALIZE THE FAKE NEWS INSTANCES
OVER TIME

The network extracted in the previous step
can help you understanding not only who cited
whom, but also how and in which direction your
fake news travelled. To reveal the circulation use
the dates that you collected in the first step of this
recipe.
◊◊ Arrange the network of instances extracted
in the previous step chronologically. You can
use different visual styles to represent the
different kind of citations (to do so, we used
a custom script of  Graph Recipes tools).

CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

73

WHAT IS THE LIFE OF
THE “POPE ENDORSES
TRUMP” STORY
ACCORDING TO PAGES
IN SEARCH ENGINE
RESULTS?

Buzzfeed
The Guardian
Truthorfiction

The Daily Beast
Commonweal Magazine

Chronological network of the crossreferences between the pages
mentioning the “Pope Endorses
Trump” story. In this image, the colour
of the nodes indicates whether the
page affirms or debunks the story
and the type of line indicates how
different pages cite each other. The
high presence of dotted lines going
from green to orange or gray nodes
shows how debunking initiatives tend to
mention original sources but not link to
them. This technique is used to flag fake
websites without increasing their online
visibility by explicitly linking to them.
source in original list
source identified through
the analysis
CITED
SOURCE

SOURCE

TYPE OF PAGE
Fake News

No sharing of the story URL from

Debunk

TYPE OF CITATION
source affirming
the story cited as
hyperlink
source affirming
the story cited
as text
source affirming
the story cited as
copy-paste

GOOGLE RANK
n. 1

n. 100

WITHOUT DATE

JANUARY

FEBRUARY

MARCH

APRIL

2016
74

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

MAY

TheJournal.ie
Vox
Dinar Vets
Godlike Productions

TruthOrFiction.com

Linkedin Pulse
Google Groups

Catholic News Service

Earn The Necklace

Just Plain Politics!

WTOE 5 News

Newsbreakshere.com
Bass Barn

Snopes

New York Magazine
Nieman Lab
The New York Times
The New York Times
Business Insider UK
Morning News USA
The Guardian

Hoax-Slayer
ThatsFake.com
Green Valley News
Every News Here
Kypo6

Hacking Humanity

Buzzfeed
The Independent
New Statesman
SCO News

Rapture Forums

Class CNBC

MSN
YouTube
Trend Network

Reddit
Newsweek

RedNationRising.us
EndingtheFed.com
Quora
Los Angeles Times
WGAL
Buzzfeed

Heavy.com

MSN
FactCheck.org
Slate

Buzzfeed

The Daily Dot

Complex
Buzzfeed

m March to July

Complex
Associated Press News
NBC News
Associated Press News
Fox News
The Globe and Mail
CBS News
Southend News Network
The Mercury News
Reuters
The Guardian

JUNE

JULY

AUGUST

SEPTEMBER

OCTOBER

NOVEMBER

DECEMBER

JANUARY

CATHOLA

FEBRUARY

2017
CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

75

The Mercury News

EndingtheFed.com

Commonweal Magazine
Southend News Network

Godlike Productions

Class CNBC

Heavy.com

MSN

ss Insider UK

Google Groups

Quora
Los Angeles Times

Buzzfeed

Catholic News Service

Fox News
LinkedIn Pulse

Buzzfeed
The Guardian

Truthorfiction
Earn The Necklace

TheJournal.ie
Vox
Dinar Vets

Green Valley NewsCatholic News Service

The Daily Beast

Just Plain Politics!

Commonweal Magazine

WTOE 5 News

Linkedin Pulse
Google Groups
Earn The Necklace

The Guardian

Just Plain Politics!

WTOE 5 News

Newsbreakshere.com
Bass Barn
Snopes

Dinar Vets

USA

Godlike Productions

TruthOrFiction.com

New York Magazine
Nieman Lab
The New York Times
The New York Times
Business Insider UK
Morning News USA
The Guardian

Newsbreakshere.com

Vox

SCO News
New Statesman
The Independent
TruthOrFiction.com

Hoax-Slayer

Hoax-Slayer
ThatsFake.com
Green Valley News
Every News Here
Kypo6

Hacking Humanity

Buzzfeed
The Independent
New Statesman
SCO News

Rapture Forums

Class CNBC

MSN
YouTube
Trend Network

Reddit

BuzzfeedNewsweek

RedNationRising.us
EndingtheFed.com
Quora
Los Angeles Times
WGAL

The Daily Dot
Buzzfeed

ThatsFake.com

Heavy.com

MSN
FactCheck.org
Slate

Buzzfeed
NBC News

The Daily Dot

Complex
Buzzfeed

No sharing of the story URL from March to July

Complex
Associated Press News
NBC News

CATHOLA

Associated Press News

The Guardian

Fox News
The Globe and Mail
CBS News
Southend News Network

WGAL

The Mercury News
Reuters

Buzzfeed

WGAL
MSN

WITHOUT DATE

O6

JANUARY

2016

The Guardian

The Daily Beast
FEBRUARY

MARCH

Associated Press News
APRIL

MAY

JUNE

JULY

AUGUST

SEPTEMBER

OCTOBER

NOVEMBER

DECEMBER

JANUARY

2017

YouTube

76

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

FEBRUARY

CATHO

CHAPTER 2 → RECIPE 1

SERVING SUGGESTIONS
This recipe may be used to
repurpose data obtained through
search engine results in order to
identify and follow the different
occurrences of a given fake news
story as it is cited and referenced by
different online sources, as well as
to retrace its circulation over time.
You will also be able to see when the
debunking activities took place, who
promoted them and what effect this
had on the circulation of fake news
stories and web pages.

CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

77

CHAPTER 2 → RECIPE 2

WHICH ARE THE MOST
VISIBLE SOURCES
RELATED TO A FAKE STORY?
WHEN AND BY WHOM
ARE THEY MENTIONED?

BEFORE STARTING

This recipe enables a scaling up of the approach
presented in the previous recipe, but requires a
bit more technical knowledge, as well as some
bigger datasets. In particular, you will need to
have access to:
◊◊ A web archive (we used  Radarly by
Linkfluence).
◊◊ A list of all the possible web sources in
which your chosen fake news story may
have appeared (we used the list curated by
 Le Monde Décodex).
To illustrate this recipe, we focus on a false
story that circulated during the 2017 French
presidential election and referred to the
presumed homosexuality of Emmanuel Macron.

78

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

START

define a query identifying
your fake news story

compile a list of potential sources

identify the occurrences
of your story in

websites collected
and curated
by the project
Décodex by Le Monde

Radarly

export all data
crawl the websites with

> Full text of each
occurrence
> Date of publication
> Metadata

Hyphe

search mentions
of potential sources
in the instances of your story

export a graph file
it represents
the general
connectivity
among potential
sources

Csv Rinse Repeat

export a graph file

create a basemap
network with

Gephi

Project the instances
of your story
on the basemap

b

WHICH ARE THE
SOURCES CITED IN THE
OCCURRENCES OF THE
FAKE NEWS STORY?

a

visualise

produce time-sliced
networks using

c

HOW MANY OCCURRENCES
OF THE FAKE NEWS ARE
PUBLISHED IN EACH PERIOD
OF TIME AND WHAT
SOURCES DO THEY CITE?
CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

visualise

Graph Recipes

WHAT ARE THE
MAIN SPHERES
IN THE FRENCH
MEDIA SYSTEM?

visualise

79

START

compile a list of potential sources
websites collected
and curated
by the project
Décodex by Le Monde

crawl the websites with

Hyphe

export a graph file
it represents
the general
connectivity
among potential
sources

create a basemap
network with

Gephi

visualise

a

WHAT ARE THE
MAIN SPHERES
IN THE FRENCH
MEDIA SYSTEM?

80

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 2 → RECIPE 2

a. DEFINE A BASE MAP OF NEWS
PROVIDERS

Identify (or compile) a list of all the possible
Web sources in which you think your fake
news item might have appeared (try to be as
exhaustive as possible). You can use one of the
many lists of fake news websites maintained by
debunking initiatives and combine it with a list
of mainstream media outlets.
◊◊ Identify how the sources in your list are
associated with each other through → web
crawling and hyperlink analysis. We used 
Hyphe for this.
◊◊ Visualise the resulting network and apply a
force-directed layout algorithm to identify
clusters of sources. You can use  Gephi
for this task.
◊◊ Manually highlight and name the clusters.

CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

81

WHAT ARE THE MAIN
SPHERES IN THE FRENCH
MEDIA SYSTEM?
Network analysis of the media sources
active in French public debate. The
image shows the news sources listed
by the Décodex project by Le Monde
and the hyperlinks connecting them. A
force-directed layout has been applied
to reveal the main clusters of websites
and their respective associations and
positions.

L'Yonne

GO

LOCAL MEDIA

Version Femina

Cana

EAST FRANCE MEDIA
L'Est républicain

Wanted Pe

Le Progrès
L'Alsace-Le Pays

Ouest-France

Mes Opinion
BFMTV

Le Parisien-Aujourd

Novop
Link between
providers

EXTREME-RIGHT MEDIA

Source

NUMBER OF CONNECTIONS

82

MAINSTREAM FR

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CENTRE-FRANCE
MEDIA GROUP

SOCIAL MEDIA AND OTHERS

La République du Centre

e républicaine

SATIRICAL WEBSITES

Snapchat

INTERNATIONAL MEDIA

Wikihow

OSSIP MEDIA
Instagram

US MEDIA

Facebook

Tumblr

Vice
Mashable

Reddit
The Washington Post

al+

Twitter
Bloomberg
The Guardian The New York Times

edo

YouTube

Al Kanz

BuzzFeed
The Independent

Paris Match
Dailymotion

The Daily Telegraph

Change.org
Europe 1

ns

France Inter

Debunkers de hoax

AFP
Le Figaro

Franceinfo

d’hui en France

press

Wikipédia

Slate

L'Express

Russia Today
Conscience du peuple

Les Echos

RTL

Le Monde

Lablogalupus

Blog du"Plan C", par Etienne Chouard

alalumieredunouveaumonde
Stop Mensonges
Le6emetiroir

Huffington Post
Arrêt sur images

Les Crises

Révolution Vibratoire

Reopen911
Les Moutons enragés

Marianne

Contre-Info

RUSSIAN MEDIA

Les Chroniques de Rorschach

Libération

Le Point

UK MEDIA

L'Obs

IlFattoQuotidiano.fr
Cercle des Volontaires
Les Brins d'Herbes
Agoravox

Egalité et Réconciliation

RENCH MEDIA

CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

CONSPIRACY THEORISTS

83

START

define a query identifying
your fake news story

compile a list of potential sources

identify the occurrences
of your story in

websites collected
and curated
by the project
Décodex by Le Monde

Radarly

export all data
crawl the websites with

> Full text of each
occurrence
> Date of publication
> Metadata

Hyphe

search mentions
of potential sources
in the instances of your story

export a graph file
it represents
the general
connectivity
among potential
sources

Csv Rinse Repeat

export a graph file

create a basemap
network with

Gephi

Project the instances
of your story
on the basemap

b

WHICH ARE THE
SOURCES CITED IN THE
OCCURRENCES OF THE
FAKE NEWS STORY?

84

visualise

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 2 → RECIPE 2

b. HIGHLIGHT THE OCCURRENCES OF YOUR
STORY ON THE BASE MAP

In this step we will explore how fake news stories
are associated with different sources. This is
interesting, as while a fake news story might –
for example – start out its life as a piece of satire,
as it travels it can become more prominently
associated with non-satirical sources. Here we
will identify which of the sources in the base map
of the French media system are mentioned in the
pages in which your fake news story occurs.
◊◊ Create a query that identifies the fake news
story that you want to trace. Use keywords
specifically associated with your story and
the stop-words to exclude "false positives".
◊◊ Identify the occurrences of your story,
running your query on the archive that
you have chosen to use. For each of the
results, collect the full text and the date of
publication.
◊◊ Detect, in the occurrences of the story,
mentions of the sources of your base map.
Search for for the URLs as well as for the
names of your sources (e.g. sputniknews.
com, Sputnik). In our example we used a
custom script for  CSV Rinse Repeat.
◊◊ Project the occurrences of your story onto
your base map, by connecting each of them
to the sources that they mention. While
keeping the source-nodes fixed, apply a force
directed spatialisation algorithm (you can
do this using  Gephi) to move the nodes
representing the fake story occurrences
closer to clusters of the base map that they
cite the most.

CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

85

WHICH ARE THE
SOURCES CITED IN
THE OCCURRENCES
OF THE FAKE NEWS
STORY?
Projection of the fake news occurrences
on the network of media sources. In
this image, the occurrences of the fake
news story are positioned on the base
map displayed by the previous network
according to the sources they cite. The
tendency to refer to social media is
visible as well as the relevance of Russia
Today and Sputnik International in this
particular story.

L'Yonne

Version Femina

Cana

L'Est républicain

Wanted Pe

Le Progrès
L'Alsace-Le Pays

Ouest-France

Mes Opinion
BFMTV

Link between provider
TRACKER POPULARITY
and fake news article

Le Parisien-Aujourd

Novop

Link between
providers

Source

Fake News article

NUMBER OF CONNECTIONS

86

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

La République du Centre

e républicaine
Snapchat

Wikihow
Instagram
Facebook

Tumblr

Vice
Mashable

Reddit
The Washington Post

al+

Twitter
Bloomberg
The Guardian The New York Times

edo

YouTube

Al Kanz

BuzzFeed
The Independent

Paris Match
Dailymotion

The Daily Telegraph

Change.org
Europe 1

ns

France Inter

Debunkers de hoax

AFP
Le Figaro

Franceinfo

d’hui en France

press

Le Monde

Les Chroniques de Rorschach

Lablogalupus

Blog du"Plan C", par Etienne Chouard
Libération

Le Point

alalumieredunouveaumonde
Stop Mensonges
Le6emetiroir

Huffington Post
Arrêt sur images

Les Crises

Reopen911

Révolution Vibratoire

Les Moutons enragés

Marianne

Contre-Info

Wikipédia

Slate

L'Express

Russia Today
Conscience du peuple

Les Echos

RTL

L'Obs

IlFattoQuotidiano.fr
Cercle des Volontaires
Les Brins d'Herbes
Agoravox

Egalité et Réconciliation

CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

87

START

define a query identifying
your fake news story

compile a list of potential sources

identify the occurrences
of your story in

websites collected
and curated
by the project
Décodex by Le Monde

Radarly

export all data
crawl the websites with

> Full text of each
occurrence
> Date of publication
> Metadata

Hyphe

search mentions
of potential sources
in the instances of your story

export a graph file
it represents
the general
connectivity
among potential
sources

Csv Rinse Repeat

export a graph file

create a basemap
network with

Gephi

Project the instances
of your story
on the basemap

produce time-sliced
networks using

c

HOW MANY OCCURRENCES
OF THE FAKE NEWS ARE
PUBLISHED IN EACH PERIOD
OF TIME AND WHAT
SOURCES DO THEY CITE?
88

Graph Recipes

visualise

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 2 → RECIPE 2

c. VISUALISE THE SPREAD OF YOUR FAKE
STORY ON THE BASE MAP
In this step, you will reveal how the reference
patterns identified in the previous step evolve
over time.
◊◊ Slice your network of occurrences and
references by month, by week or by day,
according to the speed of circulation of
your story. In this example we grouped
news by month and then zoomed in on
a four day window to explore the most
important period of circulation.
◊◊ While keeping the source base map stable,
visualise the different temporal slices of
fake news story occurrences.
◊◊ In order to make the changes and patterns
more legible, you can represent the fake
story occurrences not as single nodes, but
through a density heatmap (the example
has been produced using a  Graph
Recipes).

CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

89

HOW MANY
OCCURRENCES OF
THE FAKE NEWS
STORY ARE PUBLISHED
IN EACH PERIOD AND
WHAT SOURCES DO
THEY CITE?
Temporal evolution of the fake news
story in the whole observed period.
In this image, the occurrences of the
fake news story are divided in slices of
4 weeks (with an overlap of two weeks)
and represented as a density heat map
rather than as individual points. Though
mentions of the story have been present
for more than one year, its circulation
appears to spike up in February 2017,
when a new strand of the fake story is
published by the Russian website Sputnik
International.

14 MAR 2016 - 11 APR 2016

28 MAR 2016 - 25 APR 2016

11 A

6 JUN 2016 - 4 JUL 2016

20 JUN 2016 - 18 JUL 2016

4J

29 AUG 2016- 26 SEP 2016

12 SEP 2016 - 10 OCT 2016

26 S

21 NOV 2017 - 19 DEC 2016

5 DEC 2016 - 2 JAN 2017

19 D

BASEMAP
Social Media
and Others
International
Local Media
Media

Extreme
Right Media

Conspiracy
Theorists
Mainstream
French Media

Fake news density

90

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

APR 2016 - 9 MAY 2016

25 APR 2016 - 23 MAY 2016

9 MAY 2016 - 6 JUN 2016

23 MAY 2016 - 20 JUN 2016

JUL 2016 - 1 AUG 2016

18 JUL 2016 - 15 AUG 2016

1 AUG 2016 - 29 JUL 2016

15 AUG 2017 - 12 SEP 2016

SEP 2016 - 24 OCT 2016

10 OCT 2016 - 7 NOV 2017

24 OCT 2017 - 21 NOV 2017

7 NOV 2016 - 5 MAY 2016

DEC 2016 - 16 JAN 2017

2 JAN 2017 - 30 JAN 2017

16 JAN 2017 - 13 FEB 2017

30 JAN 2017 - 27 FEB 2017

CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

91

HOW MANY
OCCURRENCES OF
THE FAKE NEWS
STORY ARE PUBLISHED
IN FEBRUARY 2017
AND WHAT SOURCES
DO THEY CITE?

1 FEB 2017 - 5 FEB 2017

3 FEB 2017 - 7 FEB 2017

11 FEB 2017 - 15 FEB 2017

13 FEB 2017 - 17 FEB 2017

21 FEB 2017 - 25 FEB 2017

23 FEB 2017 - 27 FEB 2017

Temporal evolution of the fake news
story in February 2017. This image
represents a ‘temporal zoom’ of the
previous one. Here the occurrences
of the fake news story are broken up
in slices of 4 days (with an overlap of
two days).

BASEMAP
Social Media
and Others
International
Local Media
Media

Extreme
Right Media

Conspiracy
Theorists
Mainstream
French Media

Fake news density

92

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

5 FEB 2017 - 9 FEB 2017

7 FEB 2017 - 11 FEB 2017

9 FEB 2017 - 13 FEB 2017

7

15 FEB 2017 -19 FEB 2017

17 FEB 2017 - 21 FEB 2017

19 FEB 2017 - 23 FEB 2017

7

25 FEB 2017 - 1 MAR 2017

27 FEB 2017 - 3 MAR 2017

CHAPTER 2: TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB

93

CENTRE-FRANCE
MEDIA GROUP

SOCIAL MEDIA AND OTHERS

La République du Centre
L'Yonne républicaine

La République du Centre

SATIRICAL WEBSITES

Snapchat

L'Yonne républicaine

Snapchat

CHAPTER 2 → RECIPE 2

GOSSIP MEDIA

INTE

Wikihow
Wikihow

Instagram
Instagram

Facebook

20 JUN 2016 - 18 JUL 2016

4 JUL 2016 - 1 AUG 2016

Version Femina

AL MEDIA

L'Est républicain

Wanted Pedo
Al Kanz

Mashable

Twitter

5 FEB 2017 - 9 FEB 2017

BASEMAP

France Inter

Extreme
Right Media

Conspiracy
Theorists
Mainstream
French Media

Franceinfo
Fake news density

Wikipédia

Debunkers de hoax

13 FEB 2017 - 17 FEB 2017

Slate
L'Express

Le Monde
Libération

21 FEB 2017 - 25 FEB 2017

Novopress

Le Point

The Daily T

Russ
Russia Today
RUS
Conscience du peupleConscience du peuple

Wikipédia

Les Chroniques de Rorschach
Les Chroniques de Rorschach

17 FEB 2017 - 21 FEB 2017

19 FEB 2017 - 23 FEB 2017

Slate

Le Monde Lablogalupus

Lablogalupus

Franceinfo
Blog du"Plan C", par Etienne Chouard
Libération
23 FEB 2017 - 27 FEB 2017

Le Point

Novopress Arrêt sur images

25 FEB 2017 - 1 MAR 2017

Stop Mensonges

27 FEB 2017 - 3 MAR 2017

Huffington
10 OCTPost
2016 - 7
Arrêt sur images

Les Crises

Marianne
SERVING
SUGGESTIONS
Marianne Contre-Info

alalumieredunouveaum

alalumieredunouveaumonde

Blog du"Plan C", par Etienne Chouard

26 SEP
2016 - 24 OCTHuffington
2016
Le Parisien-Aujourd’hui en
France Post

Le Parisien-Aujourd’hui en France

15 FEB 2017 -19 FEB 2017

The Indep

9 FEB 2017 - 13 FEB 2017

Les Echos

Les Echos

Le Figaro

BFMTV

Social Media
and Others
International
Media

L'Express

Local Media

7 FEB 2017 - 11 FEB 2017

Change.org

RTL

11 FEB 2017 - 15 FEB 2017

BuzzFeed

The Independent

3 FEB 2017 - 7 FEB 2017

Mes Opinions
DebunkersAFP
de hoax

Le Figaro

BFMTV

The Guardian The N

Paris Match
Dailymotion

France Inter
Europe 1
Europe 1 Ouest-France
RTL
AFP

Redd

YouTube

BuzzFeed

Change.org

Mes Opinions

2016 - 2

Bloomberg

YouTube

1 FEB 2017 - 5 FEB 2017

Ouest-France

Tumblr

The Guardian The New York Times
Al Kanz

Paris Match
L'Alsace-Le Pays
Dailymotion

12 SEP 2016 - 10 OCT 2016

1 AUG
Reddit

Vice

T

Twitter

Wanted Pedo

Le Progrès

L'Alsace-Le Pays

Mashable
18 JUL 2016 - 15 AUG 2016

The Washington Post

Canal+

EDIA

Le Progrès

Tumblr

Vice

Version Femina

Canal+

L'Est républicain

Facebook

Le6emetiroir
NOV 2017

Les Crises
Reopen911

Reopen911

Stop Mensonges

Le6emetiroir

24 OCT 2017 - 2

Ré

Révolution Vibratoire

Les Moutons enragés

Moutons enragés IlFattoQuotidiano.fr
Cercle desLes
Volontaires
L'Obs
IlFattoQuotidiano.fr
Les Brins d'Herbes
Cercle des Volontaires
Agoravox
Les Brins
d'Herbes

This recipe L'Obs
may beAgoravox
used to
EXTREME-RIGHT MEDIA
Egalité et Réconciliation
identifyEgalité
which
websites reference
et Réconciliation
a fake news story most often in
different spheres. These are not
CONSPIRACY THEORI
necessarily the original sources
MAINSTREAM FRENCH MEDIA
of the fake news, but are often
the most influential media outlets
that contribute to its circulation
(whether as a rumour or as a
5 DEC 2016 - 2 JAN 2017
19 DEC 2016 - 16debunked
JAN 2017
16 JAN 2017 - 1
story).2 JAN 2017 - 30 JAN 2017
Contre-Info

94

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

Chapter 3

USING TRACKER SIGNATURES
TO MAP THE TECHNOCOMMERCIAL UNDERPINNINGS
OF FAKE NEWS SITES
Do fake news sites use
different kinds of trackers from
mainstream media sites?
How can fake news and
mainstream media sites be profiled
based on their tracker usage?
How do tracker ecologies on fake
news sites change over time?
Which other websites share
the same tracker IDs as fake
news sites?
Do trackers associated with
hyper-partisan, and misleading
information sites vary across
language spheres?

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

95

Introduction - Over the past few decades many
responses to misinformation and disinformation
have focused on mapping and debunking claims
made or repeated by politicians, journalists or
other public figures. What are the prospects of
mapping fake news online not just by looking at
the circulation of claims, but by examining the
technical infrastructures of the websites through
which these claims are published?
Many websites use “trackers” – small
bits of embedded code – in order to monitor
engagement, including visitor numbers, visitor
behaviour and the effectiveness of ads. In this
section we look at how data about web trackers
can be repurposed in order to investigate the
technical and commercial underpinnings of
websites associated with fake news and other
misleading information phenomena.
96

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 3 → RECIPE 1

DO FAKE NEWS SITES
USE DIFFERENT KINDS OF
TRACKERS FROM MAINSTREAM
MEDIA SITES?

BEFORE STARTING

For this recipe you will need two lists of
URLs: one list of fake news URLs and one list
of mainstream media URLs. How these lists
are obtained is a crucial part of the research
process. You can either draw on existing lists, or
create your own (e.g. by compiling a selection,
triangulating from other sources, or obtaining
from different platforms or media sources). The
starting point that you choose will affect how to
read and what you can do with the results. To
illustrate this recipe, we start with a selection of
fake news pages obtained from a list created by
BuzzFeed News (ordered by most engaged with
content according to the  BuzzSumo tool), as
well as a list of mainstream media web pages
obtained by triangulating lists from BuzzFeed
News and Alexa.

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

97

START
obtain seed lists
of fake news URLs

input URLs in

DMI Tracker Tracker

output data to
> Extract list of trackers
associated with web pages
on each list
> Calculate tracker usage
per site type

import data in

RAWGraphs

visualise

DO MAINSTREAM
MEDIA AND FAKE NEWS
WEBSITES SHARE THE SAME
TRACKER ECOLOGY?

98

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 3 → RECIPE 1

CALCULATE TRACKER USAGE PER
SITE TYPE

From the → source code of web pages it is
often possible to see which third-party tracking
services are used.
◊◊ Collect data about trackers associated with
the web pages on each list. You may use the
 DMI Tracker Tracker tool to collect this
information.
◊◊ Count the usage of each tracker in fake
news websites and in mainstream news
websites.
◊◊ You may use a → scatter plot to visualise
the resulting data. Each circle represents
one tracker coloured by category. On the
horizontal axis, you can show, for example,
the distribution of trackers usage by
mainstream media and fake news websites.
On the vertical axis, you can indicate the
overall usage of the tracker. We used
the  RAWGraphs tool to generate this
visualisation.

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

99

DO MAINSTREAM MEDIA
AND FAKE NEWS WEBSITES
SHARE THE SAME TRACKER
ECOLOGIES?
Scatterplot representing tracker usage
on a series of fake news and mainstream
media sites. While fake news sites and
mainstream media sites share popular
tracker services such as Google Adsense,
DoubleClick and Google Analytics,
mainstream media sites appears more
mature and sophisticated in its use of
trackers in terms of the number and
diversity of trackers that it uses.

300

Goo

Tracker Popularity

250

200
Google Publisher Tags

Aggregate Knowledge
AppNexus

150
ChartBeat

Rubicon

TradeDesk

Datalogix

TRACKER POPULARITY

1

100
Moat

Facebook Custom Audience
Omniture (Adobe Analytics)

307

Outbrain
50

TRACKER CATEGORY
Ad
Analytics

Widget

100

100%

Tracker

MAINSTREAM

20

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

DoubleClick

ScoreCard Research Beacon

Google Analytics

ogle Adsense

Facebook Connect

Google Syndication

BlueKai

LiveRamp

Quantcast

eXelate

Twitter Button

DoubleClick
Ad Exchange-Seller

Facebook Social Plugins
Gravatar

Facebook Social Graph

Wordpress Stats

RevContent
Content.ad

PulsePoint
OwnerIQ

ShareThis

Disqus
Acxiom

60

80

100%

Tracker Fakeness Index (%)

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

FAKE NEWS

40

Zypmedia

101

Google Adsense

Tracker Popularity

250

Faceb

200
Google Publisher Tags

Aggregate Knowledge

BlueKai

AppNexus

CHAPTER 3 → RECIPE 1
150
ChartBeat

Rubicon

TradeDesk

LiveRamp

Qu

eXelate

Twi

Datalogix
100
Moat

ITY

Facebook Custom Audience
Omniture (Adobe Analytics)
Outbrain

50

RY

102

MAINSTREAM

100%

20

SERVING SUGGESTIONS

40

Tracker Fakeness In

This recipe can be used to profile
the tracking practices associated
with different kinds of websites –
including which trackers are either
mainly or exclusively associated
with fake news websites and what
these trackers do – as well as
identifying most commonly used
trackers. It can also be used for
exploring the “long tail” of smaller
and more specialised trackers.

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 3 → RECIPE 2

HOW CAN FAKE NEWS
AND MAINSTREAM MEDIA SITES
BE PROFILED BASED ON
THEIR TRACKER USAGE?

BEFORE STARTING

For this recipe you will need two lists of
URLs: one list of fake news URLs and one list
of mainstream media URLs. How these lists
are obtained is a crucial part of the research
process. You can either draw on existing lists, or
create your own (e.g. by compiling a selection,
triangulating from other sources, or obtaining
from different platforms or media sources). The
starting point that you choose will affect how to
read and what you can do with the results. To
illustrate this recipe, we start with a selection of
fake news pages obtained from a list created by
BuzzFeed News (ordered by most engaged with
content according to the  BuzzSumo tool), as
well as a list of mainstream media web pages
obtained by triangulating lists from BuzzFeed
News and Alexa.

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

103

START
obtain seed lists
of fake news URLs

input URLs in

DMI Tracker Tracker

output data to
> Extract list of trackers
associated with web pages
on each list

import data in

Gephi

visualise

HOW DO FAKE NEWS SITES
AND MAINSTREAM MEDIA
CLUSTER ACCORDING THEIR
TRACKER USAGE?

104

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 3 → RECIPE 2

GRAPH RELATIONS BETWEEN PAGES
AND TRACKERS

In order to explore how different URLs share
the same patterns of tracker usage we can create
a → network graph to highlight associations
between web pages to their corresponding
trackers.
◊◊ Extract lists of trackers associated
with the initial lists of fake news and
mainstream media pages. You may use the
→ DMI Tracker Tracker tool to collect this
information.
◊◊ Create a network in order to show the
tracker usage patterns of the different
web pages. We used → Gephi in order to
visually explore the network using a
→ force directed network layout to help
read the data.
◊◊ You can annotate the network graph in
order to highlight the clusters of URLs
(e.g. fake news clusters, or mainstream
media clusters).

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

105

HOW DO FAKE
NEWS SITES AND
MAINSTREAM MEDIA
CLUSTER ACCORDING
THEIR TRACKER USAGE?
Bipartite network of trackers and
websites that use them. Shared tracker
signatures may be used to explore tracker
practices or strategies amongst a set of
websites or to detect fake news “media
groups.”

NUMBER OF CONNECTIONS

Fake news sites
Mainstream media sites
Tracker

106

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

empirenews.net +
news4ktla.com +
straightstoned.com

abcnews.com.co
newsexaminer.net

nationalreport.net
newslo.com
newsbiscuit.com
en.mediamass.net

thevalleyreport.com
thenewsnerd.com
satiratribune.com

newshub.info

celebtricity.com
empireherald.com
ncscooper.com

burradstreetjournal.com
notallowedto.com

baltimoregazzette.com

dailyfinesser.com
huzlers.com

realnewsrightnow.com
wm21news.com
stuppid.com

kata33. om

Landrypost.com

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

107

news4ktla.com +
straightstoned.com

abcnews.com.co
newsexaminer.net

nationalreport.net

CHAPTER 3 → RECIPE 2
newslo.com

newsbiscuit.
en.mediama

thevalleyreport.com
thenewsnerd.com
satiratribune.com
celebtricity.com
empireherald.com
ncscooper.com

bu

notallowedto.com

SERVING SUGGESTIONS
This recipe can be used to explore how a
set of web pages can be grouped based on
their tracker signatures. This provides a
complementary picture to lists or metrics
(e.g. of most and least used trackers across
the pages) by facilitating exploration of
relations between trackers and websites.
For example it could be used as a starting
point to identify potential fake news
“media groups” for further investigation,
or to explore the different web tracking
practices, styles and footprints of fake
news web pages – including comparisons
between pages associated with different
regions, issues or sources.

108

wm21news.com
stuppid.com

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

kata

Landry

CHAPTER 3 → RECIPE 3

HOW DO TRACKER ECOLOGIES
ON FAKE NEWS SITES
CHANGE OVER TIME?

BEFORE STARTING

For this recipe you will need the → source code
of the same web page (or set of web pages) at
two different moments in time. You can obtain
saved copies of the same page over time (e.g.
through manually or automatically saving the
source code yourself) or you can use public web
archiving projects such as the Internet Archive’s
 The Wayback Machine.

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

109

START
obtain seed lists
of fake news URLs

input URLs in

The Wayback Machine

obtain past versions
of the pages
merge all URLs in a single list

+

input URLs in

DMI Tracker Tracker

output data to
> Extract list of trackers
associated with web pages
on each list

import data in

Gephi

visualise

HOW DO FAKE NEWS SITES
ADAPT THEIR TRACKER USAGE
IN RESPONSE TO BLACKLISTING
FROM MAJOR AD NETWORKS?

110

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 3 → RECIPE 3

GRAPH RELATIONS BETWEEN PAGES
AND TRACKERS

This recipe can be used to identify which
trackers were being used by a given web page
at different moments in time. It might be useful
to chart changes in tracking practices – for
example by examining the impact and responses
to events like Google and Facebook’s bans of
fake news providers from their ads programs in
November 2016.
◊◊ Obtain archived copies of a webpage. You
may use  The Wayback Machine to see
how a given page changed over time.
◊◊ Identify associated trackers with the
current and previous version of the page.
You may use the  DMI Tracker Tracker
tool to collect such information.
◊◊ Identify the trackers which are only present
on the first date, the ones that which are
only present on the second date and the
ones that are shared across both dates.
◊◊ You can group trackers into three lists,
colouring them accordingly.

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

111

HOW DO FAKE NEWS SITES
ADAPT THEIR TRACKER
USAGE IN RESPONSE TO
BLACKLISTING FROM
MAJOR AD NETWORKS?
Tracker ecologies on fake news sites
before and after blacklisting from
major ad networks. While ad networks
from which fake news sites have been
blacklisted remain in the source code
of these sites and hence are present
in the graphic even after the moment
of blacklisting, the visualisation also
illustrates new ad networks that fake
news sites have moved to. A manual
review of ad services used to serve ads on
the website interface may help to further
refine this analysis and identify false
positives (i.e. tracker services that are
no longer in use but whose code remains
embedded in these sites).

TRACKER POPULARITY

TRACKER
TRACKER
TRACKER

TRACKER CATEGORY
Ad
Analytics
Tracker
Widget

112

Admarvel
Clicksor
Drawbridge
Facebook Custom Audience
Google Publisher Tags
Gumgum
Media.net
Sekindo
Doubleverify
Visible Measures
Omniture (Adobe Analytics)

TRACKERS PRESENT
ONLY BEFORE DECEMBER
A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

Doubleclick

Google Analytics
Google Adsense

Doubleclick Ad Exchange-seller
Google Syndication

Scorecard Research Beacon
Gravatar
Wordpress Stats
Facebook Connect
Exelate
Disqus

Brightroll
Adobe Audience Manager
Bluekai

Amazon Associates
Appnexus
Bidswitch
Criteo
Mediamath
Openx
Pubmatic
Tradedesk
Facebook Social Plugins
Taboola
Twitter Button

Adtech
Advertising.com
Index Exchange (Formerly Casale Media)
Pulsepoint
Quantcast
Rubicon
Spoutable
Stickyads
Tapad
Teads
Turn Inc.
Yahoo Ad Exchange
Tubemogul
Krux Digital
Liveramp

Adform
Infectious Media
Yahoo Ad Manager Plus

Acloudimages
Acuity Ads
Adscale
Eyeview
Revcontent
Smart Adserver
Sovrn (Formerly Lijit Networks)
Twitter Advertising
Zypmedia
At Internet
Twitter Analytics
Aggregate Knowledge
Lotame
Owneriq
Rocket Fuel
Videology
Addthis
Facebook Social Graph
Lockerz Share
Pinterest
Sharethis
Twitter Badge
Typekit By Adobe

Adap.tv
Adroll
Bidswitch
Crimtan
Datalogix
Dataxu
Digilant
Dstillery
Getintent
Improve Digital
Infolinks
Internet Billboard
Smaato
Smartclip
Spotxchange
Switch Concepts
Yieldlab
Kxcdn
Beeswax
Bidtheatre
Chango
Dotomi
Kixer
Mythings
Netmining
Pagefair
Radiumone
Sumome
Tumblr Dashboard

TRACKERS
ALWAYS PRESENT

TRACKERS PRESENT
ONLY AFTER JANUARY

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

113

Doubleclick
CHAPTER 3 → RECIPE 3

Google Analytics
Google Adsense

Doubleclick Ad Exchange-s
Google Syndication

Scorecard Research Beacon
Gravatar
Wordpress
Stats
SERVING SUGGESTIONS
Facebook
Given debates andConnect
proposals about

stopping the ad revenue of fake
Exelate
news, this recipe may be used to
Disqus
understand how fake news websites

Brightroll
are adapting to the measures taken
Adobe
Audience
by trackers
services,Manager
technology
companies
Bluekai and advertisers – as well

114

as how effective
these measures
Amazon
Associates
are.
For example, it can show which
Appnexus
trackers have been dropped, which
Bidswitch
remain and which are added at
Criteo
different moments in time.
Mediamath
Openx
Pubmatic
Tradedesk
Facebook Social Plugins
Taboola
Twitter Button

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

Adtech

CHAPTER 3 → RECIPE 4

WHICH OTHER
WEBSITES SHARE
THE SAME TRACKER IDS
AS FAKE NEWS SITES?

BEFORE STARTING

Before you start you will need to compile
or identify seed lists of fake news and other
misleading information websites. We illustrate
this recipe by examining which websites use
the same → Google Analytics IDs as a list of
websites from the EU Disinformation Review.

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

115

START
obtain seed lists
of fake news URLs

obtain the Google Analytics ID
associated to each page

input IDs in

Spyonweb

get the list of associated pages

obtain screenshots
of associated pages

categorise websites groups

visualise

WHAT MEDIA GROUP STRATEGIES
CAN BE DETECTED THROUGH
SHARED GOOGLE ANALYTICS IDS?

116

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 3 → RECIPE 4

IDENTIFY WEBSITES WHICH SHARE
TRACKER IDS WITH A SEED LIST OF
PAGES OR SITES

This recipe can be used to identify which other
websites share the same tracker IDs as web
pages on a given list.
◊◊ Extract the Google Analytics ID for each
URL in your starting list. You can do this
manually (e.g. by looking in the
→ source code for a string in the form
“UA-xxxxxxx”) or automatically through
→ web scraping or other tools (in this
example we wrote a custom script in
order to extract this information from the
metadata of the website).
◊◊ Obtain a list of pages associated with the
same ID. We used the → API of
 Spyonweb.com to get this information.
◊◊ Take a screenshot of each web page. We
used a script to automate the process of
obtaining screenshots, in order to visually
compare the different websites to identify
different kinds of media groups.
◊◊ Place together screenshots of pages with
the same ID to spot differences and
similarities between websites across and
within groups.

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

117

WHAT MEDIA GROUP
STRATEGIES CAN BE
DETECTED THROUGH
SHARED GOOGLE
ANALYTICS IDS?
A selection of websites which share
the same Google Analytics IDs, based
on seed list from EU Disinformation
Review. This illustrates the diversity of
online settings where claims labelled
as Russian disinformation are shared
– from large media groups such as
Russia Today, to themed clusters (e.g.
military or mysticism), and geographical
clusters (e.g. Canadian). One can also
identify distinctive visual styles and
possible shared → CMS features
amongst different websites in these
clusters, which may be used as the
basis for further investigations into the
media, publication and communication
strategies of websites associated with
misleading information online.

rt.com

xryshaygh.com

19 disinformation stories

1 disinformation story

UA-5773642

UA-4839940

Media group
(Russia Today)

Lone
Webmaster

assange.rt.com

asco.gr

doc.rt.com

royalparadise.gr

learnrussian.rt.com

discoverthassos.com

russian.rt.com

thassosinn.gr

catalog.rt.com
118

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

defensenews.com

newcoldwar.org

almanach.cz

1 disinformation story

7 disinformation stories

3 disinformation stories

UA-841082

UA-15942468

UA-3004323

Themed network
(Navy, Airforce)

Canadian socialism
and unions

Themed network
(Mysticism, Liberland)

armedforcesjournal.com

oakvillendp.ca

grand-mystical-lodge.com

sightlinemediagroup.com

socialiststudies.com

malachim.cz

militarytimes.com

ndpsocialists.ca

liberlandpress.com

airforcetimes.com

ccu-csc.ca

illuminati-journal.com

marinecorpstimes.com
CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

oldm.cz
119

rt.com

xryshaygh.com

defensenews.com

19 disinformation stories

1 disinformation story

1 disinformation story

UA-5773642

UA-4839940

UA-841082

Media group
(Russia Today)

Lone
CHAPTER 3
→ RECIPE 4
Webmaster

Themed network
(Navy, Airforce)

assange.rt.com

asco.gr

armedforcesjournal.com

doc.rt.com

royalparadise.gr

sightlinemediagroup.com

SERVING SUGGESTIONS
This recipe may be used in the
discoverthassos.com
service of expanding a group of
fake news web pages – in order
to derive lists of other websites
which share the same tracker IDs.
It may also be used to provide
russian.rt.com
thassosinn.gr
context to the
digital strategies
and “media groupings” of fake
news providers

learnrussian.rt.com

catalog.rt.com

120

militarytimes.com

airforcetimes.com

marinecorpstimes.com

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 3 → RECIPE 4

DO TRACKERS
ASSOCIATED WITH HYPERPARTISAN AND MISLEADING
INFORMATION SITES VARY ACROSS
LANGUAGE SPHERES?

BEFORE STARTING

For this recipe, you will need lists of
fake news, hyper-partisan or misleading
information sites in different language
spheres in order to compare their trackers and
tracking practices. We illustrate this recipe
with reference to hyper-partisan, fake news
and misleading information sites in Dutch,
English and German language spheres.

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

121

START
obtain seed lists
of fake news URLs

input URLs in

DMI Tracker Tracker

output data to
> Extract list of trackers
associated with web pages
on each list

compare trackers lists with

DMI Triangulation Tool

visualise

DO MISINFORMATION AND
HYPER-PARTISAN WEBSITES
IN DIFFERENT LANGUAGE
SPHERES HAVE DISTINCT
TRACKER ECOLOGIES?

122

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 3 → RECIPE 5

IDENTIFY TRACKERS PER LANGUAGE
SPHERE

◊◊ Extract trackers associated with lists of the
web pages for each language sphere. We did
this using the  DMI Tracker Tracker tool.
◊◊ Identify the trackers which are shared across
and which are unique to different languages
spheres within the dataset. We did this using
the  DMI Triangulation tool.
◊◊ You can illustrate the results using the visual
metaphor of magnets. Each of the three
languages are represented on the corner of a
triangle. The trackers are distributed in the
triangle according to their usage: if a tracker
is used by all three languages it will appear in
the middle, if it is used by two languages the
tracker will be placed on the edge between
the two and so on.

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

123

DO MISLEADING
INFORMATION AND
HYPER-PARTISAN
WEBSITES IN DIFFERENT
LANGUAGE SPHERES
HAVE DISTINCT TRACKER
ECOLOGIES?

veeseo
Mark & Mini
Ligatus
Atlas

DUTCH

AdLantic

Platform161
SkimLinks
LinkShare

Visualisation of tracker ecologies
associated with hyper-partisan or
misleading information sites across
three language spheres. While
popular ad and widget services such
as DoubleClick, Google Adsense and
Facebook Connect are shared across
language spheres, unique services per
language sphere may also be detected.
For example, trackers associated with
the Russian-language focused Mail.
ru Group are only found in the set of
websites associated with the German
language sphere.

TRACKER POPULARITY

1

78

TRACKER CATEGORY
Ad
Analytics
Tracker

INFOnline
Plista
Piwik
Tynt
adNET de
VKontakte Widgets
AMP Platform
M P NewMedia

GERMAN
Widget

124

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

Facebook
Connect

Media
Innovation
Group
DoubleClick
Ad Exchange-Seller
Gravatar
DoubleClick
Wordpress
Google Adsense
Stats
Google
Twitter Button
Syndication

SumoMe
ENGLISH
Google
Tag Manager

First Impression

ChartBeat Content ad
Spoutable
Distroscale
Zedo

Google
Analytics

CHAPTER 3: USING TRACKER SIGNATURES TO MAP THE TECHNO-COMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

125

CHAPTER 3 → RECIPE 5

Facebook
Connect
Media
Innovation
Group
DoubleClick
Ad Exchange-Seller
Gravatar
DoubleClick
Wordpress
Google Adsense
Stats
Google
Twitter Button
Syndication

SumoMe
ENGLISH
Google
Tag Manager
ChartBeat Content
Spoutable
Distroscale
Zedo

Google
Analytics

SERVING SUGGESTIONS
This recipe can be used to
identify trackers for further
investigation – including
language sphere specific and
cross-language trackers. It
may help to provide lines
of inquiry for looking into
what is distinctive about the
commercial and technical
underpinnings of fake news in
different language spheres.

ts

126

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

Chapter 4

STUDYING POLITICAL
MEMES ON FACEBOOK

How can meme spaces on
Facebook be traced?
How may the content of memes
be studied?
How do memes frame political
and media events?

Introduction - So far the recipes in this guide
have focused on media artefacts which mimic
the news genre. But successful hyper-partisan
content, misinformation, disinformation and
propaganda do not always look and feel like
news pages with the familiar combination of
headlines, pictures and text that we see in sites
like the BBC, CNN and countless other outlets.
In fact images, and particularly image-based
memes, circulate just as well (if not better) in
social media ecosystems.
According to a piece for the Columbia
Journalism Review, the media format which
generated most engagement on Breitbart’s
Facebook page is the image-meme [1]. Hence,
it is essential to consider not just how fake
news pages but also other kinds of viral content
genres such as memes participate in political
agenda setting, political processes and political
culture.

This section provides a set of approaches to
investigate political memes. We focus on memes
that take political topics, actors and events as
their object. The case study used to illustrate
these approaches is alt-right and pro-Trump
memetic activity on Facebook around the 2016
US presidential election. We shall use the term
"memetic activity" in this section to designate
the multiple ways in which users act around
memes online, including circulating, imitating
and transforming them. The first recipe focuses
on how to identify and map meme spaces on
Facebook. The second recipe explores ways to
investigate how Facebook users engage with
political events through memetic activity.The
third and final recipe provides a
series of approaches for analysing
the content of memes.
[1] See, Nausicaa Renner,
“Memes Trump articles
on Breitbart’s Facebook
page”, Columbia Journalism
Review: 2017. Available at:
http://www.cjr.org/tow_
center/memes-trump-articles-on-breitbarts-facebook-page.php

CHAPTER 4 → RECIPE 1

HOW CAN MEME
SPACES ON FACEBOOK
BE TRACED?

BEFORE STARTING

To investigate who engages with memes
around a political issue of interest on
Facebook, you should start by identifying a
page with significant following and memetic
activity around your topic of interest. As an
example, we selected the Disdainus Maximus
page, which is very active in pro-Trump and
alt-right activity. We traced the network of
connections around this page and explored its
topology.

130

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

START
Choose a Facebook page
with memetic activity

trace the network of connections with

Netvizz

extract page like network (depth 2)

import data in

Gephi

Quantitative Analysis
- InDegree
- OutDegree
- Betweenness Centrality
- Netviz Fan Count Metrics

Qualitative Analysis
- Prominent clusters identification
- Cluster thematic annotation
- Exploration of content and
self-description of pages

visualise

a
WHAT ISSUES ANIMATE
THE INTER-LIKED FACEBOOK
PAGE NETWORK SEEDED
BY A PRO-TRUMP POLITICAL
MEME REPOSITORY?

CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

131

START
Choose a Facebook page
with memetic activity

trace the network of connections with

Netvizz

extract page like network (depth 2)

import data in

Gephi

Quantitative Analysis
- InDegree
- OutDegree
- Betweenness Centrality
- Netviz Fan Count Metrics

Qualitative Analysis
- Prominent clusters identification
- Cluster thematic annotation
- Exploration of content and
self-description of pages

visualise

a
WHAT ISSUES ANIMATE
THE INTER-LIKED FACEBOOK
PAGE NETWORK SEEDED
BY A PRO-TRUMP POLITICAL
MEME REPOSITORY?

132

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 4 → RECIPE 1

IDENTIFY THE NETWORK OF TIES
AROUND A CHOSEN FACEBOOK PAGE
To trace the network of affinities around a
Facebook page we followed the “likes” from
our page to other pages (to be distinguished
from the "likes" received from users).

◊◊ A Facebook crawler may be used to
extract the “likes” network around a page.
We used  Netvizz’s “page like network”
module to “create a network of pages
connected through the likes between
them.”[1]
◊◊ We set the crawler to a depth of two
to extract the pages liked by our seed
page and those liked by them. We thus
obtained a directed network file where
nodes are pages and edges represent acts
of liking.

[1]
See Netvizz, Facebook
application, version:
1.3, 2017, at: https://
apps.facebook.com/
netvizz/

◊◊ You may use a network analysis tool
such as  Gephi to examine the graph
of interconnected Facebook pages. A
force-directed layout algorithm (such as
ForceAtlas2) can help you visualise the
shape of the network and explore the
interconnected space of memetic activity.

CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

133

START
Choose a Facebook page
with memetic activity

trace the network of connections with

Netvizz

extract page like network (depth 2)

import data in

Gephi

Quantitative Analysis
- InDegree
- OutDegree
- Betweenness Centrality
- Netviz Fan Count Metrics

Qualitative Analysis
- Prominent clusters identification
- Cluster thematic annotation
- Exploration of content and
self-description of pages

visualise

a
WHAT ISSUES ANIMATE
THE INTER-LIKED FACEBOOK
PAGE NETWORK SEEDED
BY A PRO-TRUMP POLITICAL
MEME REPOSITORY?

134

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 4 → RECIPE 1

PROFILE THE ISSUES AND THEMES THAT
ANIMATE THE MEMETIC SPACE

This step consists of qualitative and quantitative analysis
of the composition and arrangement of the network
of Facebook pages obtained in the previous step. The
configuration of the → network graph may analysed:
- Quantitatively, by:
◊◊ Identifying which pages are most popular in the
network by using a graph metric such as indegree, i.e.
the count of the likes received from other pages in the
network.
◊◊ Identifying the pages most active in liking other pages
by using a graph metric such as outdegree, i.e. the
count of likes given to other pages in the network.
◊◊ Identifying which pages bridge or connect different
clusters in the network by using a graph metric such
as betweenness centrality.
◊◊ Identifying which pages are most popular among
Facebook users by using the Facebook’s “fan count”
metric.
- Qualitatively, by:
◊◊ Identifying prominent clusters by visually exploring
the shape and density of nodes groupings in the
graph.
◊◊ Examining the content shared by the pages in the
network as well as their titles and self-descriptions to
identify shared issues of concern within each cluster.
To increase readability of the network map you may
annotate it with the resulting thematic classification
of the clusters.

CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

135

WHAT ISSUES ANIMATE
THE INTER-LIKED
FACEBOOK PAGE
NETWORK SEEDED BY A
PLURALIST APOLITICAL MEMES
PRO-TRUMP POLITICAL
Vaporwave
MEME REPOSITORY?
memes
Network of inter-liked Facebook pages
seeded from the pro-Trump meme
repository Disdainus Maximus. The
network comprises 751 pages featuring
memetic activity. Pages are connected
through ties representing likes among
them. The network is spatialised with a
force-directed layout algorithm. Another
algorithm, modularity, is used to identify
clusters. Nodes are sized according to the
number of likes received and coloured
according to their follower count. ProTrump memes are prominent within the
Facebook meme culture, as shown by the
fact that even the pages which are not
politically oriented share ties with pages
circulating pro-Trump memes (as well as
with pages dedicated to various forms of
nationalism and populism). The presence
of Donald Trump’s official page at the
center of the network may indicate that
his image plays an energising role in the
alt-right meme space.

Internet Backroom
META III
Apocalypse
Controversial Humour Hipster Fucking bitch META Penisbearcats
The Hulkamaniac Juggalo Brony Bro Army
CЯIMEWAVE
Requiem For A Meme
ゆがんだm̵ͤ͌͞e̐ͯ̄ͩ̇̀͡m̊̀ͩͬ̽̚͞e̷ͫ̃͑ͧͤ̍̒s̵̄̍ͣ̀ Top Ramen Memes
Aesthetic 超現実的な Therapy
Stick memes
Dammit meme i hate u but da pussy game ridiculous
Playing Runescape and memeing hard
A I R S H I P 我吸- 首
I fucking hate science
Gam3cubeSlut
͡ʖ
KING QUARTZ 石絵王 ͜ʖ blvckinternet千年2003
Birds tho
Million Dollar Extreme
Your Memes 貴方のミーム
Innawoods Operating Waifu
Balls to the Wall Hardcore Radical Memes
Trashbin
Shit Memes
>Implying video games are fun
Vapor Art
Ｖａｐｅｙｗａｖｅ
>greentext
Wat2Hek?
Laughing Colours 2
Hey u ok do u like cum?
Ｒｅｄｅｍｐｔｏｒｉｓ釁 ＞Implying humor is funny
Organised
Crime
Memes
Colorful
Ｍｅｍｅｓ
ＬＳＤ：ＭｅｍｅＥｍｕｌａｔｏｒ
It's called a meme you dip 2
Anime profile pics are for Losers
dadpatrol
Do Androids Dream of Electric Memes?
>Implying music is good
Laughing Colours 笑って色

Full house memes 2

Penisbearcats

Nigga you just went full plebeian

Hardcore epic shoobie dog meme's
Augmentations
Laughing Colours
Kermit de la Frog: The Mean Green Meme Machine
Special meme fresh
Nihilist Memes 4chong page that actually posts stuff from
Magenta Memes

cummingwhilecrying
Youtube Snapshots
Skrillex is overrated and stuff: Cha
Avant-garde memes from China - 中文的
The Cringe Channel

Difficulty II
fresh and slunky meme's
What Memes May Cum III
This is not a Meme Page
Minimalistic Memes

Everything Is A Social Construct

God sa

Exploding Fish Shitposting and Senseless Drivel, Inc.
Ancient Mask Memes
Ｔｅｒ ｇｉｆ ｉｃ

Classical Art Memes
Spicy Spanish memes

Jammin' Japanese
Memes
Tenacious
Teutonic Me

Edgy Egyptian Meme
Utterblax
Prestigious P
Tough Templar Memes

Violent Viking Memes
Ostentatious Austro-Hungarian Memes
Proud Pythagorean Memes
Outrageous Ottoman Memes

Amazin

Rough Roman Mem

Crazy Cool Celtic Memes
Crafty Conquistador Memes

Marketable Mayan Memes
Sensational Scythian

IMPERIAL MEMES

136

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

Unironic Trump activism
Establishment populism

RIGHT WING POPULIST
(MEME) ACTIVISM

White pride
Dr. Ben& Candy Carson
The Hill

Diamond And Silk
Make America Great Again Memes
Drudge Report

God Emperor Trump
Daily Trump Memes
Milo Yiannopoulos

Smash Cultural Marxism

Breitbart
NRA Institute for Legislative Action

American White History Month 2
Letters From White South Africa

Donald J. Trump
Being Classically Liberal

Patri-archie Comics

Politically Incorrect - /pol/

White People World Wide 1

I'm white

Nigel Farage

The Daily DfP

Génération Identitaire

Nordic Beauty
This is the Netherlands
Justnationalistgirlsღ
This is Italy
This is Germany

This is Europa

m halfchan

apter II

The Traditionalist
Architectural Revival

Disdain for Plebs

ave our gracious meme

Arktos

Traditionalist Western Art

Communism Kills
Disdainus Maximus

European
nationalism

The Golden One
The Patriarchy

Daily Reminder

This is Sweden
This is Ireland

A Handbook of Traditional Living

The Straight, White, Capitalist

WESTERN

Earl of Grey
UKball
USABall
POLANDBALL

NATIONALISM

Deus Vult
Countryball

Duchy of Burgundyball

emes

es

Cute Confederate Memes

Prussian Memes

ng American Memes

mes

AMOUNT OF FOLLOWERS

n Memes

0

26,368,463

NUMBER OF LIKES BY OTHER
PAGES
0

CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

61

137

CHAPTER
4 → APOLITICAL
RECIPE 1
PLURALIST
MEMES

Vaporwave
memes

RIGH
(MEM

Internet Backroom
META III
Apocalypse
Controversial Humour Hipster Fucking bitch META Penisbearcats
The Hulkamaniac Juggalo Brony Bro Army
Requiem For A Meme
Top Ramen Memes

CЯIMEWAVE
ゆがんだm̵ͤ͌͞e̐ͯ̄ͩ̇̀͡m̊̀ͩͬ̽̚͞e̷ͫ̃͑ͧͤ̍̒s̵̄̍ͣ̀
Aesthetic 超現実的な Therapy

Full house memes 2

Stick memes
Dammit meme i hate u but da pussy game ridiculous
Playing Runescape and memeing hard
A I R S H I P 我吸- 首
I fucking hate science
Gam3cubeSlut
͡ʖ
KING QUARTZ 石絵王 ͜ʖ blvckinternet千年2003
Birds tho
Million Dollar Extreme
Your Memes 貴方のミーム
Innawoods Operating Waifu
Balls to the Wall Hardcore Radical Memes
Trashbin
Shit Memes
>Implying video games are fun
Vapor Art
Ｖａｐｅｙｗａｖｅ
>greentext
Wat2Hek?
Laughing Colours 2
Hey u ok do u like cum?
Ｒｅｄｅｍｐｔｏｒｉｓ釁 ＞Implying humor is funny
Organised Crime Memes
Colorful
Ｍｅｍｅｓ
ＬＳＤ：ＭｅｍｅＥｍｕｌａｔｏｒ
It's called a meme you dip 2
Anime profile pics are for Losers
dadpatrol
Do Androids Dream of Electric Memes?
>Implying music is good
Laughing Colours 笑って色

Penisbearcats

Nigga you just went full plebeian

Hardcore epic shoobie dog meme's
Patri-archie Comi
Po
Augmentations
Laughing Colours
Kermit de la Frog: The Mean Green Meme Machine
Special meme fresh
Nihilist Memes 4chong page that actually posts stuff from halfchan
Magenta Memes
cummingwhilecrying
Youtube Snapshots
Skrillex is overrated and stuff: Chapter II
Avant-garde memes from China - 中文的
The Cringe Channel

Difficulty II
fresh and slunky meme's
What Memes May Cum III

SERVING SUGGESTIONS
This is not a Meme Page
Minimalistic Memes

Everything Is A Social Construct

This recipe may be used
to identify political meme
repositories on Facebook and
the relations between them as
well as the themes that lend
themselves to memefication.

Daily
God save our gracious meme

Exploding Fish Shitposting and Senseless Drivel, Inc.
Ancient Mask Memes
Ｔｅｒ ｇｉｆ ｉｃ

Classical Art Memes
Spicy Spanish memes

Duchy of Bu

Jammin' Japanese
Memes
Tenacious
Teutonic Memes

Edgy Egyptian Memes Cute Confederate Mem
Utterblax
Prestigious Prussian Memes
Tough Templar Memes

Violent Viking Memes
Ostentatious Austro-Hungarian Memes
Proud Pythagorean Memes
Outrageous Ottoman Memes

Amazing American Memes

Rough Roman Memes

Crazy Cool Celtic Memes
Crafty Conquistador Memes

Marketable Mayan Memes
Sensational Scythian Memes

IMPERIAL MEMES

138

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 4 → RECIPE 2

HOW DO MEMES
FRAME POLITICAL
AND MEDIA EVENTS?

BEFORE STARTING

To identify how memes frame political or
media events the first step is to identify the
events to be examined. To illustrate this
recipe we used the following key events
associated with the US elections:
Date

Description

March 3, 2016

The eleventh Republican debate

June 3, 2016

Trumps’ Mexican judge remark

September 9, 2016

Clinton’s ‘basket of deplorables’
comment

October 8, 2016

Trump’s taped comments about
women

October 29, 2016

Podesta emails

January 27, 2017

Trump announces first travel ban

CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

139

START
Choose a set
of Facebook pages
with memetic activity

Make a list of relevant
political events

Netvizz

extract photos timeline metadata
(including image URLs)
for the analysed timeframe

get images URLs

input images URLs in

DownThemAll!
or

Tab Save

create a corpus of images

Create an Image Montage with

visualise

visualise

Image J

140

a
HOW DO MEMETIC REACTIONS
TO DIFFERENT POLITICAL
EVENTS COMPARE?
b
HOW DO MEMETIC REACTIONS
AROUND A SINGLE POLITICAL
EVENT COMPARE ACROSS
FACEBOOK PAGES?
A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 4 → RECIPE 2

IDENTIFY MEMES RELATED TO EVENTS
To illustrate this recipe we use a sub-selection
of 46 pages from the corpus identified in recipe
4.1. The selection was done based on a set of
qualitative and quantitative criteria, including
their → engagement counts and the thematic
cluster that they belong to.
◊◊ Define a timeframe for meme selection.
For this analysis, we selected three days
starting with the date of the event under
examination.
◊◊ Extract a list of memes posted to your
corpus of pages in the selected timeframe.
You may use a tool such as  Netvizz
to collect a list of images and related
metadata.
◊◊ Download the images for each URL. You
may use a browser extension
such as  Tab Save for Chrome or
 DownThemAll! for Firefox.
◊◊ Visually juxtapose the images grouping
them by event to enable comparison of
memetic reactions across events. You may
use an image montage tool such as
 ImageJ (4.2.a).
◊◊ You can also explore how different
pages react to a single event to enable
comparison across pages (4.2.b).
CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

141

HOW DO MEMETIC
REACTIONS TO
DIFFERENT POLITICAL
EVENTS COMPARE?

Eleventh Republican
debate; blood coming out
of her… wherever
3rd of March 2016

Trumps' Mexican Judge
remark
3rd of June 2016

Image montage of memetic activity on
selected Facebook pages around six
key events in the 2016 US presidential
campaign. Memetic activity around
different events may be compared in
terms of scale and framing.

142

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

Basket of Deplorables
comment

Trump's Taped Comments
About Women

Podesta emails

Election

11th of September 2016

8th of October 2016

29th of October 2016

9th of november 2016

CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

143

Facebook Pages

HOW DO MEMETIC
REACTIONS AROUND
A SINGLE POLITICAL
EVENTS COMPARE
ACROSS FACEBOOK
PAGES?
Image montage of a particular
interpretative frame prominent in
memetic activity around Trump’s
taped comments about women across
a set of pro-Trump Facebook pages.
The visualisation shows how a number
of pages pointed to a particular image
of Bill Clinton confronted with alleged
victims of sexual harassment from his
own past in response to the “groping
tape” event. Notably, this interpretative
frame appears consistent with the frame
proposed by Breitbart.

FIRST

ae911truth
alexanderemerickjones
americaonenationundergod
apac1787
breitbart
brushfires
dailytrumpmemes
donald trump all the way to the white house 2016
donaldtrump
get
globalpublicalert
guardiansoffreedom.com.au
infowarcoalition
infowars
marinelepen
myiannopoulos
nigelfarageofficial
oneblgmeme
polandball
politicalcorrectnessgonewild
politicallyincorrectpol
postingdanktrumps
rightsrevoked
sarahpalin
specialairserviceball
straightpatriotman
succfucczucccucc
thealexjoneschannel
theantimedia
thearcanefront
thedailydfp
thepatriotfederation
trollsfortrump
trumpmillennials
trumppoliticalmovement
truthaboutdonaldtrump
wakeupandreclaimamerica
wearechange.org
worldtruthtv

8 OCTOBER

A 01:55 (GMT)

B

truthaboutdonaldtrump

02:21

C

02:28

politicalcorrectnessgonewild

IMAGES POSTED
on each Facebook page

IMAGES CONTAINING
BILL CLINTON over time
A

B

C

I

IMAGES ENGAGEMENT

J 05:10

04:30

K

05:11

GET

thealexjoneschannel

2,000 >
1,500 - 2,000
1,000 - 1,500
500 - 1,000
100 -500
< 100

P 11:28

Q

wakeupandreclaimamerica
144

14:39

R

16:40

GET

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

a

S

E

D

H K

M

N

Q

R

L
T

C

LAST
V

B

I J
F
U

G
O

A

P

FIRST

9 OCTOBER

D 03:18

10 OCTOBER

E 03:34

GET

L

05:22

M 06:46

16:52

16:52

infowars

CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

G 04:05

trumpmillennials

O

GET

T

alexanderemerickjones

03:43

theantimedia

N 10:34

guardiansoffreedom.com.au

S

F

dailytrumpmemes

H 04:23

GET

11:12

wakeupandreclaimamerica

U

18:23

thedailydfp

V 21:23

LAST

politicalcorrectnessgonewild
145

Judge

Basket of Deplorables
comment

Trump's Taped Comments
About Women

Podesta emails

Election

11th of September 2016

8th of October 2016

29th of October 2016

9th of november 2016

S

E

D

H K

M

N

Q

R

L
T

C

LAST
V

B

I J
F
U

G
O

A

P

FIRST

9 OCTOBER

02:28

D 03:18

10 OCTOBER

E 03:34

GET

L

05:22

M 06:46

S

16:52

alexanderemerickjones

146

03:43

theantimedia

N 10:34

guardiansoffreedom.com.au

16:40

F

dailytrumpmemes

O

GET

T

16:52

infowars

G 04:05

trumpmillennials

H 04:23

GET

11:12

wakeupandreclaimamerica

U

18:23

thedailydfp

V 21:23

LAST

politicalcorrectnessgonewild

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 4 → RECIPE 2

SERVING SUGGESTIONS
This recipe may be used to
explore participatory production
of visual culture around political
events and how it contributes to
agenda-setting efforts.

CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

147

CHAPTER 4 → RECIPE 3

HOW MAY THE
CONTENT OF MEMES
BE STUDIED?

BEFORE STARTING

The recipe illustrates techniques to support
content analysis of texts and images
contained in memes as well as the detection
of genres or styles of memetic activity.
As examples, we use two Facebook pages
which feature pro-Trump memes: Breitbart
and God Emperor Trump. Even if the images
posted to these pages do not comply with
classic meme formats (e.g. image macros),
they exhibit memetic features such as
virality, user-driven remixing, imitation and
intertextuality. Breitbart has been selected
due to its central role in animating the altright culture[1]. The God Emperor Trump page
is one of the most popular pro-Trump, altright meme pages with over 245,000 likes
and → followers as of the time of writing.
In Breitbart only the page administrator can
post images, while in God Emperor Trump
users may submit their productions to the
administrator for posting.

148

[1]
Y., Faris, R., Roberts,
H., & Zuckerman,
E. (2017, March 3).
Study: Breitbart-led
right-wing media ecosystem altered broader
media agenda. Retrieved March 8, 2017,
from http://www.cjr.
org/analysis/breitbart-media-trump-harvard-study.php

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

get images URLs

START

input pages in

Choose one or more
Facebook pages with
memetic activity

Netvizz

input images URLs in

DownThemAll!
or

Tab Save
create a corpus of images

analyse corpus with

Google Vision API

Use optical character
recognition (OCR)
to extract the text
contained within
each image

computational linguistic analysis

visualise

Cortext

visualise

filter images according
to computational
linguistic analysis

CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

a
WHAT THEMES LEND
THEMSELVES TO MEMETIC
ACTIVITY ON BREITBART’S
FACEBOOK PAGE?
b
CAN WE DETECT DISTINCT
VISUAL STYLES WITHIN
A MEME REPOSITORY?
149

get images URLs

START

input pages in

Choose one or more
Facebook pages with
memetic activity

Netvizz

input images URLs in

DownThemAll!
or

Tab Save
create a corpus of images

analyse corpus with

Google Vision API

Use optical character
recognition (OCR)
to extract the text
contained within
each image

computational linguistic analysis

visualise

Cortext

visualise

filter images according
to computational
linguistic analysis

150

a
WHAT THEMES LEND
THEMSELVES TO MEMETIC
ACTIVITY ON BREITBART’S
FACEBOOK PAGE?
b
CAN WE DETECT DISTINCT
VISUAL STYLES WITHIN
A MEME REPOSITORY?
A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 4 → RECIPE 3

CREATE A CORPUS OF IMAGES POSTED
TO THE CHOSEN FACEBOOK PAGE AND
THEIR ASSOCIATED METADATA

To create a corpus choose a timeframe of interest
(for instance the days around a particular political
or media event - see recipe 4.2) or use all images
posted to a Facebook page.
◊◊ The Facebook → API enables the extraction
of metadata associated with images posted to
a page and available via the “Photos” tab.
◊◊ Metadata capture may be done with a data
extraction tool such as  Netvizz, using the
“page timeline images” module.
◊◊ The outcome of running  Netvizz’s “page
timeline images module” is a tab-separated
file containing metadata associated with each
image, including its creation date, its URL
and its "likes", reactions and comment count.
◊◊ A browser extension such as  Tab Save for
Chrome or  DownThemAll! for Firefox may
be used to download the images.

[2]
See memespector script
written by Bernhard
Rieder, University of
Amsterdam, at https://
github.com/bernorieder/
memespector

◊◊ To extract the text contained within each
image you may run the images through
some optical character recognition (OCR)
software. For this recipe, we used
 Google's Vision API and a script feeding
the list of image URLs to the Vision API[2]].
◊◊ You may also use a piece of image analysis
software to generate additional metadata
for your corpus of images through the
detection and labelling of entities, objects
and attributes.

CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

151

get images URLs

START

input pages in

Choose one or more
Facebook pages with
memetic activity

Netvizz

input images URLs in

DownThemAll!
or

Tab Save
create a corpus of images

analyse corpus with

Google Vision API

Use optical character
recognition (OCR)
to extract the text
contained within
each image

computational linguistic analysis

visualise

Cortext

visualise

filter images according
to computational
linguistic analysis

152

a
WHAT THEMES LEND
THEMSELVES TO MEMETIC
ACTIVITY ON BREITBART’S
FACEBOOK PAGE?
b
CAN WE DETECT DISTINCT
VISUAL STYLES WITHIN
A MEME REPOSITORY?
A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 4 → RECIPE 3

EXAMINE THEMES EXPLOITED IN
THE CORPUS OF MEMES WITH TEXT
ANALYSIS (EXPERIMENTAL)

To examine the issues that trigger memetic
activity you may analyse the text extracted from
the images through manual qualitative analysis
or semi-automated semantic analysis. The results
of this analysis are affected by the quality of the
OCR. A computational linguistics tool (e.g.
 CorText) can be used, but the analyst’s
judgement remains crucial to evaluate the
relevance of the extracted terms and to set the
parameters of the tool (what algorithms to use,
what types of words to keep, how frequently
should they occur, etc.).
◊◊ Lexical analysis may help you identify the
most relevant terms used in your memes.
◊◊ You may also run queries on the OCR
outputs to explore the resonance of
particular issues.
◊◊ Analysis of term co-occurrences across the
corpus of memes allows you to explore the
main themes within the corpus and their
relationships. The network of term cooccurrence may be visually explored in order
to identify prominent thematic clusters and
the relationships between them and identify
how key political issues are articulated
through associations of terms in memes.

CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

153

WHAT THEMES LEND
THEMSELVES TO
MEMETIC ACTIVITY
ON BREITBART’S
FACEBOOK PAGE?

Michelle Obama

hous
r

SINGLE

Ted Cr
REPUBLICAN

Do

Network of nouns and adjectives cooccurring in images posted in 2016 on
Breitbart’s Facebook page (two words
are connected if they are present in the
same image). Colors identify clusters
according to Louvain modularity.
The two most prominent clusters are
centered around Donald Trump (top,
blue) and Hillary Clinton (bottom,
yellow). One may examine the terms
present in the Hillary Clinton cluster and
in clusters in its proximity in terms of
framing and agenda setting.

NOUNS

ADJECTIVES

POLITICAL
AWKWARD

GOOD

national security

Obama c

RICH

SPECIAL

YOUR

DEAL

Clinton cash

CO-OCCURRENCES
of nouns and adjectives
in images

right side of hist
GLOBAL

taliban supporter

CLUSTERS
according to Louvain
modularity

TERRORIST

fake news
FAKE

DONALD
TRUMP
CLUSTER

154

HILLARY
CLINTON
CLUSTER

TOLERANT
international business times
other dictators
mayor arm deals
clintonfoundation dource
conservative media

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

most specific images for
the Hillary Clinton cluster

se and senate
republican caucuses
marco rubio
north carolina

ruz

most specific images for
the Donald Trump cluster

CORRUPT
PRIMARY
STRONGER
WHOLE
show the whole world
millions of irredeemable deplorables
33,000 deleted emails
breitbart posts

onald Trump

BETTER

MINIMUM
minimum wage

YOUNG

care

LIBERAL

OWN

PROUD
FIRST
CRIMINAL
criminal fbi investigation

NEW

BLACK

tory
PRESIDENTIAL
MORE
presidential candidate
SAFE

VIOLENT

FREE

Hillary Clinton
Clinton foundation

T
FOREIGN
MAYOR

Algeria Saudi Arabia
Oman Algeria Saudi
foreign dollars
donated millions

OTHER
american families

CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

155

get images URLs

START

input pages in

Choose one or more
Facebook pages with
memetic activity

Netvizz

input images URLs in

DownThemAll!
or

Tab Save
create a corpus of images

analyse corpus with

Google Vision API

Use optical character
recognition (OCR)
to extract the text
contained within
each image

computational linguistic analysis

visualise

Cortext

visualise

filter images according
to computational
linguistic analysis

156

a
WHAT THEMES LEND
THEMSELVES TO MEMETIC
ACTIVITY ON BREITBART’S
FACEBOOK PAGE?
b
CAN WE DETECT DISTINCT
VISUAL STYLES WITHIN
A MEME REPOSITORY?
A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 4 → RECIPE 3

EXAMINE VISUAL STYLES OF MEMES
WITH IMAGE ANALYSIS

To explore the visual styles deployed in a
meme repository, the outputs of the image
analysis software described in step 4.2.a may
be used, namely the labels or tags generated to
describe the entities and attributes detected in
our image corpus. We illustrate this analysis
on the God Emperor Trump page.
◊◊ Images are analysed with Google Vision
to extract tags describing the images and
textual content.
◊◊ We used  CorText to examine
associations between images based on
shared labels.
◊◊ The configuration of the → network
graph may be visually explored in order
to identify and describe visual styles per
cluster.
◊◊ This operation may be repeated with
different pages to compare their different
style.

CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

157

CAN WE DETECT
Cluster A
DISTINCT VISUAL
STYLES WITHIN A MEME
REPOSITORY?

COMMUNITY

ECOSYSTEM

PEOPLE

WATERCRAFT

AUDIENCE

ESTATE

TEAM

PALACE

CROWD

SELFIE

YOUTH

SOCIAL GROUP

LAWN

PROTEST

LANDMARK

Network of images in the God Emperor
Trump corpus, linked by similarity of
meme profile, according to shared tags.
The fuzzy boundaries between clusters
show that memes share a similarity of
composition, which may be associated
with the memetic logic. Some genres
or styles of memetic activity may be
detected. A screenshot-based meme
cluster may be noticed as well as a
comics and cartoons cluster, including
Pepe the Frog.

Cluster B
SPEECH
PERSON
PROFESSION
NEWS
SOLDIER
CAR
SPEAKER
VEHICLE
AUTOMOTIVE DESIGN
TROOP
MODEL
NEWSCASTER
CONVERSATION
AUDIENCE

Cluster C
SPEECH
ENERGY
PRESENTATION

MEMES

Cluster D

NEWS CONFERENCE

NOSE

BRAND

FACE

PERSON

MOUTH
HEAD
HAIRSTYLE
HAIR
VISION CARE
GLASSES

SIMILARITY of meme profile
according to shared tags

FACIAL HAIR
SKIN
JAW

TAGS

FINGER
FACIAL EXPRESSION
DOG

CLUSTERS
memes that share
a similarity of composition

ORGAN

Cluster E
PRESETNATION
SENSE
MAN
BANNER
CAP
EDUCATION
BRAND
CONVERSATION
SEA
CHILD
IMAGE
PLAN
INTERACTION
EMOTION
MUSCLE

158

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

Cluster N
POSTER
ALBUM
MUSICAL THEATRE
ACTION FILM
PERFORMING ARTS
TRAFFIC SIGN

SCREENSHOT
ANIME
IMAGE
VEHICLE
COMICS
ILLUSTRATION

Cluster M
COMIC BOOK

MIDNIGHT

SCREENSHOT

DISCO

FICTION

RELIGION

CARTOON

ACTION FIGURE

PC GAME

MANGAKA

CONVENTION

EMOTION

COSTUME

ILLUSTRATION

ANIME

Cluster L
ILLUSTRATION

PRIMATE

CARTOON

LIGHT

COMICS

HALLOWEEN

ART

LANTERN

CLIP ART

MAP

SKETCH

GAMES

PUMPKIN

MYTHOLOGY

TOY

Cluster I
PRODUCT
WEB PAGE
DIAGRAM
LINE
FONT
TEXT
LOGO
PLAN

Cluster H
DOCUMENT
WEB PAGE

Cluster F

LABEL

HUMAN ACTION

FOOD

INTERIOR DESIGN

TEXT

BIOLOGY

AMPHIBIAN

HUMAN BODY

LINE

ORGAN

PATTERN

DESSERT

FONT

DISH

BRAIN

LUNCH

BRAND

FROG

CUISINE

CONDOMINIUM

BIOLOGY

Cluster G
SHAPE

COLOR

LINE

IMAGE

FONT

WEB PAGE

TEXT

BRAND

CHAPTER 4: STUDYING POLITICAL MEMES ON FACEBOOK

159

RICH
OWN
YOUR

PROUD
FIRST
CRIMINAL

DEAL

CHAPTER 4 → RECIPE
NEW3

criminal fbi investigation

Clinton cash
BLACK
right side of history
taliban supporter

GLOBAL

PRESIDENTIAL
MORE
presidential candidate

TERRORIST

VIOLENT

SAFE
FREE

fake news
FAKE

Hillary Clinton
Clinton foundation

TOLERANT
FOREIGN

international business times
MAYOR
other dictators
mayor arm deals
clintonfoundation dource
conservative media

Algeria Saudi Arabia
Oman Algeria Saudi
foreign dollars
donated millions

OTHER
SERVING
SUGGESTIONS

american families

Cluster D

This recipe may be used
to explore how memes
frame political issues,
events and personalities,
in what may be considered
a form of “participatory
propaganda”.[3]

NOSE
FACE
MOUTH
HEAD
HAIRSTYLE
HAIR
VISION CARE
160

GLASSES

[3] See, Alicia Wanless, “Have You Fallen
Down a Participatory Propaganda Rabbit
Hole?”, Politcs Means Politics, at: https://
politicsmeanspolitics.com/have-you-fallen-down-a-participatory-propaganda-rabbit-hole-6f71c83f04fa

Cluster E

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

PRESETNATION

Chapter 5

MAPPING
TROLL-LIKE PRACTICES
ON TWITTER
How may we detect Twitter
accounts which negatively target
political representatives?
How may we characterise the
sources of troll-like activity?
How may troll-like practices be
characterised?

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

161

Introduction - Tactics such as trolling and the
use of bots and “sock-puppet” accounts have been
linked to the spread of political disinformation
and propaganda online [1]. In the lead up to the
2017 general elections for the Dutch parliament,
journalists pointed to the use of sock puppets (i.e.
false online identities assumed to deceive and
influence opinion) by some political parties to
amplify their messages online and to attack their
political opponents on social media [2].
162

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

In this recipe set we provide some methods
that can be employed to detect and profile
misleading information practices by taking trolllike behaviour around the 2017 Dutch election
campaign as a case study. We focus on political
troll-like practices, a term which we use in the
narrower sense of attacks addressed at political
representatives.
We focus on three aspects of political
trolling: the sources of troll-like activity, the
characteristics of these practices and their targets.
[1] See, for example, Alice
Marwick and Rebecca Lewis,
“Media Manipulation And
Disinformation Online”,
Data Society, May 2017:
https://datasociety.net/
output/media-manipulation-and-disinfo-online/
[2] See, Andreas Kouwenhoven
and Hugo Logtenberg,
“Hoe Denk met ‘trollen’
politieke tegenstanders
monddood probeert te
maken”, NRC, 2017. Available at: https://www.nrc.
nl/nieuws/2017/02/10/
de-trollen-van-denk6641045-a1545547

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

163

164

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 5 → RECIPE 1

HOW MAY WE DETECT
TWITTER ACCOUNTS WHICH
NEGATIVELY TARGET POLITICAL
REPRESENTATIVES?
BEFORE STARTING

To detect political troll-like activities
and their sources, you should start from
compiling a list of potential targets, for
instance a set of Twitter accounts associated
with political representatives. We used a set
of 28 Twitter accounts associated with the
leaders of parties participating in the 2017
Dutch general elections.

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

165

START
Make a list
of potential targets

query Twitter accounts with

TCAT

get all tweets that
@mention them

Select most
active “mentioners”
per target

Examine mentions
(qualitative analysis)

Select top negative
“mentioners” among
the most active ones

visualise

a
HOW ARE ATTACKS DISTRIBUTED
ACROSS THE POLITICAL SPECTRUM
IN THE NETHERLANDS?
166

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 5 → RECIPE 1

IDENTIFY WHO NEGATIVELY TARGETS
POLITICAL LEADERS ON TWITTER

To identify who negatively mentions a political
leader on Twitter start from identifying all
references of a political leader account on Twitter.
Following Twitter practices, you can operationalise
references with "mentions" defined by the “@” sign.
◊◊ Capture all @mentions of a set of accounts.
We used a Twitter data extraction and analysis
toolset called  TCAT.
◊◊ We capture all tweets mentioning at least one of
the 28 political leader Twitter accounts between
8 February and 8 March 2017, the month before
the Dutch general elections. The result is a
collection of 519,245 tweets which we use in all
the recipes in this set.
◊◊ To identify the most active “mentioners”, find
all the accounts mentioning one of the target
accounts more than a given threshold. In our
example, we retained the accounts mentioning
one of the political leaders more than 100 times
in our dataset.
◊◊ Examine mentions by most active “mentioners”
identified in the previous step to qualify the
nature of each of their references (i.e. whether it
is in support of the political leader or negatively
targeting them).
◊◊ Retain only the users who consistently
negatively target one or more political
representatives (in our case we narrowed down
our list to 25 users).

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

167

HOW ARE ATTACKS
DISTRIBUTED ACROSS
THE POLITICAL
SPECTRUM IN THE
NETHERLANDS?
Visualisation of Dutch political leaders
who are targets of positive and negative
mentions on Twitter (from users who
mention them more than 100 times
in a one-month period). Red circles
represent users launching attacks and
green circles represent users making
favourable mentions. The size of the
circle represents the total number
of users mentioning a party leader.
The asymmetry of the distribution of
targets of troll-like behaviour across the
political spectrum is notable as left-wing
politicians are most often targeted by
negative mentions.

@sybrandbuma (CDA)

@thierrybaudet (Forum voor Democratie)

@

MENTIONS

@JacquesMonasch (Nieuwe Wegen)

Each dot represents
one mention

MENTION TYPE
Attack
Not Attack

168

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

@MinPres (VVD)

@markrutte (VVD)

@mariannethieme (PvdD)
@ncilla (Pirate Party)
@geertwilderspvv (PVV)

@LodewijkA(PvdA)
Gertjansegers(CU)
@HenkKrol (50plus)

@emileroemer (SP)

@APechtold (D66)
@LavieJanRoos (VNL)

@Tunahankuzu (DENK)

@keesvdstaaij (SGP)

@jesseklaver (GroenLinks)

@jndkgrf (GeenPeil)

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

@KleinNorbert (VZP)

169

@MinPres (VVD)

@markrutte (VVD)

@mariannethieme (PvdD)
@ncilla (Pirate Party)
@geertwilderspvv (PVV)
@sybrandbuma (CDA)

@LodewijkA(PvdA)
Gertjansegers(CU)
@HenkKrol (50plus)

et (Forum voor Democratie)
@emileroemer (SP)

@APechtold (D66)
@LavieJanRoos (VNL)
@Tunahankuzu (DENK)

@keesvdstaaij (SGP)

quesMonasch (Nieuwe Wegen)

@jesseklaver (GroenLinks)

@jndkgrf (GeenPeil)

170

@KleinNorbert (VZP)

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 5 → RECIPE 1

SERVING SUGGESTIONS
This recipe can be used to
identify sources of personal
attacks on Twitter and can be
extended beyond the context of
political trolling.

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

171

CHAPTER 5 → RECIPE 2

HOW MAY WE
CHARACTERISE THE SOURCES
OF TROLL-LIKE ACTIVITY?

BEFORE STARTING

For this recipe we take as a starting point
the 25 accounts that mention at least one
political leader at least 100 times identified
in the previous recipe (we discarded one
account because it was no longer active).

172

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

START
Make a list
of accounts

use

get profile information

Twitter API’s
GET friends/ids

extract the IDs of
all users followed by
the trolling accounts

search profile pictures with

get metadata

Google Image Search

Twitter API

examine IDs with

Twitter API’s
GET users/lookup

identify fake images

get creation
date of profiles

Friends
metadata

convert to network with

visualise

Table2Net

explore network with

Gephi

visualise

a
HOW CAN WE
CHARACTERISE
SOURCES OF TROLLING
ACTIVITY BASED ON
THEIR PROFILE
INFORMATION?

b
HOW CAN WE CHARACTERISE
SOURCES OF TROLL-LIKE
ACTIVITY BASED ON THEIR
SHARED FRIENDS?

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

173

START
Make a list
of accounts

use

get profile information

Twitter API’s
GET friends/ids

extract the IDs of
all users followed by
the trolling accounts

search profile pictures with

get metadata with

Google Image Search

Twitter API

examine IDs with

Twitter API’s
GET users/lookup

identify fake images

get creation
date of profiles

retrieve friends
metadata

visualise

convert to network with

Table2Net

a
HOW CAN WE
CHARACTERISE
SOURCES OF TROLLING
ACTIVITY BASED ON
THEIR PROFILE
INFORMATION?

explore network with

Gephi

visualise

b
HOW CAN WE CHARACTERISE
SOURCES OF TROLL-LIKE
ACTIVITY BASED ON THEIR
SHARED FRIENDS?

174

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 5 → RECIPE 2

a. CHARACTERISE SOURCES OF TROLLLIKE ACTIVITY THROUGH THEIR
PROFILE INFORMATION

One way in which you can characterise the
sources of troll-like activity is by examining
their profile information.
◊◊ Visit each of the accounts and collect their
profile information from the Twitter
interface (description, profile picture and
banner).
◊◊ Analyse this information in order
to identify political issues, hashtags
mentioned and affiliations.
◊◊ Take note of the presence or absence of
profile images and upload any images
identified to  Google Image Search to
detect whether any of the accounts use
fake profile images.
◊◊ Using the Twitter → API, extract the
creation date of the account and examine
whether several accounts in your corpus
have been created around the same date.

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

175

HOW CAN WE
CHARACTERISE
SOURCES OF TROLLING
ACTIVITY BASED
ON THEIR PROFILE
INFORMATION?
Clustering of 24 accounts engaging
in troll-like activity around the Dutch
elections. The profile information is
clustered according to similarities.
Three users have very similar profiles
and are created in a short amount of
time: this helps us to identify them
as ‘sock-puppet’ account created for
trolling activities. Other six promote the
same anti-islam agenda, but without
being fake accounts.

CREATION DATE

PROFILE PICTURE

17/12/2009
4/6/2010
9/9/2010
‘real’ photos

22/5/2011
24/10/2011
2/9/2012

nationalistic

4/3/2013
15/5/2013
14/7/2013
12/11/2013

animals

25/4/2014
23/8/2014
12/2/2015
15/3/2016

political content

17/6/2016
17/10/2016
9/11/2016
28/11/2016

others

8/12/2016
13/12/2016
13/1/2017
8/2/2017
26/6/2017
(unknown)

176

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

ELEMENTS FOR CHARACTERIZATION ANALYSIS

@name
Description profile
 joined

BANNER

BIO DESCRIPTION

@D***er
BOER, Economist, Father of six, Doctor/MBA Patriotically correct.
Romantically conservative, islamophobe, aspirant lid PVV, lid FvD

TROLL
OR NOT TROLL?

?

@M***te
(no bio description)

@H***an
(no bio description)

?

@l***dy
#de-islamize #stoppolicor #stopterror#voteMarine
#voteWilders#minderminderminder #lesslessless#maroccons

?

@M***00
Ben fascist noch racist,maar realist.Links is het nieuwe rechts en
blank het nieuwe zwart.Stop Brussels EU omvolking,Stop massa
migratie,Stop Rutte & Timmermans

?

@A***rt
Een kern van (mijn) waarheid overgoten met sarcasme, sadisme en
galgenhumor. Neem niets persoonlijk op, maar neem het wel op!

?

@g***91
Twitter Censors Everything, their Board members are Izlamic.
MOVE to Gab.ai for true FREE SPEECH FOR ALL
http://gab.ai/ger2519 #PVV #MEGA #MHGA #BanIslam

?

@Y***NL
(account suspended)

?

@j***33
PVV Geert Wilders Nexit, Trump Trooper MAGA, Love Le Pen Vive
la France MEGA, Anti Islam, De-islamization Forever
@j***33
Geert Wilders PVV MEGA Nexit De-Islamization worldwide, Love Le
Pen Superwoman, Trump MAGA, Close borders, Anti islam, Anti EU,
Jail Obama, Jail Hillary...
@k***33
Geert Wilders Nexit, Vive Le Pen, Trump MAGA, AfD Germany,
Brexit, Pro Israël,
, Anti EU, De-islamization...

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

177

START
Make a list
of accounts

use

get profile information

Twitter API’s
GET friends/ids

extract the IDs of
all users followed by
the trolling accounts

search profile pictures with

get metadata with

Google Image Search

Twitter API

examine IDs with

Twitter API’s
GET users/lookup

identify fake images

get creation
date of profiles

retrieve friends
metadata

visualise

convert to network with

Table2Net

a
HOW CAN WE
CHARACTERISE
SOURCES OF TROLLING
ACTIVITY BASED ON
THEIR PROFILE
INFORMATION?

explore network with

Gephi

visualise

b
HOW CAN WE CHARACTERISE
SOURCES OF TROLL-LIKE
ACTIVITY BASED ON THEIR
SHARED FRIENDS?

178

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 5 → RECIPE 2

b. CHARACTERISE SOURCES OF
TROLLING ACTIVITY THROUGH
THEIR FRIENDS

Another way in which sources of trolllike activity may be characterised is by
examining who they follow on Twitter, also
known as their friends.
◊◊ Use the "GET friends/ids" function of
the Twitter API to extract the IDs of all
users followed by the trolling-accounts
you have identified.[1]
◊◊ Use the "GET users/lookup" function
of the Twitter API to retrieve the
profile information for each friend.[2]
◊◊ Use  Table2Net to convert the table
containing the studied accounts and
their friends in a network of Twitter
accounts connected by the “friendship”
relationships between them.
◊◊ You may use a network analysis and
visualisation tool such as  Gephi to
explore the shared friends or followees
between the accounts engaging in trolllike practices.

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

[1] See Twitter Developer Platform,
API reference: GET
friends/ids, 2017,
at: https://developer.
twitter.com/en/docs/
accounts-and-users/
follow-search-get-users/api-reference/getfriends-ids
[2] See Twitter Developer Platform,
API reference: GET
users/lookup, 2017,
at: https://developer.
twitter.com/en/docs/
accounts-and-users/
follow-search-getusers/api-reference/
get-users-lookup

179

HOW CAN WE
CHARACTERISE
SOURCES OF TROLLLIKE ACTIVITY BASED
ON THEIR SHARED
FRIENDS?
Visualisation of shared friends or
followees of 24 accounts engaging
in trolling activity around the Dutch
elections. The density of connections in
the network shows that accounts share
multiple followees. Multiple accounts
which may be described as right-leaning
based on their profile information
(description, picture and banner) are
present at the core of the network thus
confirming earlier findings pertaining to
the right/left asymmetry of sources and
targets of attacks.

FOLLOWERS IN THE SUB
NETWORK
61

1

ACCOUNT TYPE
Accounts engaging
in negative targeting
Political journalists
Anti-EU accounts
Other accounts
Politician - Party for
Freedom
Politician - Christian
Democratic Appeal
Politician - For the
Netherlands
Other politicians

180

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

181

22/5/2011
24/10/2011
2/9/2012

nationalistic

@H***an
(no bio descript

4/3/2013
15/5/2013

@l***dy
#de-islamize #
#voteWilders#m

14/7/2013
12/11/2013

animals

25/4/2014

@A***rt
Een kern van (m
galgenhumor. N

23/8/2014
12/2/2015
15/3/2016

@M***00
Ben fascist noch
blank het nieuw
migratie,Stop R

@g***91
Twitter Censors
MOVE to Gab.a
http://gab.ai/ge

political content

17/6/2016

@Y***NL
(account suspen

17/10/2016
9/11/2016
28/11/2016
8/12/2016
13/12/2016
13/1/2017
8/2/2017
26/6/2017
(unknown)

182

others

@j***33
PVV Geert Wild
la France MEGA

@j***33
Geert Wilders P
Pen Superwoma
Jail Obama, Jail

@k***33
Geert Wilders N
Brexit, Pro Isra

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 5 → RECIPE 2

SERVING SUGGESTIONS
This recipe may be used to
profile trolling accounts in
the context of other political
campaigns.

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

183

CHAPTER 5 → RECIPE 3

HOW MAY
TROLL-LIKE PRACTICES
BE CHARACTERISED?

BEFORE STARTING

For this recipe, take as a starting point the
accounts identified in the previous recipes
and the tweets posted from these accounts in
the timeframe of the study (see recipe 5.1).

184

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

START

make a list of
accounts engaging
in negative targeting

capture tweets posted
by these accounts with

TCAT

extract hashtags
by their frequency

extract hosts
by their frequency

a
WHAT ISSUES
ARE PRESENT
IN TWEETS
@MENTIONING
A POLITICAL
CANDIDATE?

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

visualise

visualise

visualise

examine hosts
(qualitative analysis)

examine hashtags
(qualitative analysis)

b
WHAT MEDIA SOURCES
ARE PRESENT IN TWEETS
@MENTIONING
A POLITICAL CANDIDATE?

c
HOW IS URL POSTING
DISTRIBUTED ACROSS
THE USERS ENGAGING
IN NEGATIVE TARGETING?
185

START

make a list of
accounts engaging
in negative targeting

capture tweets posted
by these accounts with

TCAT

extract hashtags
by their frequency

extract hosts
by their frequency

a
WHAT ISSUES
ARE PRESENT
IN TWEETS
@MENTIONING
A POLITICAL
CANDIDATE?

186

visualise

visualise

visualise

examine hosts
(qualitative analysis)

examine hashtags
(qualitative analysis)

b
WHAT MEDIA SOURCES
ARE PRESENT IN TWEETS
@MENTIONING
A POLITICAL CANDIDATE?

c
HOW IS URL POSTING
DISTRIBUTED ACROSS
THE USERS ENGAGING
IN NEGATIVE TARGETING?

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 5 → RECIPE 3

INVESTIGATE THE HASHTAGS THAT
SOURCES OF TROLL-LIKE ACTIVITY USE

To identify what issues are associated with
troll-like activity, examine the hashtags that are
used in tweets that mention a politician posted
by the users that frequently engage in negative
targeting.
◊◊ Rank hashtags by their frequency using the
“hashtag frequency” feature of  TCAT.
◊◊ Manually analyse the most frequently used
hashtags to identify issues that animate
activities of users engaged in trolling
practices.

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

187

WHAT ISSUES ARE
PRESENT IN TWEETS
@MENTIONING A
POLITICAL CANDIDATE?

@emileroemer
SP
PVV

AFA

Islam

→ Bubble graph of issues expressed

through hashtags in tweets @mentioning
candidates in the 2017 Dutch elections
posted by the set of 24 accounts
engaging in troll-like activity. Issues are
coloured by type, sized by frequency of
occurrences and grouped according to the
candidate mentioned in the tweet which
contains them. Most tweets with hashtags
mention the right-wing populist candidate
Geert Wilders. Most prominent are issues
related to PVV’s political message (“Nexit”,
“StopIslam” and “BanIslam”) as well as
those pertaining to expressions of Dutch
patriotism. Generally speaking, we can
conclude that right-wing politicians receive
mainly support from “troll-like users,”
while other politicians are the targets of
attacks (as discussed in recipe 5.1).

@sybrandbuma
stempvv Nexit
PVV2017
CDA
Islam

@LodewijkA

MuslimBan
PvdA

D66

PVV

GL
destillebeving

@APechtold
radio1

pensioendief

VNL

Pechtold
Roemer

RTLdebat

FREQUENCY HASHTAG

Buma

EU

stempvv

ISLAM

pvv
TK17

D66

PVDA
Klaver

GL

@MarkRutte
Extreme right wing politics

Election

Other political parties/leaders

Anti EU

Pvv
StemPVV
Dutch

Media

188

Statement

Islam

Foreign politics/events

Dutch patriotism

EU

Issues

Not (yet) classified

penj

nexit
oprutten

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

@geertwilderspvv

ChanceForChange

FN

referendum
GrenzenDicht

BanIslam
islam

DeIslamiseren

brexit
EUexit

kamergotchi
WNL

Buitenhof

tijdvoorMAX

fakenews

EXITEU

zodus

Nexit

stemwijzer

luistereenkeer

penj
jinek

FreePeterSweden

VoteLeave
Ozturk

stopislam

trump

ouderenzorg

muslimban
DeIslamize
bankoran

15maart

hoedan
kominverzet

EU

TK17
CDA

isgeertnogveilig

Dutch

VVD

Netherlands

PVDA

isGeertwelveilig

VOTEPVV

stemPVV

NederlandWeerVanOns

OVC16
Wilders

GeertWilders

PVVop1

Deport

PVV

PVV2017

wijstaanachterGeert

@SylvanaSimons
@jndkgrf

@thierrybaudet
@briefjevanjan

@jesseklaver
@mariannethieme
PvdD

marokkaanse
ISLAM

CDA SP

Pechthold

PVV
stempvv
Roemer

Islam

@laviejanroos
pvv

VNL
PVV

RTLdebat
d66

GL

keesvdstaaij
fvd

@HenkKrol

VVD
Jinek

@tunahankuzu
@JacquesMonasch

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

189

START

make a list of
accounts engaging
in negative targeting

capture tweets posted
by these accounts with

TCAT

extract hashtags
by their frequency

extract hosts
by their frequency

a
WHAT ISSUES
ARE PRESENT
IN TWEETS
@MENTIONING
A POLITICAL
CANDIDATE?

190

visualise

visualise

visualise

examine hosts
(qualitative analysis)

examine hashtags
(qualitative analysis)

b
WHAT MEDIA SOURCES
ARE PRESENT IN TWEETS
@MENTIONING
A POLITICAL CANDIDATE?

c
HOW IS URL POSTING
DISTRIBUTED ACROSS
THE USERS ENGAGING
IN NEGATIVE TARGETING?

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CHAPTER 5 → RECIPE 3

INVESTIGATE THE MEDIA SOURCES
SHARED BY THE ACCOUNTS ENGAGED
IN NEGATIVE TARGETING

To identify the content shared in tweets posted
by the users engaged in troll-like activities,
examine the URLs inserted in their tweets.
◊◊ Use the “hosts frequency” feature of
 TCAT to extract the media sources
ranked by frequency of occurrence.
◊◊ Manually analyse the most frequently used
media sources to determine their profile.
◊◊ Analysis of URL sharing behaviour across
the set of users may be used as a means to
detect troll-like activity.

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

191

WHAT MEDIA
SOURCES ARE
PRESENT IN TWEETS
@MENTIONING
A POLITICAL
CANDIDATE?

OTHER MEDIA
libelletv.nl

Venn diagram of most resonant media
sources in tweets @mentioning
candidates in the 2017 Dutch elections
posted by the set of 24 accounts
engaging in trolling activity. The most
tweeted source is the Dutch alt-right blog
fenixx.org followed by the anti-islam site
Jihad Watch and the right-wing think
tank Gatestone Institute.

POLITICAL PARTY
briefjevanjan.nl

geenpeil.nl

forumvoordemocratie.nl

AMOUNT OF URLS

11

192

378

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

P

PUBLIC BROADCASTER
NEWS

wnl.tv

nu.nl
nos.nl

elsevier.nl

dailymail.co.uk

telegraaf.nl

nieuws.tpo.nl

rtlnieuws.nl
ad.nl

nrc.nl

ANTI-ISLAM
gatestoneinstitute.org
unitedwithisrael.org

clarionproject.org

PRO-ISRAEL
jihadwatch.org

ejbron.wordpress.com

worldisraelnews.com

fenixx.org
dagelijksestandaard.nl

kudtkoekiewet.nl
joostniemoller.nl

ALT-RIGHT
breitbart.com

tpo.nl

DUTCH-RIGHT
CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

193

HOW IS URL POSTING
DISTRIBUTED ACROSS
THE USERS ENGAGING
IN NEGATIVE
TARGETING?
→ Network graph of distribution

of URLs shared across the 24 users
engaging in negative targeting of
politicians in the month before the
Dutch elections. The graph shows two
users to be responsible for the majority of
URLs posted in the studied timeframe.

joop.nl
npo.nl
volkskrant.nl
gasbaten.nl

nporadio1.nl

conservativehome.com

zn

rtlnie

trouw.nl

fd.nl

OCCURRENCES

independent.co.uk

1

3,643

krapuul.nl

MENTION TYPE
Websites
Twitter Users

194

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

brugesgroup.com
educate-yourself.org viid.me
news.wespeakhealthy.com

vanews.co.vu
925.nl

truthfeed.com
billionbibles.org
agendaofevil.com

express.co.uk

pamelageller.com

periscope.tv

geenpeil.nl

themuslimissue.wordpress.com

briefjevanjan.nl
forumvoordemocratie.nl
libelletv.nl
facebook.com

dailymail.co.uk

liefdevoorholland.com
nieuws.nl

krone.at

youtube.com

joostniemoller.nl
linkis.com
breitbart.com
dagelijksestandaard.nl
m.telegraaf.nl
nos.nl

barneveldsekrant.nl
tacticalinvestor.com
bild.de
bnr.nl
nl.wikipedia.org

elsevier.nl
telegraaf.nl

instagram.com
foxnews.com

politiek.tpo.nl
kudtkoekiewet.nl
opiniez.com

euws.nl

wnl.tv

nrc.nl

ad.nl
nu.nl

mkbbelangen.nl
twitlonger.com

nieuws.tpo.nl

rt.com

metronieuws.nl
mathildedoethetnuook.blogspot.nl
worldisraelnews.com

fenixx.org
shr.gs
pauwnieuws.nl
ejbron.wordpress.com
clarionproject.org
media.tpo.nl
jihadwatch.org
gatestone.eu
gatestoneinstitute.org
verenoflood.nu
unitedwithisrael.org
zeepertje.com

dw.com

regio.tpo.nl

bilderbergmeetings.org
eo.nl
vice.com
npofocus.nl
epochtimes.de

nl.gatestoneinstitute.org
conservativedailypost.com
parool.nl
thegatewaypundit.com
gatesofvienna.net
limburger.nl
dailywire.com
westmonster.com

partijvoorsoest.nl
hit-radio.nl
newobserveronline.com

boilingpoints.wordpress.com

huffingtonpost.com

CHAPTER 5: MAPPING TROLL-LIKE PRACTICES ON TWITTER

195

ANTI-ISLAM
gatestoneinstitute.org
unitedwithisrael.org

@geertwilderspvv

RO-ISRAEL

brugesgroup.com

Buitenhof

truthfeed.com

EXITEU

zodus

volkskrant.nl
jinek

pamelageller.com

Nexit

periscope.tv

stemwijzer

luistereenkeer

penj

15maart

hoedan

EU
geenpeil.nlTK17
Netherlands
CDA
briefjevanjan.nl
Dutch
VVD
forumvoordemocratie.nl
PVDA
libelletv.nl
isGeertwelveilig
facebook.com
kominverzet

isgeertnogveilig

fenixx.org

themuslimissue.wordpress.com

dagelijksestandaard.nl
VOTEPVV

mPVV

NederlandWeerVanOns

OVC16

Wilders

dailymail.co.uk

PVVop1
liefdevoorholland.com
GeertWilders
Deport
znieuws.nl

conservativehome.com

krone.at

youtube.com

SERVING SUGGESTIONS
joostniemoller.nl
linkis.com
PVV

dagelijksestandaard.nl

PVV2017

elsevier.nl
telegraaf.nl
@SylvanaSimons

@jndkgrf

@thierrybaudet

@jesseklaver

@briefjevanjan

@mariannethieme

marokkaanse
ISLAM

GL

PvdD

CDA SP

Pechthold

PVV
stempvv

Roemer

keesvdstaaij

fvd

Islam

@laviejanroos

pvv

VNL

PVV

RTLdebat
d66

breitbart.com

barneveldsekrant.nl
tacticalinvestor.com
bild.de
bnr.nl
nl.wikipedia.org

This recipe can be used to profile
instagram.com
foxnews.com
politiek.tpo.nl
the issues
and
media
sources
rt.com
kudtkoekiewet.nl
opiniez.com
metronieuws.nl
mathildedoethetnuook.blogspot.nl
associated
with
social
media
rtlnieuws.nl
worldisraelnews.com
kudtkoekiewet.nl
wnl.tv
nieuws.tpo.nl
ad.nl
nrc.nl
accounts engaging
in trollfenixx.org
joostniemoller.nlshr.gspauwnieuws.nl
nu.nl
like behaviour –ejbron.wordpress.com
building
up a
clarionproject.org
media.tpo.nl
jihadwatch.org
mkbbelangen.nl
gatestone.eu
trouw.nl
richer picture of these
activities
twitlonger.com
gatestoneinstitute.org
verenoflood.nu
unitedwithisrael.org
and their context. If you
are
zeepertje.com
fd.nl
dw.com
exploring
personal accounts you
regio.tpo.nl
nl.gatestoneinstitute.org
conservativedailypost.com
parool.nl
thegatewaypundit.com
should
make
sure
to
consider
bilderbergmeetings.org
gatesofvienna.net
eo.nl
limburger.nl
vice.com
npofocus.nl
thetpo.nl
potential legal
andwestmonster.com
ethicaldailywire.com
epochtimes.de
independent.co.uk
implications of your inquiry,
partijvoorsoest.nl
hit-radio.nl
boilingpoints.wordpress.com
newobserveronline.com
and ensure
that public-interest
huffingtonpost.com
arguments are weighed against
privacy considerations. You
may consider whether to focus
on networks and relationships
between of a number of accounts,
and whether to remove or redact
personally identifying information.
m.telegraaf.nl
nos.nl

wijstaanachterGeert

l.nl

jihadwatch.org

billionbibles.org
agendaofevil.com

express.co.uk

EUexit

fakenews

tijdvoorMAX

aten.nl

brexit

kamergotchi

islam

DeIslamiseren

FreePeterSweden

VoteLeave
Ozturk

BanIslam

WNL

FN

ejbron.wordpress.com

GrenzenDicht

worldisraelnews.com
npo.nl

vanews.co.vu
CHAPTER 5 → RECIPE
3

trump

925.nl

referendum

joop.nl

stopislam

educate-yourself.org viid.me
news.wespeakhealthy.com

ChanceForChange

ouderenzorg

muslimban
DeIslamize
bankoran

clarionproject.org

@HenkKrol

VVD
Jinek

@tunahankuzu

@JacquesMonasch

DUTCH-RIGHT

Conclusion
Glossaries
Contributors and
acknowledgements

197

198

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CONCLUSION

Fake news, it can safely be said, is not a neglected issue.
Every day it seems as though a new newspaper article, blog
post, research report or project is released on the subject,
and more and more academic articles are produced to
reconnect public debates to scholarly literature. Over the
course of this project we have been in touch with media
organisations, journalists, civil society groups, public
institutions, companies, researchers and students from
around the world eager to understand, investigate, address
and study to this issue
Amidst this intense public debate and mediatisation,
concerns have been raised about the term “fake news”, and
suggestions have been made to retire it. As we mention in
the introduction, amongst other things, fake news has been
said to be vague, politically dangerous (as it is appropriated
as a tactical term by various parties), and indistinguishable
from previous forms of propaganda, disinformation and
misinformation. While in this guide we have not abandoned
the term we have sought to address these legitimate
concerns in a different way.
Over the course of the pages above, readers will not have
failed to notice, we move away from a focus on defining
CONCLUSIONS

199

and identifying fake news based on its content. While such
interest is certainly justified, we believe that attempts to
classify and demarcate the terrain of associated phenomena
should be grounded in empirical investigation of not just
the features but also of the social lives of a variety of cases.
We hope that such work will contribute to the development
a more granular analytical, conceptual and theoretical
vocabulary to describe the constellation of phenomena
associated with the term.
What is to be done about fake news? As we mention in the
introduction, if there is one single thing that we hope to
achieve with this Field Guide, it is to broaden the emphasis
of research, journalism and public debate to include a more
substantive focus on the social lives of news and the digital
environments in which they move. We hope that this work
makes at least some modest contribution to the rather
grander task of inspiring, mobilising and assembling publics
who are capable of not only of studying and interpreting these
environments but also changing them.
A year after we started work on this project around the time
of the 2016 US presidential elections, the issue of fake news
has “gone global”. The cast of characters has multiplied from
an initial narrative focusing on grassroots hyper partisan
propagandists, opportunistic Macedonian teenagers and
Russian political operators targeting the United States,
to include actors as diverse as Google and Facebook, the
European Commission, the Chinese Communist Party, the
Italian Five Star movement, UK’s intelligence agency MI6,
Wikipedia, Web Inventor Tim Berners-Lee, election bots,
messaging apps, nuclear threats, tech startups, security firms
and “dark” money in numerous countries around the world.
And the question “what is to be done about fake news?”
has broadened out into a series of questions not only about
online misleading information, but also about online
platforms and the broader digital cultures, practices and
technologies associated with them. What started as a matter
200

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

of identifying and “weeding out” offending articles and
deviant users has unfolded into a much bigger series of
questions and debates about the organisation of public life
online, and the attendant infrastructures and institutions
through which information, knowledge and culture is
created, vetted, shared, used and made meaningful.
As with all controversies, there are different ways of
diagnosing, defining and scoping the problem, as well as
different solutions and conceptions of how responsibility
(and blame) should be apportioned to corporations, markets,
states, politicians, policy-makers, media organisations,
educational institutions, civil society groups and others. And
as with many crises, there are a range of actors lined up with
different agendas but sharing the same sentiment of “not
letting a good crisis go to waste”.
While there are many pressures for a quick response to
the issue of fake news, we hope the approaches that we
explore in this guide encourage readers not to be over-hasty
in appraising the situation, diagnosing the problem and in
proposing fixes. Through the series of recipes in this guide
we hope to provide some pointers about how to spend time
with the phenomenon. In particular we hope the guide
will inform and support research, investigations and public
debate around one aspect of this broader set of concerns: the
mediating capacities and cultures of online platforms and the
web. While the set of recipes that we have provided focuses
on following fake news and other fabrications online, many
of them can easily be repurposed to examine many other
aspects of knowledge politics, issues and controversies, and
the online spaces and digital infrastructures upon which they
play out.
In repurposing digital traces to study knowledge politics, we
also advocate a shift from the examination and evaluation of
claims in themselves and in isolation, to looking more closely
at the various networks in which they are embedded. We thus
propose a shift from the atomistic study of fake news artefacts
CONCLUSIONS

201

(apart from their contexts of circulation), to looking at their
networked and distributed character, the social and cultural
practices of meaning-making that emerge around them, as
well as the media systems which underpin their circulation.
In other words, we urge investigators to consider items
which are classified as fake news not only in terms of truth,
falsity, and the extent to which they accurately depict states
of affairs in the world, but also in terms of how they are
shared, amongst whom, what they depend on, and the many
varieties of value and significance that may be attributed to
them by different publics.
Why might we want to make such a shift? Firstly, a richer
picture of social and cultural processes of making meaning
around digital content might help to open up different
kinds of questions. Is a particular group sharing something
because it considers it is literally true, or because they think
it is funny, germane, ridiculous, intolerable or resonant
with other beliefs and backgrounds? Stemming the flow
of a particular piece of content may have negligible (or
even counterproductive) effects in addressing the beliefs,
practices and concerns of groups which share it. Factchecking corrections risk falling on fallow ground if they
miss the point or punchline, which requires knowledge
of the background against which their claims becomes
poignant, salient or amusing. This is not to suggest that we
should flatten the difference between subjective salience
and objective accuracy, but that both depend on a shared
background of social institutions and cultural practices which
should not be taken for granted.
Secondly, a shift from atomistic to networked investigations
of fake news may enable us to learn more about the specific
ways in which social institutions and culture practices are
enabled, constrained and organised through digital platforms
and infrastructures. While we adopt the metaphor of the
field guide from natural histories, the online spaces that we
study should not be understood as natural ecosystems, but
rather as manufactured landscapes where social and cultural
202

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

[1] Here we are inspired by
work on “technological
landscapes” in the study
of science and technology,
see, for example, Richard
Rogers, Technological
Landscapes, London: Royal
College of Art, 1999.

[2] Sabine Niederer, Networked
Content Analysis: The Case
of Climate Change, unpublished manuscript, 2016.

life unfolds in tandem with specific technological devices and
algorithms.[1] At the same time the web and online platforms
cannot determine how they are used, and so we must look
“across” them to understand not only their techno-political
“shape” but also how online life unfolds around them. In
studying the social life of fake news and other fabrications
we can explore both the capacities of online platforms and
infrastructures and the social practices of their users.
Through this Field Guide we have sought to make clear how
the issue of fake news may foreground central aspects of our
digital environments and thus provides a good opportunity
to study their dynamics. And that these dynamics can and
should be empirically investigated. To illustrate different
aspects of these dynamics we attended to the networked
character of fake news and to its technicity, that is the way
in which fake news is formatted, ordered, metrified, datafied
and thus co-produced with digital platforms.[2] Thus, chapters
1 and 2 discuss the publics and modes of circulation afforded
by these platforms. Chapter 3 investigates the tracking
networks in which online content is embedded and through
which its readers are rendered into data. Chapter 4 analyses
the media artefacts that circulate well online, namely imagebased memes, and chapter 5 explores how platform features
may be mobilised in the service of attacks directed at political
representatives.
However, while empirical approaches to studying the social
life of fake news and other fabrications online are necessary,
they also brings a number of challenges. While we emphasize
the need for studying how fake news circulates, the current
configurations of digital platforms, for good (and less good)
reasons, do not always allow this. As a consequence, all the
recipes described in this book are meant to study the public
circulation of fake news. Our study of Facebook provides a
perfect illustration of such challenges. The API of the social
network allows scholars to retrieve the contents of public
pages, but prevents them from accessing the information
exchanged through personal accounts (although some of

CONCLUSIONS

203

this information may be accessed via public pages). Online
platforms have their own ways of organising the boundaries
between public and private. And in the case of fake news, we
also have to deal with the consequences of technological fixes
to the phenomenon on possibilities to study it. In the case of
Facebook, measures to remove problematic posts from the
platform mean that researchers are unable to examine how
users engaged with these items. How platforms, regulators,
policy-makers, users and others negotiate these unfolding
questions of the configuration of these emerging spaces
of publicity and privacy, their attendant mechanisms of
accountability, remains to be seen.
Beyond questions about how digital landscapes are studied,
organised and reshaped, we hope this guide may also serve
as inspiration for how digital methods may be used to study
and intervene around data politics in the contemporary
moment. Who will have the capacities to shape how data
is created and used? How can data be used to not only to
close debate but to enrich it? How can different kinds of
data help us to pursue objectivity not just through a single
picture, but through a plurality of different perspectives?
How does the configuration of digital infrastructures shape
what is hearable, sayable, seeable and doable with data? Who
and what will stand to benefit from the data society? The
Field Guide to Fake News is the first of an ongoing series of
activities and experiments with the Public Data Lab through
which we hope to continue to explore these themes.

Jonathan Gray (@jwyg), Liliana Bounegru (@bb_liliana),
Tommaso Venturini (@TommasoVenturin)
Paris, November 2017

204

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

TOOLS GLOSSARY

In this section we provide brief descriptions and links to
various tools that are referenced throughout the field guide.
These descriptions are intended to be sufficiently informative
to enable readers to follow the text. It should be noted that
it is very important in any research project or investigation
to develop an appreciation of precisely how they work
and what they do (and what they do not do). Hence we
advise you to refer to the documentation and more detailed
descriptions on the websites listed below before using them
in your own project.
 BuzzSumo: a social analytics service which enables

users to explore the most “engaged” content relative to
a given topic or domain. You can filter the results by
language, country, word count and content type (article,
infographic, interviews, videos). (http://buzzsumo.com/)

 CorText: an online application used to analyse textual

data. It allows users to create various types of statistical
and network visualisations. (http://www.cortext.net/)

 CrowdTangle: digital tool that allows users to track

how content spreads through the web and follow the
performance of posts and accounts on Facebook, Twitter,
YouTube, Instagram and Vine.
(http://www.crowdtangle.com)

 CSV Rinse Repeat: a JavaScript based tool to clean and
GLOSSARY

205

structure a csv files, including filtering, clustering,
parsing, merging and matching regular expressions.
(http://tools.medialab.sciences-po.fr/csv-rinse-repeat/)
 DMI Tracker Tracker: a web-based tool which uses data

from the Ghostery project to detect a set of over 900
“fingerprints” of analytics tools, widgets, social plugins,
and other trackers in a given set of URLs.
(https://tools.digitalmethods.net/beta/trackerTracker/)

 DMI Triangulation Tool: identifies common items in two

or more lists.
(https://tools.digitalmethods.net/beta/triangulate/)

 DownThemAll!: a Mozilla Firefox extension that allows

you to collect all the links and images contained in a web
page. (https://addons.mozilla.org/en-US/firefox/addon/
downthemall/)

 Gephi: network analysis and visualization software.

Gephi is particularly helpful for finding patterns, trends
and highlights in large datasets. (http://gephi.org)

 Google Image Search: a search service provided by

Google, which allows users to retrieve images related to a
keyword or a query. (http://images.google.com)

 Google News Search: a news aggregator from Google

which provides results on news articles, sorting them by
date and time of publication. (https://news.google.com/)

 Google's Vision API: an image analysis tool which allows

you to categorise pictures, detect objects or individual
faces, as well as to extract textual content.
(https://cloud.google.com/vision/)

 Google Web Search: a search engine which provides

results based on “Page Rank” (see concept dictionary).

206

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

 Graph Recipes: an online Javascript tool that allows you

to generate static images and compute statistical metrics
about networks. A number of default scripts are offered
by the tool, but others can be added by the user.
(http://tools.medialab.sciences-po.fr/graph-recipes)

 Hyphe: a semi-automated web crawler allowing users

to identify and follow the hyperlinks a on a series of
webpages, to define and categorize a corpus of websites
and to generate networks of web-entities and their
connections. (http://hyphe.medialab.sciences-po.fr/)

 Image J: an open source, Java-based program used to

edit, calibrate, process, measure and analyse visual data.
(https://imagej.nih.gov/ij/)

 Le Monde Décodex: tool that helps users check the source

of information circulating online and identify rumours or
distortions.
(http://www.lemonde.fr/verification/)

 Netvizz app: a Facebook application that extracts a variety

of data from different sections of the platform, including
groups, fan pages and search function.
(https://apps.facebook.com/netvizz/)

 Radarly: a commercial tool to monitor social media,

which allows you to track what is being said about
particular topics, people or events online.
(http://linkfluence.com/en/products/radarly/)

 RAWGraphs: allows you to create vector-based

visualizations of your dataset. Based on the svg format,
RAWGraphs is highly customizable and visualizations
can be imported in and edited with vector graphics
applications for further refinements, or directly
embedded into web pages. (http://rawgraphs.io)

 Spyonweb: allows you to identify websites associated
GLOSSARY

207

with the same IDs by querying the WHOIS protocol
of registered users or assignees of an Internet resource.
(http://spyonweb.com)
 Tab Save: a Google Chrome extension that allows you to

collect and save files such as PDFs, images or list of URLs
available on a web page.
(https://chrome.google.com/webstore/detail/tab-save/
lkngoeaeclaebmpkgapchgjdbaekacki?hl=en)

 Table2Net: allows you to transform tables (.csv) into

networks (.gexf).
(http://tools.medialab.sciences-po.fr/table2net)

 TCAT: (Twitter Capture and Analysis Toolset) a tool that

allows you to retrieve and collect data from Twitter.
The datasets can be collected based on keyword, user or
hashtag queries.
(https://wiki.digitalmethods.net/Dmi/ToolDmiTcat)

 The Wayback Machine: an initiative by the Internet

Archive, which archives versions of websites at regular
intervals. (https://archive.org/web/)

208

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

CONCEPTS GLOSSARY
→ API (Application Programming Interface): a set of clearly

defined methods of communication that allows two
pieces of software to communicate with each. In Web
research APIs are often used to extract data from public
or private datasets (typically those collected by social
platforms), without having direct access to the database
that contains them.

→ Bubble Graph: a type of scatterplot (see the definition in

Bubble Graph

this glossary) in which the size of the projected point is
proportional to a third variable.

→ Circle packing: a type of data visualisation used to

visualize hierarchically structured data. Each cluster/
group is represented by a circle. The circle is then packed
with smaller circles representing sub-groups. The size of
the circle can represent different quantitative properties.

→ Click-bait: online content with the main purpose of

Circle packing

attracting attention and encouraging users to click on a
link to a particular page.

→ CMS (Content Management System): is a software

application used to create and manage digital content and
websites in particular.

→ Emergent coding: a technique to classify items through

categories that are not presupposed before the
observation, but are iteratively defined in the exploration
process. The purpose of this type of coding is to remain
as close as possible to the categories used by the studied
actors themselves instead of fitting data into preestablished categories.

GLOSSARY

209

→ Facebook page or group followers: number of users who

have liked a Facebook page or joined a group.

→ Force-directed network layout: a graph drawing algorithm

used to spatialize items inside a network and help make
sense of the data. The force-directed layout uses repulsive
forces between the nodes while applying attractive forces
between adjacent nodes.

→ Google Analytics ID: an identification assigned by Google

Analytics (Google's service to tracks and reports website
traffic) to identify a user account.

→ Heatmap: a type of data visualisation in which the

variation of values present in a table or matrix are
represented by a gradient of colors.

Heatmap

→ Interactions/engagement: the total number of likes, shares

and comments on a Facebook post (source: http://www.
crowdtangle.com/resources/glossary).

→ Network analysis and visualisation: the process of

Network Graph

investigating the connections/relationships between
individuals, webpages, accounts or any other group of
entities. Using visualization tools such as Gephi it is
possible to characterize associative phenomena in terms
of nodes (individual actors, people, items) and the ties, or
links, that connect them.

→ Network Graph: a type of data visualisation used to

highlight the relationship between entities, where nodes
(or points) represent the entities and lines (or arc, or
edges) represent the relationship between them.

→ PageRank: the algorithm used by Google Search to rank

the results of its queries. While there are several factors
that influence the position of a website on a query
(combined in ways that are not publicly known), the basis
of the ranking is the recursive count to the references

210

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

pointing to a website (how many pages point to a page
and how many pages point to those pages).
→ Scatter plot: a type of data visualisation in which points

Scatter plot

are positioned in a Cartesian diagram according to
the value that they have on two different variables
(corresponding to the axes of the diagram). This type
of diagram is most often used to reveal a correlation
between the two variables it represents.

→ Source code: a set of instructions written in programing

languages, such as HTML or JavaScript, defining how
software should function or a document be displayed.

→ Subscriber Count on CrowdTangle Output Spreadsheet:

the number of subscribers the account had when the post
was published - in contrast to the subscriber count found
on the account, which represents the current number of
subscribers an account has (source: https://github.com/
CrowdTangle/API/wiki/Post#statistics).
→ Treemap: a type of data visualisation used to represented

Treemap

a hierarchical categorisation through nested rectangles.
Each category is associated to a rectangle, whose size
is proportional to the importance or weight of the
group and which is then filled with smaller rectangles
representing sub-groups.

→ Web crawling: the process of extracting the network

of hyperlinks connecting an ensemble of websites or
webpages. Crawling is generally performed by automatic
or semi-automatic tools called 'spiders' capable to identify
and follow all the hyperlinks present on a set of HTML
pages.

→ Web scraping: a method for extracting structured

information or content from a website (and saving it in a
tabular format).

GLOSSARY

211

CONTRIBUTORS AND
ACKNOWLEDGEMENTS

Contributors
Co-investigators
Liliana Bounegru, University of Groningen, University of Ghent
Jonathan Gray, King's College London
Tommaso Venturini, Univ Lyon, Inria, ENS de Lyon, CNRS, UCB
Michele Mauri, Density Design, Politecnico di Milano
Research and Editorial Assistant
Daniela Demarchi, University of Amsterdam
Editorial Design
Ángeles Briones, Politecnico di Milano
Carlo De Gaetano, Politecnico di Milano
DensityDesign Researchers
Agata Brilli, Politecnico di Milano
Ángeles Briones, Politecnico di Milano
Carlo De Gaetano, Politecnico di Milano
Gabriele Colombo, Politecnico di Milano
Mariasilvia Poltronieri, Politecnico di Milano
Michele Invernizzi, Politecnico di Milano
Michele Mauri, Politecnico di Milano
Paolo Ciuccarelli, Politecnico di Milano
Tommaso Elli, Politecnico di Milano
Facilitators
Alex Gekker, University of Amsterdam
Anders Munk, Aalborg University
Bilel Benbouzid, University of Paris-Est Marne-la-Vallée
Erik Borra, University of Amsterdam
Esther Weltevrede, University of Amsterdam
Jonathan Gray, King's College London
Jorn Preuss, University of Siegen
Liliana Bounegru, University of Ghent, University of Groningen

Analysts
Anders Grundtvig, Aalborg University
Anna Keuchenius, University of Amsterdam
Antonio Martella, University of Amsterdam
Asbjørn Fleinert Mathiasen, Aalborg University
Asger Gehrt Olesen, Aalborg University
Carlo Santagiustina, University of Amsterdam
Charlotte Leclercq, University of Amsterdam
Daniel Bach, Aalborg University
Daniela Demarchi, University of Amsterdam
Ecesu Erol, University of Amsterdam
Emil Jørgensen, Aalborg University
Joep Voorn, University of Amsterdam
Jörn Preuss, University of Siegen
Kaspar Beelen, University of Amsterdam
Katerina Gladkova, University of Amsterdam
Lieke Kersten, University of Amsterdam
Lisanne Blomberg, University of Amsterdam
Manon van Hoek, University of Amsterdam
Maria Hayat, University of Amsterdam
Marlene Scherf, University of Amsterdam
Michel Blonk, University of Amsterdam
Mintsje de Witte, University of Amsterdam
Mischa Benjamin Szpirt, Aalborg University
Pieter Vliegenthart, University of Amsterdam
Rina Tsubaki, European Journalism Centre
Ronja Ingeborg Lofstad, Aalborg University
Sal Hagen, University of Amsterdam
Stefani Mans, University of Amsterdam
Stefanie Voortman, University of Amsterdam
Talia Castellanos Usigli, University of Amsterdam
Zoë Versteegen, University of Amsterdam

Marc Tuters, University of Amsterdam
Mathieu Jacomy, Sciences Po
Natalia Sánchez-Querubín, University of Amsterdam
Nicolas Baya-Laffite, University of Lausanne
Paolo Ciuccarelli, Politecnico di Milano
Richard Rogers, University of Amsterdam
Sabine Niederer, Hogeschool van Amsterdam
Tommaso Venturini, Univ Lyon, Inria, ENS de Lyon, CNRS, UCB

212

A FIELD GUIDE TO "FAKE NEWS" AND OTHER INFORMATION DISORDERS

Contributors by chapter
chapter 1
MAPPING FAKE NEWS HOTSPOTS ON FACEBOOK
Facilitators:

Erik Borra, University of Amsterdam

chapter 3
USING TRACKER SIGNATURES TO MAP THE TECHNOCOMMERCIAL UNDERPINNINGS OF FAKE NEWS SITES

Liliana Bounegru, University of Groningen,

Facilitator:

University of Ghent

Jonathan Gray, King’s College London

Jonathan Gray, King’s College London

Natalia Sánchez-Querubín, University of

Richard Rogers, University of Amsterdam

Amsterdam
Esther Weltevrede, University of Amsterdam
Members:

Liliana Bounegru, University of Groningen,

University of Ghent

Members:

Michele Invernizzi, DensityDesign Research Lab
Mischa Szpirt, Aalborg University

Lisanne Blomberg, University of Amsterdam
Talía Castellanos, University of Amsterdam
Tommaso Elli, DensityDesign Research Lab
Stefanie Voortman, University of Amsterdam

chapter 4
HOW TO STUDY POLITICAL MEMES ON FACEBOOK

Stefani Mans, University of Amsterdam

Facilitators:

Mintsje de Witte, University of Amsterdam

Nicolas Baya-Laffite, University of Lausanne

Antonio Martella, University of Amsterdam

Bilel Benbouzid, University of Paris-Est

Rina Tsubaki, European Journalism Centre

Marne-la-Vallée

Manon van Hoek, University of Amsterdam

Marc Tuters, University of Amsterdam

Zoë Versteegen, University of Amsterdam
Joep Voorn, University of Amsterdam

Members:

Daniel Bach, Aalborg University
Carlo De Gaetano, DensityDesign Research Lab
Sal Hagen, University of Amsterdam
Emil Jørgensen, Aalborg University

chapter 2
TRACING THE CIRCULATION OF FAKE NEWS ON THE WEB
Facilitators:

Mathieu Jacomy, Sciences Po
Anders Grundtvig, Aalborg University
Tommaso Venturini, Univ Lyon, Inria, ENS de
Lyon, CNRS, UCB

Members:

chapter 5
MAPPING TROLL-LIKE PRACTICES ON TWITTER
Facilitators:

Erik Borra, University of Amsterdam
Sabine Niederer, Hogeschool van Amsterdam

Agata Brilli, DensityDesign Research Lab

Jörn Preuß, University of Siegen

Daniela Demarchi, University of Amsterdam

Esther Weltevrede, University of Amsterdam

Ronja Lofstad, Aalborg University
Anders Kristian Munk, Aalborg University

Members:

Ángeles Briones, DensityDesign Research Lab
Michel Blonk, University of Amsterdam
Lieke Kersten, University of Amsterdam
Carlo Santagiustina, University of Amsterdam
Marlene Scherf, University of Amsterdam
Pieter Vliegenthart, University of Amsterdam

We would also like to register our gratitude to the
following people who provided invaluable input,
feedback, advice and support at various stages: Claire
Wardle and Jenni Sargent at First Draft; Ida EklundLindwall at East Stratcom Task Force; Jayson Harsin,
The American University of Paris (AUP); Craig Silverman
and Lam Thuy Vo at BuzzFeed News; friends and
colleagues at Le Monde, NRC, the New York Times and
other organisations with whom we corresponded with
about the guide. We also benefited from discussions
with participants at public talks, workshops and events
including the International Journalism Festival 2017 in
Perugia; "Fake News, Algorithmic Accountability and
the Role of Data Journalism in the Post-Truth Era" at
the Centre for Research in the Arts, Social Sciences
and Humanities (CRASSH), University of Cambridge;
“Data Publics” at Lancaster University; “Politics, Fake
News and the Post-Truth Era” at the University of Bath;
"Les fausses nouvelles : le nouveau visage d’un vieux
problème" at the Montréal University; “Data Storytelling:
Engaging Visual Narratives” at Data for Culture
conference, Katowice Miasto Ogrodów; the “Social Life
of Fake News Online” at King’s College London and
“Social Media and Democracy: New Challenges for
Political Communication Research” at Lund University
and the University of Copenhagen.

http://fakenews.publicdatalab.org/



---

Source Exif Data:

File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.4
Linearized                      : Yes
Language                        : en-GB
Tagged PDF                      : Yes
XMP Toolkit                     : Adobe XMP Core 5.6-c137 79.159768, 2016/08/11-13:24:42
Create Date                     : 2018:01:07 10:07:59+01:00
Metadata Date                   : 2018:01:07 10:11:35+01:00
Modify Date                     : 2018:01:07 10:11:35+01:00
Creator Tool                    : Adobe InDesign CC 2017 (Macintosh)
Instance ID                     : uuid:a68bff1f-ee73-194c-9fcb-4e0d5fdef2b5
Original Document ID            : xmp.did:afbc507a-8986-4a91-9a64-61ec0b174a17
Document ID                     : xmp.id:00db96d1-379e-46d2-9e75-6b41070b63ea
Rendition Class                 : proof:pdf
Derived From Instance ID        : xmp.iid:a6f759f6-61e1-407b-ac63-4eecf444af0c
Derived From Document ID        : xmp.did:cc4a0ec2-e478-4da9-b57e-8e644e573d52
Derived From Original Document ID: xmp.did:afbc507a-8986-4a91-9a64-61ec0b174a17
Derived From Rendition Class    : default
History Action                  : converted
History Parameters              : from application/x-indesign to application/pdf
History Software Agent          : Adobe InDesign CC 2017 (Macintosh)
History Changed                 : /
History When                    : 2018:01:07 10:07:59+01:00
Format                          : application/pdf
Producer                        : Adobe PDF Library 15.0
Trapped                         : False
Page Count                      : 216
Creator                         : Adobe InDesign CC 2017 (Macintosh)

EXIF Metadata provided by EXIF.tools