Repository for the COLING 2020 paper "Explainable Automated Fact-Checking: A Survey."

Overview

Explainable Fact Checking: A Survey

arXiv Maintenance Last Commit Contribution_welcome

This repository and the accompanying webpage contain resources for the paper "Explainable Fact Checking: A Survey". In the paper, we offer a critical review of the state-of-the-art in automated fact-checking with a particular focus on interpretability.

Reference

If you find our work useful, please cite the paper as formatted below.

  @inproceedings{kotonya-toni-2020-explainable-automated,
      title = "Explainable Automated Fact-Checking: A Survey",
      author = "Kotonya, Neema  and
        Toni, Francesca",
      booktitle = "Proceedings of the 28th International Conference on Computational Linguistics",
      month = dec,
      year = "2020",
      address = "Barcelona, Spain (Online)",
      publisher = "International Committee on Computational Linguistics",
      url = "https://www.aclweb.org/anthology/2020.coling-main.474",
      pages = "5430--5443"
  }

Contents

Here is an overview of papers mentioned in this work, and more recent papers which have been added.

Introduction

Fact checking is the process of establishing the veracity of claims i.e., to distinguish between false stories (e.g., misattributions, rumours, hoaxes) and facts.

Over the past few years the use of deep learning methods for fact checking and fake news detection have become a popular. Indeed, several exciting breakthroughs have occurred in automated fact checking thanks in large part due to new datasets (e.g., FEVER) and advances in machine learning for NLP. However there are still some limitations in this research area, the one we focus on in this work in our work is explanations for automated fact checking.

The pipeline commonly employed for automated fact-checking consists of four parts (subtasks). We propose that post-hoc explanations are an important and necessary extension of this pipeline.

fact-checking-pipeline

For an overview of the data and results mentioned in our survey, please visit this webpage.

Task Formulations

Here we list papers which address varied tasks related to fact checking and fake news detection.

  • Check-worthy Claim Detection

    • Toward Automated Fact-Checking: Detecting Check-worthy Factual Claims by ClaimBuster (Hassan et al., 2017). [paper] [bib]
  • Fauxtography and Multimodal Fake News Detection

    • FauxBuster: A Content-free Fauxtography Detector Using Social Media Comments (Zhang et al., 2018). [paper] [bib] [slides]
    • Fact-Checking Meets Fauxtography: Verifying Claims About Images (Zlatkova et al., 2019). [paper] [bib]
    • Eann: Event adversarial neural networks for multi-modal fake news detection (Wang et al., 2018). [paper] [bib]
  • Identifying Previously Fact-Checked Claims

    • That is a Known Lie: Detecting Previously Fact-Checked Claims (Shaar et al., 2020). [paper] [bib]
    • Where Are the Facts? Searching for Fact-checked Information to Alleviate the Spread of Fake News (Vo and Lee, 2020). [paper] [bib]
  • Neural Fake News Detection

    • Defending against neural fake news (Zellers et al., 2019). [paper] [bib]
    • Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News (Tan et al., 2020). [paper] [bib]
  • Rumour Verification and Resolution

    • SemEval-2019 Task 7: RumourEval, Determining Rumour Veracity and Support for Rumours (Gorrell et al., 2019). [paper] [bib]
    • Can Rumour Stance Alone Predict Veracity? (Dungs et al., 2018). [paper] [bib]
    • SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours (Derczynski et al., 2017). [paper] [bib]
  • Stylometric Analysis of News Articles

    • A stylometric inquiry into hyperpartisan and fake news (Potthast et al., 2017) [paper] [bib] [video]
    • Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking (Rashkin et al., 2017). [paper] [bib]
  • Table-based Fact Verification

    • TabFact: A Large-scale Dataset for Table-based Fact Verification (Chen et al., 2020). [paper] [bib]
  • Multi-hop Fact Checking

  • Error Correction of Claims

Fact Checking Datasets

List of fact checking, rumour verification and fake news detection datasets:

Datasets of naturally occurring claims

Social media

Claims from social media platforms sources e.g., Twitter, Facebook.

  • r/Fakeddit: A New Multimodal Benchmark Dataset for Fine-grained Fake News Detection (Nakamura et al., 2020). [paper] [data] [bib]
  • SemEval-2019 Task 7: RumourEval, Determining Rumour Veracity and Support for Rumours (Gorrell et al., 2019). [paper] [data] [bib]
  • All-in-one: Multi-task Learning for Rumour Verification (Kochkina et al., 2018). [paper] [data]† [bib]
  • SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours (Derczynski et al., 2017). [paper] [data] [bib]
  • Hyperpartisan Facebook Pages Are Publishing False And Misleading Information At An Alarming Rate (Silverman et al., 2017). [article] [data]
  • Detect Rumors in Microblog Posts Using Propagation Structure via Kernel Learning (Ma et al., 2017). [paper] [data] [bib]
  • Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threads (Zubiaga et al., 2016). [paper] [data] [bib]
  • CREDBANK: A Large-Scale Social Media Corpus with Associated Credibility Annotations (Mitra and Gilbert, 2015). [paper] [data] [bib]

† This dataset is an extention of the PHEME rumour dataset.

Fact checking and news websites

Claims for news and fact-checking platforms e.g., Snopes, Politifact.

  • Explainable Automated Fact-Checking for Public Health Claims (Kotonya and Toni, 2020). [paper] [data] [bib]
  • STANDER: An Expert-Annotated Dataset for News Stance Detection and Evidence Retrieval (Conforti et al., 2020). [paper] [data] [bib]
  • FakeCovid-- A Multilingual Cross-domain Fact Check News Dataset for COVID-19 (Shahi and Nandini, 2020). [paper] [data] [bib]
  • MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims (Augenstein et al., 2019). [paper] [bib] [data]
  • A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking (Hanselowski et al., 2019). [paper] [code] [data] [bib]
  • Integrating Stance Detection and Fact Checking in a Unified Corpus (Baly et al., 2018). [paper] [data] [bib]
  • FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media (Shu et al., 2018). [paper] [data]
  • “Liar, Liar Pants on Fire”: A New Benchmark Dataset for Fake News Detection (Wang, 2017). [paper] [data] [bib]
  • Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking (Rashkin et al., 2017). [paper] [data] [bib]
  • The Fake News Challenge (Pomerleau and Rao, 2017) [data]

Hand crafted

This covers claims which are generated manually e.g. through re-writing statements.

Wikipedia

  • TabFact: A Large-scale Dataset for Table-based Fact Verification (Chen et al., 2020). [paper] [data] [bib]
  • FEVER: a Large-scale Dataset for Fact Extraction and VERification (Thorne et al., 2018). [paper] [data] [bib]
  • Automated Fact-Checking of Claims from Wikipedia (Sathe et al., 2020). [paper] [data] [bib]
  • Generating Fact Checking Briefs (Fan et al., 2020). [paper] [bib]

Scientific journals

  • Fact or Fiction: Verifying Scientific Claims (Wadden et al., EMNLP 2020). [paper] [data] [bib]

Fact Checking Systems

A list of fact-checking and fake news detection systems.

Systems by Dataset

LIAR

  • Where is your Evidence: Improving Fact-checking by Justification Modeling (Alhindi et al., 2018). [paper] [bib] [code]
  • Generating Fact Checking Explanations (Atanasova et al., 2020). [paper] [bib]

FEVER

  • FEVER 1.0 Baseline
  • Combining Fact Extraction and Verification with Neural Semantic Matching Networks (Nie et al., 2019). [paper] [bib] [code]
  • UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF) (Yoneda et al., 2018). [paper] [bib] [code]
  • UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification (Hanselowski et al., 2018). [paper] [bib] [code]
  • Team Papelo: Transformer Networks at FEVER (Malon, 2018). [paper] [bib] [code]
  • Team DOMLIN: Exploiting Evidence Enhancement for the FEVER Shared Task (Stammbach and Neumann, 2019). [paper] [bib] [code]
  • GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification [paper] [bib] [code]
  • Fine-grained Fact Verification with Kernel Graph Attention Network [paper] [bib] [code]

MultiFC

  • Time-Aware Evidence Ranking for Fact-Checking (Allein et al., 2020). [paper] [bib]

Systems by Method

Support Vector Machines

  • Fake News or Truth? Using Satirical Cues to Detect Potentially Misleading News. [paper] [bib]

Convolutional Neural Networks

  • “Liar, Liar Pants on Fire”: A New Benchmark Dataset for Fake News Detection (Wang, 2017). [paper] [bib]
  • FAKTA: An Automatic End-to-End Fact Checking System (Nadeeem et al., 2019). [paper] [bib]

Recurrent Neural Networks

  • CSI: A Hybrid Deep Model for Fake News Detection (Ruchansky et al., 2017). [paper]
  • DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning (Popat et al., 2018). [paper] [bib]
  • Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking (Rashkin et al., 2017). [paper] [bib]
  • Where is your Evidence: Improving Fact-checking by Justification Modeling (Alhindi et al., 2018). [paper] [bib]

Transformers and Attention Networks

  • Two Stage Transformer Model for COVID-19 Fake News Detection and Fact Checking (Vijjali et al., 2020). [paper] [bib]

Hybrid

  • GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social Media (Lu and Li, 2020). [paper] [bib]
  • DTCA: Decision Tree-based Co-Attention Networks for Explainable Claim Verification (Wu et al., 2020). [paper] [bib]
  • XFake: Explainable Fake News Detector with Visualizations (Yang et al., 2019). [paper] [bib]

Shared Tasks

📣
indicates the shared task is ongoing!

Explainable Fact Checking

Systems

A list of works on explainability in fact-checking and fake news detection:

Saliency maps

  • dEFEND: Explainable Fake News Detection (Shu et al., 2019). [paper] [bib] [video]
  • GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social Media (Lu and Li, 2020). [paper] [bib]
  • DTCA: Decision Tree-based Co-Attention Networks for Explainable Claim Verification (Wu et al., 2020). [paper] [bib]
  • DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning (Popat et al., 2018). [paper] [bib]
  • CredEye: A Credibility Lens for Analyzing and Explaining Misinformation (Popat et al., 2018). [paper] [bib] [demo]
  • XFake: Explainable Fake News Detector with Visualizations (Yang et al., 2019). [paper] [bib]
  • Credibility Assessment of Textual Claims on the Web (Popat et al., 2017). [paper] [bib]

Rule discovery and graph based methods

  • ExFaKT: A Framework for Explaining Facts over Knowledge Graphs and Text (Gad-Elrab et al., 2019). [paper] [bib]
  • Explainable Fact Checking with Probabilistic Answer Set Programming (Ahmadi et al., 2019). [paper] [bib]
  • Linked Credibility Reviews for Explainable Misinformation Detection (Denaux et al., 2020). [paper [bib]

Summarization

  • e-FEVER: Explanations and Summaries for Automated Fact Checking (Stammbach and Ash, 2020). [paper] [bib]
  • Generating Fact Checking Explanations (Atanasova et al., 2020). [paper] [bib]
  • Explainable Automated Fact Checking for Public Health Claims (Kotonya and Toni, 2020). [paper] [bib]

Adversarial Training for Automated Fact Checking

List of papers which look at adversarial training for robust fact-checking:

  • Generating Label Cohesive and Well-Formed Adversarial Claims (Atanasova et al.,2020). [paper] [bib] [code]
  • Evaluating adversarial attacks against multiple fact verification systems (Thorne et al., 2019). [paper] [bib] [video]
  • DeSePtion: Dual Sequence Prediction and Adversarial Examples for Improved Fact-Checking (Hidey et al., 2020). [paper] [bib]
  • Fake news detection via NLP is vulnerable to adversarial attacks (Zhou et al., 2019). [paper] [bib]
  • GEM: Generative enhanced model for adversarial attacks (Niewinski et al., 2019). [paper] [bib]
  • FEVER Breaker’s Run of Team NbAuzDrLqg (Kim and Allan, 2019). [paper] [bib]

Error Correction of Claims

  • Factual Error Correction of Claims (Thorne and Vlachos, 2020). [paper] [bib]

Fact Checking with Intrepretable Features

  • A Language-Based Approach to Fake News Detection Through Interpretable Features and BRNN (Qiao et al., 2020). [paper] [bib]

Multihop Reasoning for Facting Checking

Multi-hop reasoning is closely important for explainability in fact-checking, recently there have been a number of papers which look to address this task:

  • HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verification (Jiang et al., 2020). [paper] [bib] [project]
  • Multi-Hop Fact Checking of Political Claims (Ostrowski et al., 2020). [paper] [bib] [code]
  • Chains-of-Reasoning at TextGraphs 2019 Shared Task: Reasoning over Chains of Facts for Explainable Multi-hop Inference (Das et al., 2019). [paper] [bib]

Surveys

Automated fact checking and fake news detection surveys:

  • A Survey of Fake News: Fundamental Theories, Detection Methods, and Opportunities (Zhou and Zafarani, 2020). [paper] [bib]
  • A Review on Fact Extraction and VERification: The FEVER case (Bekoulis et al., 2020). [paper] [bib]
  • A Survey on Fake News and Rumour Detection Techniques (Bondielli and Marcelloni, 2020). [paper] [bib]
  • A Survey on Natural Language Processing for Fake News Detection (Oshikawa et al., 2020). [paper] [bib]
  • Fake News Detection using Stance Classification: A Survey (Lillie and Middelboe, 2019). [paper] [bib]
  • Detection and Resolution of Rumours in Social Media: A Survey (Zubiaga et al., 2018). [paper] [bib]
  • Automated Fact Checking: Task Formulations, Methods and Future Directions (Thorne and Vlachos, 2018). [paper] [bib]
  • Media-Rich Fake News Detection: A Survey (Parikh and Atrey, 2018). [paper] [bib]
  • A Content Management Perspective on Fact-Checking (Cazalens et al., 2018). [paper] [bib]
  • Fake News Detection on Social Media: A Data Mining Perspective (Shu et al., 2017). [paper] [bib]

Tutorials

Contact

Please feel free to contact Neema Kotonya if you have any questions or comments.

Owner
Neema Kotonya
PhD Student at Imperial College London.
Neema Kotonya
Official implementation of "OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association" in PyTorch.

openpifpaf Continuously tested on Linux, MacOS and Windows: New 2021 paper: OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Te

VITA lab at EPFL 50 Dec 29, 2022
PyTorch Implementation of our paper Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation

PyTorch Implementation of our paper Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation

Zechen Bai 12 Jul 08, 2022
Open source repository for the code accompanying the paper 'PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations'.

PatchNets This is the official repository for the project "PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations". For details,

16 May 22, 2022
Nvdiffrast - Modular Primitives for High-Performance Differentiable Rendering

Nvdiffrast – Modular Primitives for High-Performance Differentiable Rendering Modular Primitives for High-Performance Differentiable Rendering Samuli

NVIDIA Research Projects 675 Jan 06, 2023
FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation.

FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation [Project] [Paper] [arXiv] [Home] Official implementation of FastFCN:

Wu Huikai 815 Dec 29, 2022
Energy consumption estimation utilities for Jetson-based platforms

This repository contains a utility for measuring energy consumption when running various programs in NVIDIA Jetson-based platforms. Currently TX-2, NX, and AGX are supported.

OpenDR 10 Jun 17, 2022
Code for the paper One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation, CVPR 2021.

One Thing One Click One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation (CVPR2021) Code for the paper One Thi

44 Dec 12, 2022
Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

Fairseq(-py) is a sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language mod

20.5k Jan 08, 2023
PyTorch implementation of Munchausen Reinforcement Learning based on DQN and SAC. Handles discrete and continuous action spaces

Exploring Munchausen Reinforcement Learning This is the project repository of my team in the "Advanced Deep Learning for Robotics" course at TUM. Our

Mohamed Amine Ketata 10 Mar 10, 2022
Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation.

Understanding Minimum Bayes Risk Decoding This repo provides code and documentation for the following paper: Müller and Sennrich (2021): Understanding

ZurichNLP 13 May 01, 2022
Learning Facial Representations from the Cycle-consistency of Face (ICCV 2021)

Learning Facial Representations from the Cycle-consistency of Face (ICCV 2021) This repository contains the code for our ICCV2021 paper by Jia-Ren Cha

Jia-Ren Chang 40 Dec 27, 2022
Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive Learning.

Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive Learning. Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive

<a href=[email protected](SZ)"> 7 Dec 16, 2021
Security evaluation module with onnx, pytorch, and SecML.

🚀 🐼 🔥 PandaVision Integrate and automate security evaluations with onnx, pytorch, and SecML! Installation Starting the server without Docker If you

Maura Pintor 11 Apr 12, 2022
Pytorch code for our paper "Feedback Network for Image Super-Resolution" (CVPR2019)

Feedback Network for Image Super-Resolution [arXiv] [CVF] [Poster] Update: Our proposed Gated Multiple Feedback Network (GMFN) will appear in BMVC2019

Zhen Li 539 Jan 06, 2023
Generative Exploration and Exploitation - This is an improved version of GENE.

GENE This is an improved version of GENE. In the original version, the states are generated from the decoder of VAE. We have to check whether the gere

33 Mar 23, 2022
AttGAN: Facial Attribute Editing by Only Changing What You Want (IEEE TIP 2019)

News 11 Jan 2020: We clean up the code to make it more readable! The old version is here: v1. AttGAN TIP Nov. 2019, arXiv Nov. 2017 TensorFlow impleme

Zhenliang He 568 Dec 14, 2022
YOLTv5 rapidly detects objects in arbitrarily large aerial or satellite images that far exceed the ~600×600 pixel size typically ingested by deep learning object detection frameworks

YOLTv5 rapidly detects objects in arbitrarily large aerial or satellite images that far exceed the ~600×600 pixel size typically ingested by deep learning object detection frameworks.

Adam Van Etten 145 Jan 01, 2023
Space Ship Simulator using python

FlyOver Basic space-ship simulator using python How to run? Just double click run.py What modules do i need? All modules that i currently using is bui

0 Oct 09, 2022
Using VideoBERT to tackle video prediction

VideoBERT This repo reproduces the results of VideoBERT (https://arxiv.org/pdf/1904.01766.pdf). Inspiration was taken from https://github.com/MDSKUL/M

75 Dec 14, 2022
CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation

CPT This repository contains code and checkpoints for CPT. CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Gener

fastNLP 341 Dec 29, 2022