This repository contains the code, data, and models of the paper titled "XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages" published in Findings of the Association for Computational Linguistics: ACL 2021.

Overview

XL-Sum

This repository contains the code, data, and models of the paper titled "XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages" published in Findings of the Association for Computational Linguistics: ACL 2021.

Table of Contents

Datasets

Disclaimer: You must agree to the license and terms of use before using the dataset.

We are releasing two versions of the dataset: an older version that has been reported in the paper; and a newer version with another added language (Traditional Chinese), more data, better formatting, better extraction, larger evaluation splits, and deduplication. We recommend using the latter and thus have organized the repository with data counts and benchmarks of the newer version. The new version contains a total of 1.35 million article-summary pairs, making XL-Sum the largest text summarization dataset publicly available.

All dataset files are in .jsonl format i.e. one JSON per line. One example from the english dataset is given below in JSON format. The fields are self-explanatory.

{
  "id": "technology-17657859",
  "url": "https://www.bbc.com/news/technology-17657859",
  "title": "Yahoo files e-book advert system patent applications",
  "summary": "Yahoo has signalled it is investigating e-book adverts as a way to stimulate its earnings.",
  "text": "Yahoo's patents suggest users could weigh the type of ads against the sizes of discount before purchase. It says in two US patent applications that ads for digital book readers have been \"less than optimal\" to date. The filings suggest that users could be offered titles at a variety of prices depending on the ads' prominence They add that the products shown could be determined by the type of book being read, or even the contents of a specific chapter, phrase or word. The paperwork was published by the US Patent and Trademark Office late last week and relates to work carried out at the firm's headquarters in Sunnyvale, California. \"Greater levels of advertising, which may be more valuable to an advertiser and potentially more distracting to an e-book reader, may warrant higher discounts,\" it states. Free books It suggests users could be offered ads as hyperlinks based within the book's text, in-laid text or even \"dynamic content\" such as video. Another idea suggests boxes at the bottom of a page could trail later chapters or quotes saying \"brought to you by Company A\". It adds that the more willing the customer is to see the ads, the greater the potential discount. \"Higher frequencies... may even be great enough to allow the e-book to be obtained for free,\" it states. The authors write that the type of ad could influence the value of the discount, with \"lower class advertising... such as teeth whitener advertisements\" offering a cheaper price than \"high\" or \"middle class\" adverts, for things like pizza. The inventors also suggest that ads could be linked to the mood or emotional state the reader is in as a they progress through a title. For example, they say if characters fall in love or show affection during a chapter, then ads for flowers or entertainment could be triggered. The patents also suggest this could applied to children's books - giving the Tom Hanks animated film Polar Express as an example. It says a scene showing a waiter giving the protagonists hot drinks \"may be an excellent opportunity to show an advertisement for hot cocoa, or a branded chocolate bar\". Another example states: \"If the setting includes young characters, a Coke advertisement could be provided, inviting the reader to enjoy a glass of Coke with his book, and providing a graphic of a cool glass.\" It adds that such targeting could be further enhanced by taking account of previous titles the owner has bought. 'Advertising-free zone' At present, several Amazon and Kobo e-book readers offer full-screen adverts when the device is switched off and show smaller ads on their menu screens, but the main text of the titles remains free of marketing. Yahoo does not currently provide ads to these devices, and a move into the area could boost its shrinking revenues. However, Philip Jones, deputy editor of the Bookseller magazine, said that the internet firm might struggle to get some of its ideas adopted. \"This has been mooted before and was fairly well decried,\" he said. \"Perhaps in a limited context it could work if the merchandise was strongly related to the title and was kept away from the text. \"But readers - particularly parents - like the fact that reading is an advertising-free zone. Authors would also want something to say about ads interrupting their narrative flow.\""
}

Download the complete dataset. See the legacy section for the older version(s).

We used a 80%-10%-10% split for all languages with a few exceptions. English was split 93%-3.5%-3.5% for the evaluation set size to resemble that of CNN/DM and XSum; Scottish Gaelic, Kyrgyz and Sinhala had relatively fewer samples, their evaluation sets were increased to 500 samples for more reliable evaluation. Same articles were used for evaluation in the two variants of Chinese and Serbian to prevent data leakage in multilingual training. Individual dataset download links with train-dev-test example counts are given below:

Language ISO 639-1 Code BBC subdomain(s) Train Dev Test Total Link
Amharic am https://www.bbc.com/amharic 5761 719 719 7199 Download
Arabic ar https://www.bbc.com/arabic 37519 4689 4689 46897 Download
Azerbaijani az https://www.bbc.com/azeri 6478 809 809 8096 Download
Bengali bn https://www.bbc.com/bengali 8102 1012 1012 10126 Download
Burmese my https://www.bbc.com/burmese 4569 570 570 5709 Download
Chinese (Simplified) zh-CN https://www.bbc.com/ukchina/simp, https://www.bbc.com/zhongwen/simp 37362 4670 4670 46702 Download
Chinese (Traditional) zh-TW https://www.bbc.com/ukchina/trad, https://www.bbc.com/zhongwen/trad 37373 4670 4670 46713 Download
English en https://www.bbc.com/english, https://www.bbc.com/sinhala * 306522 11535 11535 329592 Download
French fr https://www.bbc.com/afrique 8697 1086 1086 10869 Download
Gujarati gu https://www.bbc.com/gujarati 9119 1139 1139 11397 Download
Hausa ha https://www.bbc.com/hausa 6418 802 802 8022 Download
Hindi hi https://www.bbc.com/hindi 70778 8847 8847 88472 Download
Igbo ig https://www.bbc.com/igbo 4183 522 522 5227 Download
Indonesian id https://www.bbc.com/indonesia 38242 4780 4780 47802 Download
Japanese ja https://www.bbc.com/japanese 7113 889 889 8891 Download
Kirundi rn https://www.bbc.com/gahuza 5746 718 718 7182 Download
Korean ko https://www.bbc.com/korean 4407 550 550 5507 Download
Kyrgyz ky https://www.bbc.com/kyrgyz 2266 500 500 3266 Download
Marathi mr https://www.bbc.com/marathi 10903 1362 1362 13627 Download
Nepali np https://www.bbc.com/nepali 5808 725 725 7258 Download
Oromo om https://www.bbc.com/afaanoromoo 6063 757 757 7577 Download
Pashto ps https://www.bbc.com/pashto 14353 1794 1794 17941 Download
Persian fa https://www.bbc.com/persian 47251 5906 5906 59063 Download
Pidgin** n/a https://www.bbc.com/pidgin 9208 1151 1151 11510 Download
Portuguese pt https://www.bbc.com/portuguese 57402 7175 7175 71752 Download
Punjabi pa https://www.bbc.com/punjabi 8215 1026 1026 10267 Download
Russian ru https://www.bbc.com/russian, https://www.bbc.com/ukrainian * 62243 7780 7780 77803 Download
Scottish Gaelic gd https://www.bbc.com/naidheachdan 1313 500 500 2313 Download
Serbian (Cyrillic) sr https://www.bbc.com/serbian/cyr 7275 909 909 9093 Download
Serbian (Latin) sr https://www.bbc.com/serbian/lat 7276 909 909 9094 Download
Sinhala si https://www.bbc.com/sinhala 3249 500 500 4249 Download
Somali so https://www.bbc.com/somali 5962 745 745 7452 Download
Spanish es https://www.bbc.com/mundo 38110 4763 4763 47636 Download
Swahili sw https://www.bbc.com/swahili 7898 987 987 9872 Download
Tamil ta https://www.bbc.com/tamil 16222 2027 2027 20276 Download
Telugu te https://www.bbc.com/telugu 10421 1302 1302 13025 Download
Thai th https://www.bbc.com/thai 6616 826 826 8268 Download
Tigrinya ti https://www.bbc.com/tigrinya 5451 681 681 6813 Download
Turkish tr https://www.bbc.com/turkce 27176 3397 3397 33970 Download
Ukrainian uk https://www.bbc.com/ukrainian 43201 5399 5399 53999 Download
Urdu ur https://www.bbc.com/urdu 67665 8458 8458 84581 Download
Uzbek uz https://www.bbc.com/uzbek 4728 590 590 5908 Download
Vietnamese vi https://www.bbc.com/vietnamese 32111 4013 4013 40137 Download
Welsh cy https://www.bbc.com/cymrufyw 9732 1216 1216 12164 Download
Yoruba yo https://www.bbc.com/yoruba 6350 793 793 7936 Download

* A lot of articles in BBC Sinhala and BBC Ukrainian were written in English and Russian respectively. They were identified using Fasttext and moved accordingly.

** West African Pidgin English

Models

We are releasing a multilingual model checkpoint trained for 50k steps on the new data. To use this model for evaluation/inference refer to Training & Evaluation.

Benchmarks

Multilingual model scores on test sets are given below. We are also releasing the model-generated outputs for future analysis.

Language ROUGE-1 / ROUGE-2 / ROUGE-L
Amharic 20.0485 / 7.4111 / 18.0753
Arabic 34.9107 / 14.7937 / 29.1623
Azerbaijani 21.4227 / 9.5214 / 19.3331
Bengali 29.5653 / 12.1095 / 25.1315
Burmese 15.9626 / 5.1477 / 14.1819
Chinese (Simplified) 39.4071 / 17.7913 / 33.406
Chinese (Traditional) 37.1866 / 17.1432 / 31.6184
English 37.601 / 15.1536 / 29.8817
French 35.3398 / 16.1739 / 28.2041
Gujarati 21.9619 / 7.7417 / 19.86
Hausa 39.4375 / 17.6786 / 31.6667
Hindi 38.5882 / 16.8802 / 32.0132
Igbo 31.6148 / 10.1605 / 24.5309
Indonesian 37.0049 / 17.0181 / 30.7561
Japanese 48.1544 / 23.8482 / 37.3636
Kirundi 31.9907 / 14.3685 / 25.8305
Korean 23.6745 / 11.4478 / 22.3619
Kyrgyz 18.3751 / 7.9608 / 16.5033
Marathi 22.0141 / 9.5439 / 19.9208
Nepali 26.6547 / 10.2479 / 24.2847
Oromo 18.7025 / 6.1694 / 16.1862
Pashto 38.4743 / 15.5475 / 31.9065
Persian 36.9425 / 16.1934 / 30.0701
Pidgin 37.9574 / 15.1234 / 29.872
Portuguese 37.1676 / 15.9022 / 28.5586
Punjabi 30.6973 / 12.2058 / 25.515
Russian 32.2164 / 13.6386 / 26.1689
Scottish Gaelic 29.0231 / 10.9893 / 22.8814
Serbian (Cyrillic) 23.7841 / 7.9816 / 20.1379
Serbian (Latin) 21.6443 / 6.6573 / 18.2336
Sinhala 27.2901 / 13.3815 / 23.4699
Somali 31.5563 / 11.5818 / 24.2232
Spanish 31.5071 / 11.8767 / 24.0746
Swahili 37.6673 / 17.8534 / 30.9146
Tamil 24.3326 / 11.0553 / 22.0741
Telugu 19.8571 / 7.0337 / 17.6101
Thai 37.3951 / 17.275 / 28.8796
Tigrinya 25.321 / 8.0157 / 21.1729
Turkish 32.9304 / 15.5709 / 29.2622
Ukrainian 23.9908 / 10.1431 / 20.9199
Urdu 39.5579 / 18.3733 / 32.8442
Uzbek 16.8281 / 6.3406 / 15.4055
Vietnamese 32.8826 / 16.2247 / 26.0844
Welsh 32.6599 / 11.596 / 26.1164
Yoruba 31.6595 / 11.6599 / 25.0898

Multilingual ROUGE

Training & Evaluation

License

Contents of this repository are restricted to only non-commercial research purposes under the Creative Commons Attribution 4.0 International License. Copyright of the dataset contents belongs to the original copyright holders.

Citation

If you use any of the datasets, models or code modules, please cite the following paper:

@inproceedings{hasan-etal-2021-xlsum,
    title = "XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages",
    author = "Hasan, Tahmid  and
      Bhattacharjee, Abhik  and
      Islam, Md Saiful and
      Samin, Kazi  and
      Li, Yuan-Fang and
      Kang, Yong-Bin and 
      Rahman, M. Sohel  and
      Shahriyar, Rifat",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2021",
    month = "August",
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "http://arxiv.org/abs/2106.13822"
}
Wind Speed Prediction using LSTMs in PyTorch

Implementation of Deep-Forecast using PyTorch Deep Forecast: Deep Learning-based Spatio-Temporal Forecasting Adapted from original implementation Setu

Onur Kaplan 151 Dec 14, 2022
A user-friendly research and development tool built to standardize RL competency assessment for custom agents and environments.

Built with ❤️ by Sam Showalter Contents Overview Installation Dependencies Usage Scripts Standard Execution Environment Development Environment Benchm

SRI-AIC 1 Nov 18, 2021
iBOT: Image BERT Pre-Training with Online Tokenizer

Image BERT Pre-Training with iBOT Official PyTorch implementation and pretrained models for paper iBOT: Image BERT Pre-Training with Online Tokenizer.

Bytedance Inc. 435 Jan 06, 2023
The official pytorch implementation of our paper "Is Space-Time Attention All You Need for Video Understanding?"

TimeSformer This is an official pytorch implementation of Is Space-Time Attention All You Need for Video Understanding?. In this repository, we provid

Facebook Research 1k Dec 31, 2022
Recall Loss for Semantic Segmentation (This repo implements the paper: Recall Loss for Semantic Segmentation)

Recall Loss for Semantic Segmentation (This repo implements the paper: Recall Loss for Semantic Segmentation) Download Synthia dataset The model uses

32 Sep 21, 2022
Source code for our CVPR 2019 paper - PPGNet: Learning Point-Pair Graph for Line Segment Detection

PPGNet: Learning Point-Pair Graph for Line Segment Detection PyTorch implementation of our CVPR 2019 paper: PPGNet: Learning Point-Pair Graph for Line

SVIP Lab 170 Oct 25, 2022
[NeurIPS'20] Multiscale Deep Equilibrium Models

Multiscale Deep Equilibrium Models 💥 💥 💥 💥 This repo is deprecated and we will soon stop actively maintaining it, as a more up-to-date (and simple

CMU Locus Lab 221 Dec 26, 2022
Code for "Solving Graph-based Public Good Games with Tree Search and Imitation Learning"

Code for "Solving Graph-based Public Good Games with Tree Search and Imitation Learning" This is the code for the paper Solving Graph-based Public Goo

Victor-Alexandru Darvariu 3 Dec 05, 2022
Semantic graph parser based on Categorial grammars

Lambekseq "Everyone who failed Greek or Latin hates it." This package is for proving theorems in Categorial grammars (CG) and constructing semantic gr

10 Aug 19, 2022
Research into Forex price prediction from price history using Deep Sequence Modeling with Stacked LSTMs.

Forex Data Prediction via Recurrent Neural Network Deep Sequence Modeling Research Paper Our research paper can be viewed here Installation Clone the

Alex Taradachuk 2 Aug 07, 2022
Repository providing a wide range of self-supervised pretrained models for computer vision tasks.

Hierarchical Pretraining: Research Repository This is a research repository for reproducing the results from the project "Self-supervised pretraining

Colorado Reed 53 Nov 09, 2022
The official code repository for examples in the O'Reilly book 'Generative Deep Learning'

Generative Deep Learning Teaching Machines to paint, write, compose and play The official code repository for examples in the O'Reilly book 'Generativ

David Foster 1.3k Dec 29, 2022
Custom studies about block sparse attention.

Block Sparse Attention 研究总结 本人近半年来对Block Sparse Attention(块稀疏注意力)的研究总结(持续更新中)。按时间顺序,主要分为如下三部分: PyTorch 自定义 CUDA 算子——以矩阵乘法为例 基于 Triton 的 Block Sparse A

Chen Kai 2 Jan 09, 2022
maximal update parametrization (µP)

Maximal Update Parametrization (μP) and Hyperparameter Transfer (μTransfer) Paper link | Blog link In Tensor Programs V: Tuning Large Neural Networks

Microsoft 694 Jan 03, 2023
Official implementation of "An Image is Worth 16x16 Words, What is a Video Worth?" (2021 paper)

An Image is Worth 16x16 Words, What is a Video Worth? paper Official PyTorch Implementation Gilad Sharir, Asaf Noy, Lihi Zelnik-Manor DAMO Academy, Al

213 Nov 12, 2022
Mixup for Supervision, Semi- and Self-Supervision Learning Toolbox and Benchmark

OpenSelfSup News Downstream tasks now support more methods(Mask RCNN-FPN, RetinaNet, Keypoints RCNN) and more datasets(Cityscapes). 'GaussianBlur' is

AI Lab, Westlake University 332 Jan 03, 2023
Using contrastive learning and OpenAI's CLIP to find good embeddings for images with lossy transformations

The official code for the paper "Inverse Problems Leveraging Pre-trained Contrastive Representations" (to appear in NeurIPS 2021).

Sriram Ravula 26 Dec 10, 2022
Code for Understanding Pooling in Graph Neural Networks

Select, Reduce, Connect This repository contains the code used for the experiments of: "Understanding Pooling in Graph Neural Networks" Setup Install

Daniele Grattarola 37 Dec 13, 2022
TAUFE: Task-Agnostic Undesirable Feature DeactivationUsing Out-of-Distribution Data

A deep neural network (DNN) has achieved great success in many machine learning tasks by virtue of its high expressive power. However, its prediction can be easily biased to undesirable features, whi

KAIST Data Mining Lab 8 Dec 07, 2022
Codes for the ICCV'21 paper "FREE: Feature Refinement for Generalized Zero-Shot Learning"

FREE This repository contains the reference code for the paper "FREE: Feature Refinement for Generalized Zero-Shot Learning". [arXiv][Paper] 1. Prepar

Shiming Chen 28 Jul 29, 2022