KuaiRec: A Fully-observed Dataset for Recommender Systems (Density: Almost 100%)

KuaiRec is a real-world dataset collected from the recommendation logs of the video-sharing mobile app Kuaishou. For now, it is the first dataset that contains a fully observed user-item interaction matrix. For the term "fully observed", we mean there are almost no missing values in the user-item matrix, i.e., each user has viewed each video and then left feedback.

The following figure illustrates the user-item matrices in traditional datasets and KuaiRec.

With all user's preference known, KuaiRec can used in offline evaluation (i.e., offline A/B test) for recommendation models. It can benefit lots of research directions, such as unbiased recommendation, interactive/conversational recommendation, or reinforcement learning (RL) and off-policy evaluation (OPE) for recommendation.

If you use it in your work, please cite our paper:

@article{gao2022kuairec,
  title={KuaiRec: A Fully-observed Dataset for Recommender Systems}, 
  author={Chongming Gao and Shijun Li and Wenqiang Lei and Biao Li and Peng Jiang and Jiawei Chen and Xiangnan He and Jiaxin Mao and Tat-Seng Chua},
  journal={arXiv preprint arXiv:2202.10842},
  year={2022}
}

This repository lists the example codes in evaluating conversational recommendation as described in the paper.

We provide some simple statistics of this dataset here . It is generated by Statistics_KuaiRec.ipynb. You can do it online at Google Colab .

News ! ! ! ! !

2022.05.16: We update the dataset to version 2.0. We made the following changes:

• We removed the unused video ID=1225 from all tables having the field video_id and reindex the rest videos, i.e., ID = ID - 1 if ID > 1225.
• We added two tables to enhance the side information for users and videos, respectively. See 4.item_daily_feet.csv and 5. user_feat.csv under the data description section for details.

Download the data

We provides several options to download this dataset:

Option 1. Download via the "wget" command.

 wget https://chongming.myds.me:61364/data/KuaiRec.zip --no-check-certificate
 unzip KuaiRec.zip

Option 2. Download manually throughs the following links:

• Optional link 1: Google Drive

• Optional link 2: USTC Drive (中科大)

The script loaddata.py provides a simple way to load the data via Pandas in Python.

Data Descriptions

KuaiRec contains millions of user-item interactions as well as the side information include the item categorires and social network. Four files are included in the download data:

KuaiRec
├── data
│   ├── big_matrix.csv          
│   ├── small_matrix.csv
│   ├── social_network.csv
│   └── item_categories.csv

The statistics of the small matrix and big matrix in KuaiRec.

Note that the density of small matrix is 99.6% instead of 100% because some users have explicitly indicated that they would not be willing to receive recommendations from certain authors. I.e., They blocked these videos.

1. Descriptions of the fields in big_matrix.csv and small_matrix.csv.

The "watch_ratio" can be deemed as the label of the interaction. Note: there is no "like" signal for this dataset. If you need this binary signal in your scenarios, you can create it yourself. E.g., like = 1 if watch_ratio > 2.0.

2. Descriptions of the fields in social_network.csv

3. Descriptions of the fields in item_categories.csv.

4. Descriptions of the fields in item_daily_feet.csv. (Added on 2022.05.16)

5. Descriptions of the fields in user_feat.csv (Added on 2022.05.16)