A MNIST-like fashion product database. Benchmark

Overview

Fashion-MNIST

GitHub stars Gitter Readme-CN Readme-JA License: MIT Year-In-Review

Table of Contents

Fashion-MNIST is a dataset of Zalando's article imagesโ€”consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes. We intend Fashion-MNIST to serve as a direct drop-in replacement for the original MNIST dataset for benchmarking machine learning algorithms. It shares the same image size and structure of training and testing splits.

Here's an example of how the data looks (each class takes three-rows):

Why we made Fashion-MNIST

The original MNIST dataset contains a lot of handwritten digits. Members of the AI/ML/Data Science community love this dataset and use it as a benchmark to validate their algorithms. In fact, MNIST is often the first dataset researchers try. "If it doesn't work on MNIST, it won't work at all", they said. "Well, if it does work on MNIST, it may still fail on others."

To Serious Machine Learning Researchers

Seriously, we are talking about replacing MNIST. Here are some good reasons:

Get the Data

Many ML libraries already include Fashion-MNIST data/API, give it a try!

You can use direct links to download the dataset. The data is stored in the same format as the original MNIST data.

Name Content Examples Size Link MD5 Checksum
train-images-idx3-ubyte.gz training set images 60,000 26 MBytes Download 8d4fb7e6c68d591d4c3dfef9ec88bf0d
train-labels-idx1-ubyte.gz training set labels 60,000 29 KBytes Download 25c81989df183df01b3e8a0aad5dffbe
t10k-images-idx3-ubyte.gz test set images 10,000 4.3 MBytes Download bef4ecab320f06d8554ea6380940ec79
t10k-labels-idx1-ubyte.gz test set labels 10,000 5.1 KBytes Download bb300cfdad3c16e7a12a480ee83cd310

Alternatively, you can clone this GitHub repository; the dataset appears under data/fashion. This repo also contains some scripts for benchmark and visualization.

git clone [email protected]:zalandoresearch/fashion-mnist.git

Labels

Each training and test example is assigned to one of the following labels:

Label Description
0 T-shirt/top
1 Trouser
2 Pullover
3 Dress
4 Coat
5 Sandal
6 Shirt
7 Sneaker
8 Bag
9 Ankle boot

Usage

Loading data with Python (requires NumPy)

Use utils/mnist_reader in this repo:

import mnist_reader
X_train, y_train = mnist_reader.load_mnist('data/fashion', kind='train')
X_test, y_test = mnist_reader.load_mnist('data/fashion', kind='t10k')

Loading data with Tensorflow

Make sure you have downloaded the data and placed it in data/fashion. Otherwise, Tensorflow will download and use the original MNIST.

from tensorflow.examples.tutorials.mnist import input_data
data = input_data.read_data_sets('data/fashion')

data.train.next_batch(BATCH_SIZE)

Note, Tensorflow supports passing in a source url to the read_data_sets. You may use:

data = input_data.read_data_sets('data/fashion', source_url='http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/')

Also, an official Tensorflow tutorial of using tf.keras, a high-level API to train Fashion-MNIST can be found here.

Loading data with other machine learning libraries

To date, the following libraries have included Fashion-MNIST as a built-in dataset. Therefore, you don't need to download Fashion-MNIST by yourself. Just follow their API and you are ready to go.

You are welcome to make pull requests to other open-source machine learning packages, improving their support to Fashion-MNIST dataset.

Loading data with other languages

As one of the Machine Learning community's most popular datasets, MNIST has inspired people to implement loaders in many different languages. You can use these loaders with the Fashion-MNIST dataset as well. (Note: may require decompressing first.) To date, we haven't yet tested all of these loaders with Fashion-MNIST.

Benchmark

We built an automatic benchmarking system based on scikit-learn that covers 129 classifiers (but no deep learning) with different parameters. Find the results here.

You can reproduce the results by running benchmark/runner.py. We recommend building and deploying this Dockerfile.

You are welcome to submit your benchmark; simply create a new issue and we'll list your results here. Before doing that, please make sure it does not already appear in this list. Visit our contributor guidelines for additional details.

The table below collects the submitted benchmarks. Note that we haven't yet tested these results. You are welcome to validate the results using the code provided by the submitter. Test accuracy may differ due to the number of epoch, batch size, etc. To correct this table, please create a new issue.

Classifier Preprocessing Fashion test accuracy MNIST test accuracy Submitter Code
2 Conv+pooling None 0.876 - Kashif Rasul ๐Ÿ”—
2 Conv+pooling None 0.916 - Tensorflow's doc ๐Ÿ”—
2 Conv+pooling+ELU activation (PyTorch) None 0.903 - @AbhirajHinge ๐Ÿ”—
2 Conv Normalization, random horizontal flip, random vertical flip, random translation, random rotation. 0.919 0.971 Kyriakos Efthymiadis ๐Ÿ”—
2 Conv <100K parameters None 0.925 0.992 @hardmaru ๐Ÿ”—
2 Conv ~113K parameters Normalization 0.922 0.993 Abel G. ๐Ÿ”—
2 Conv+3 FC ~1.8M parameters Normalization 0.932 0.994 @Xfan1025 ๐Ÿ”—
2 Conv+3 FC ~500K parameters Augmentation, batch normalization 0.934 0.994 @cmasch ๐Ÿ”—
2 Conv+pooling+BN None 0.934 - @khanguyen1207 ๐Ÿ”—
2 Conv+2 FC Random Horizontal Flips 0.939 - @ashmeet13 ๐Ÿ”—
3 Conv+2 FC None 0.907 - @Cenk BircanoฤŸlu ๐Ÿ”—
3 Conv+pooling+BN None 0.903 0.994 @meghanabhange ๐Ÿ”—
3 Conv+pooling+2 FC+dropout None 0.926 - @Umberto Griffo ๐Ÿ”—
3 Conv+BN+pooling None 0.921 0.992 @gchhablani ๐Ÿ”—
5 Conv+BN+pooling None 0.931 - @Noumanmufc1 ๐Ÿ”—
CNN with optional shortcuts, dense-like connectivity standardization+augmentation+random erasing 0.947 - @kennivich ๐Ÿ”—
GRU+SVM None 0.888 0.965 @AFAgarap ๐Ÿ”—
GRU+SVM with dropout None 0.897 0.988 @AFAgarap ๐Ÿ”—
WRN40-4 8.9M params standard preprocessing (mean/std subtraction/division) and augmentation (random crops/horizontal flips) 0.967 - @ajbrock ๐Ÿ”— ๐Ÿ”—
DenseNet-BC 768K params standard preprocessing (mean/std subtraction/division) and augmentation (random crops/horizontal flips) 0.954 - @ajbrock ๐Ÿ”— ๐Ÿ”—
MobileNet augmentation (horizontal flips) 0.950 - @่‹ๅ‰‘ๆž— ๐Ÿ”—
ResNet18 Normalization, random horizontal flip, random vertical flip, random translation, random rotation. 0.949 0.979 Kyriakos Efthymiadis ๐Ÿ”—
GoogleNet with cross-entropy loss None 0.937 - @Cenk BircanoฤŸlu ๐Ÿ”—
AlexNet with Triplet loss None 0.899 - @Cenk BircanoฤŸlu ๐Ÿ”—
SqueezeNet with cyclical learning rate 200 epochs None 0.900 - @snakers4 ๐Ÿ”—
Dual path network with wide resnet 28-10 standard preprocessing (mean/std subtraction/division) and augmentation (random crops/horizontal flips) 0.957 - @Queequeg ๐Ÿ”—
MLP 256-128-100 None 0.8833 - @heitorrapela ๐Ÿ”—
VGG16 26M parameters None 0.935 - @QuantumLiu ๐Ÿ”— ๐Ÿ”—
WRN-28-10 standard preprocessing (mean/std subtraction/division) and augmentation (random crops/horizontal flips) 0.959 - @zhunzhong07 ๐Ÿ”—
WRN-28-10 + Random Erasing standard preprocessing (mean/std subtraction/division) and augmentation (random crops/horizontal flips) 0.963 - @zhunzhong07 ๐Ÿ”—
Human Performance Crowd-sourced evaluation of human (with no fashion expertise) performance. 1000 randomly sampled test images, 3 labels per image, majority labelling. 0.835 - Leo -
Capsule Network 8M parameters Normalization and shift at most 2 pixel and horizontal flip 0.936 - @XifengGuo ๐Ÿ”—
HOG+SVM HOG 0.926 - @subalde ๐Ÿ”—
XgBoost scaling the pixel values to mean=0.0 and var=1.0 0.898 0.958 @anktplwl91 ๐Ÿ”—
DENSER - 0.953 0.997 @fillassuncao ๐Ÿ”— ๐Ÿ”—
Dyra-Net Rescale to unit interval 0.906 - @Dirk Schรคfer ๐Ÿ”— ๐Ÿ”—
Google AutoML 24 compute hours (higher quality) 0.939 - @Sebastian Heinz ๐Ÿ”—
Fastai Resnet50+Fine-tuning+Softmax on last layer's activations 0.9312 - @Sayak ๐Ÿ”—

Other Explorations of Fashion-MNIST

Fashion-MNIST: Year in Review

Fashion-MNIST on Google Scholar

Generative adversarial networks (GANs)

Clustering

Video Tutorial

Machine Learning Meets Fashion by Yufeng G @ Google Cloud

Machine Learning Meets Fashion

Introduction to Kaggle Kernels by Yufeng G @ Google Cloud

Introduction to Kaggle Kernels

ๅŠจๆ‰‹ๅญฆๆทฑๅบฆๅญฆไน  by Mu Li @ Amazon AI

MXNet/Gluonไธญๆ–‡้ข‘้“

Apache MXNet์œผ๋กœ ๋ฐฐ์›Œ๋ณด๋Š” ๋”ฅ๋Ÿฌ๋‹(Deep Learning) - ๊น€๋ฌดํ˜„ (AWS ์†”๋ฃจ์…˜์ฆˆ์•„ํ‚คํ…ํŠธ)

Apache MXNet์œผ๋กœ ๋ฐฐ์›Œ๋ณด๋Š” ๋”ฅ๋Ÿฌ๋‹(Deep Learning)

Visualization

t-SNE on Fashion-MNIST (left) and original MNIST (right)

PCA on Fashion-MNIST (left) and original MNIST (right)

UMAP on Fashion-MNIST (left) and original MNIST (right)

PyMDE on Fashion-MNIST (left) and original MNIST (right)

Contributing

Thanks for your interest in contributing! There are many ways to get involved; start with our contributor guidelines and then check these open issues for specific tasks.

Contact

To discuss the dataset, please use Gitter.

Citing Fashion-MNIST

If you use Fashion-MNIST in a scientific publication, we would appreciate references to the following paper:

Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. Han Xiao, Kashif Rasul, Roland Vollgraf. arXiv:1708.07747

Biblatex entry:

@online{xiao2017/online,
  author       = {Han Xiao and Kashif Rasul and Roland Vollgraf},
  title        = {Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms},
  date         = {2017-08-28},
  year         = {2017},
  eprintclass  = {cs.LG},
  eprinttype   = {arXiv},
  eprint       = {cs.LG/1708.07747},
}

Who is citing Fashion-MNIST?

License

The MIT License (MIT) Copyright ยฉ [2017] Zalando SE, https://tech.zalando.com

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the โ€œSoftwareโ€), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED โ€œAS ISโ€, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Owner
Zalando Research
Repositories of the research branch of Zalando SE
Zalando Research
Using deep actor-critic model to learn best strategies in pair trading

Deep-Reinforcement-Learning-in-Stock-Trading Using deep actor-critic model to learn best strategies in pair trading Abstract Partially observed Markov

281 Dec 09, 2022
Vignette is a face tracking software for characters using osu!framework.

Vignette is a face tracking software for characters using osu!framework. Unlike most solutions, Vignette is: Made with osu!framework, the game framewo

Vignette 412 Dec 28, 2022
Parasite: a tool allowing you to compress and decompress files, to reduce their size

๐Ÿฆ  Parasite ๐Ÿฆ  Parasite is a tool written in Python3 allowing you to "compress" any file, reducing its size. โญ Features โญ + Fast + Good optimization,

Billy 30 Nov 25, 2022
Do you like Quick, Draw? Well what if you could train/predict doodles drawn inside Streamlit? Also draws lines, circles and boxes over background images for annotation.

Streamlit - Drawable Canvas Streamlit component which provides a sketching canvas using Fabric.js. Features Draw freely, lines, circles, boxes and pol

Fanilo Andrianasolo 325 Dec 28, 2022
Organseg dags - The repository contains the codebase for multi-organ segmentation with directed acyclic graphs (DAGs) in CT.

Organseg dags - The repository contains the codebase for multi-organ segmentation with directed acyclic graphs (DAGs) in CT.

yzf 1 Jun 12, 2022
Step by Step on how to create an vision recognition model using LOBE.ai, export the model and run the model in an Azure Function

Step by Step on how to create an vision recognition model using LOBE.ai, export the model and run the model in an Azure Function

El Bruno 3 Mar 30, 2022
This project deals with the detection of skin lesions within the ISICs dataset using YOLOv3 Object Detection with Darknet.

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. Skin Lesion detection using YOLO This project deal

Lalith Veerabhadrappa Badiger 1 Nov 22, 2021
Source code for the ACL-IJCNLP 2021 paper entitled "T-DNA: Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation" by Shizhe Diao et al.

T-DNA Source code for the ACL-IJCNLP 2021 paper entitled Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adapta

shizhediao 17 Dec 22, 2022
Prml - Repository of notes, code and notebooks in Python for the book Pattern Recognition and Machine Learning by Christopher Bishop

Pattern Recognition and Machine Learning (PRML) This project contains Jupyter notebooks of many the algorithms presented in Christopher Bishop's Patte

Gerardo Durรกn-Martรญn 1k Jan 07, 2023
minimizer-space de Bruijn graphs (mdBG) for whole genome assembly

rust-mdbg: Minimizer-space de Bruijn graphs (mdBG) for whole-genome assembly rust-mdbg is an ultra-fast minimizer-space de Bruijn graph (mdBG) impleme

BarฤฑลŸ Ekim 148 Dec 01, 2022
Distance correlation and related E-statistics in Python

dcor dcor: distance correlation and related E-statistics in Python. E-statistics are functions of distances between statistical observations in metric

Carlos Ramos Carreรฑo 108 Dec 27, 2022
The Balloon Learning Environment - flying stratospheric balloons with deep reinforcement learning.

Balloon Learning Environment Docs The Balloon Learning Environment (BLE) is a simulator for stratospheric balloons. It is designed as a benchmark envi

Google 87 Dec 25, 2022
Repository for benchmarking graph neural networks

Benchmarking Graph Neural Networks Updates Nov 2, 2020 Project based on DGL 0.4.2. See the relevant dependencies defined in the environment yml files

NTU Graph Deep Learning Lab 2k Jan 03, 2023
Pytorch implement of 'Unmixing based PAN guided fusion network for hyperspectral imagery'

Pgnet There's a improved version compared with the publication in Tgrs with the modification in the deduction of the PDIN block: https://arxiv.org/abs

5 Jul 01, 2022
auto-tuning momentum SGD optimizer

YellowFin YellowFin is an auto-tuning optimizer based on momentum SGD which requires no manual specification of learning rate and momentum. It measure

Jian Zhang 288 Nov 19, 2022
Contrastive unpaired image-to-image translation, faster and lighter training than cyclegan (ECCV 2020, in PyTorch)

Contrastive Unpaired Translation (CUT) video (1m) | video (10m) | website | paper We provide our PyTorch implementation of unpaired image-to-image tra

1.7k Dec 27, 2022
Code accompanying the paper Say As You Wish: Fine-grained Control of Image Caption Generation with Abstract Scene Graphs (Chen et al., CVPR 2020, Oral).

Say As You Wish: Fine-grained Control of Image Caption Generation with Abstract Scene Graphs This repository contains PyTorch implementation of our pa

Shizhe Chen 178 Dec 29, 2022
EGNN - Implementation of E(n)-Equivariant Graph Neural Networks, in Pytorch

EGNN - Pytorch Implementation of E(n)-Equivariant Graph Neural Networks, in Pytorch. May be eventually used for Alphafold2 replication. This

Phil Wang 259 Jan 04, 2023
Fashion Landmark Estimation with HRNet

HRNet for Fashion Landmark Estimation (Modified from deep-high-resolution-net.pytorch) Introduction This code applies the HRNet (Deep High-Resolution

SVIP Lab 91 Dec 26, 2022
This repository contains code released by Google Research.

This repository contains code released by Google Research.

Google Research 26.6k Dec 31, 2022