Efficient Deep Learning Systems

This repository contains materials for the Efficient Deep Learning Systems course taught at the Faculty of Computer Science of HSE University and Yandex School of Data Analysis.

Syllabus

Week 1: Introduction
- Lecture: Course overview and organizational details. Core concepts of the GPU architecture and CUDA API.
- Seminar: CUDA operations in PyTorch. Introduction to benchmarking.
Week 2: Basics of distributed ML
- Lecture: Introduction to distributed training. Process-based communication. Parameter Server architecture.
- Seminar: Multiprocessing basics. Parallel GloVe training.
Week 3: Data-parallel training and All-Reduce
- Lecture: Data-parallel training of neural networks. All-Reduce and its efficient implementations.
- Seminar: Introduction to PyTorch Distributed. Data-parallel training primitives.
Week 4: Memory-efficient and model-parallel training
Week 5: Profiling DL code, training-time optimizations
Week 6: Basics of Python application deployment
Week 7: Software for serving neural networks
Week 8: Optimizing models for faster inference
Week 9: Experiment tracking, model and data versioning
Week 10: Testing, debugging and monitoring of models

Grading

There will be a total of 4 home assignments (some of them spread over several weeks). The final grade is a weighted sum of per-assignment grades. Please refer to the course page of your institution for details.

Efficient Deep Learning Systems course

Related tags

Overview

Efficient Deep Learning Systems

Syllabus

Grading

Staff

Owner

Max Ryabinin

PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in clustering (CVPR2021)

Benchmark spaces - Benchmarks of how well different two dimensional spaces work for clustering algorithms

General Multi-label Image Classification with Transformers

TensorFlow implementation of Adaptive Information Transfer Multi-task (AITM) framework. Code for the paper submitted to KDD21: Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning for Customer Acquisition.

Code for the Image similarity challenge.

pytorch implementation for Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network arXiv:1609.04802

A PyTorch implementation of SIN: Superpixel Interpolation Network

Contains code for the paper "Vision Transformers are Robust Learners".

3D-Reconstruction 基于深度学习方法的单目多视图三维重建

Weakly Supervised Segmentation by Tensorflow.

Suite of 500 procedurally-generated NLP tasks to study language model adaptability

Easy-to-use library to boost AI inference leveraging state-of-the-art optimization techniques.

You can draw the corresponding bounding box into the image and save it according to the result file (txt format) run by the tracker.

Code for the paper: Audio-Visual Scene Analysis with Self-Supervised Multisensory Features

A model that attempts to learn and benefit from data collected on card counting.

PyTorch implementation of Value Iteration Networks (VIN): Clean, Simple and Modular. Visualization in Visdom.

Anonymize BLM Protest Images

Practical and Real-world applications of ML based on the homework of Hung-yi Lee Machine Learning Course 2021

Official implementation of cosformer-attention in cosFormer: Rethinking Softmax in Attention

Code for Mesh Convolution Using a Learned Kernel Basis