ML Optimizers from scratch using JAX

Last update: Jul 29, 2022

Overview

ML Optimizers from scratch using JAX

Implementations of some popular optimizers from scratch for a simple model i.e., Linear Regression on a dataset of 5 features. The goal of this project was to understand how these optimizers work under the hood and try to do a toy implementation myself. I also use a bit of JAX magic to perform the differentiation of the loss function w.r.t to the weights and the bias without explicitly writing their derivatives as a separate function. This can help to generalize this notebook for other types of loss functions as well.

The optimizers I have implemented are -

Batch Gradient Descent
Batch Gradient Descent + Momentum
Nesterov Accelerated Momentum
Adagrad
RMSprop
Adam
Adamax
Nadam
Adabelief

References -

Owner

Shreyansh Singh

AI Engineer at @Mastercard. Ex-Intern at @Samsung. CS Grad, IIT (BHU) Varanasi. CTF player with @abs0lut3pwn4g3

GitHub Repository

We have a dataset of user performances. The project is to develop a machine learning model that will predict the salaries of baseball players.

Salary-Prediction-with-Machine-Learning 1. Business Problem Can a machine learning project be implemented to estimate the salaries of baseball players

9 Oct 14, 2022

An open source framework that provides a simple, universal API for building distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library.

Ray provides a simple, universal API for building distributed applications. Ray is packaged with the following libraries for accelerating machine lear

23.3k Dec 31, 2022

The easy way to combine mlflow, hydra and optuna into one machine learning pipeline.

mlflow_hydra_optuna_the_easy_way The easy way to combine mlflow, hydra and optuna into one machine learning pipeline. Objective TODO Usage 1. build do

9 Sep 09, 2022

PLUR is a collection of source code datasets suitable for graph-based machine learning.

PLUR (Programming-Language Understanding and Repair) is a collection of source code datasets suitable for graph-based machine learning. We provide scripts for downloading, processing, and loading the

76 Nov 25, 2022

Apple-voice-recognition - Machine Learning

Apple-voice-recognition Machine Learning How does Siri work? Siri is based on large-scale Machine Learning systems that employ many aspects of data sc

1 Oct 22, 2021

Programming assignments and quizzes from all courses within the Machine Learning Engineering for Production (MLOps) specialization offered by deeplearning.ai

Machine Learning Engineering for Production (MLOps) Specialization on Coursera (offered by deeplearning.ai) Programming assignments from all courses i

173 Jan 05, 2023

Scikit-Learn useful pre-defined Pipelines Hub

Scikit-Pipes Scikit-Learn useful pre-defined Pipelines Hub Usage: Install scikit-pipes It's advised to install sklearn-genetic using a virtual env, in

1 Apr 26, 2022

QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.

152 Jan 02, 2023

This jupyter notebook project was completed by me and my friend using the dataset from Kaggle

ARM This jupyter notebook project was completed by me and my friend using the dataset from Kaggle. The world Happiness 2017, which ranks 155 countries

1 Jan 23, 2022

Management of exclusive GPU access for distributed machine learning workloads

TensorHive is an open source tool for managing computing resources used by multiple users across distributed hosts. It focuses on granting

131 Dec 12, 2022

Hypernets: A General Automated Machine Learning framework to simplify the development of End-to-end AutoML toolkits in specific domains.

A General Automated Machine Learning framework to simplify the development of End-to-end AutoML toolkits in specific domains.

216 Dec 23, 2022

A Streamlit demo to interactively visualize Uber pickups in New York City

Streamlit Demo: Uber Pickups in New York City A Streamlit demo written in pure Python to interactively visualize Uber pickups in New York City. View t

230 Dec 28, 2022

Uplift modeling and causal inference with machine learning algorithms

Disclaimer This project is stable and being incubated for long-term support. It may contain new experimental code, for which APIs are subject to chang

3.7k Jan 07, 2023

Tools for diffing and merging of Jupyter notebooks.

nbdime provides tools for diffing and merging of Jupyter Notebooks.

2.3k Jan 03, 2023

AutoX是一个高效的自动化机器学习工具，它主要针对于表格类型的数据挖掘竞赛。它的特点包括: 效果出色、简单易用、通用、自动化、灵活。

English | 简体中文 AutoX是什么？ AutoX一个高效的自动化机器学习工具，它主要针对于表格类型的数据挖掘竞赛。它的特点包括: 效果出色: AutoX在多个kaggle数据集上，效果显著优于其他解决方案(见效果对比)。简单易用: AutoX的接口和sklearn类似，方便上手使用。

431 Dec 28, 2022

Automated Machine Learning Pipeline for tabular data. Designed for predictive maintenance applications, failure identification, failure prediction, condition monitoring, etc.

10 May 15, 2022

ML Optimizers from scratch using JAX

Related tags

Overview

ML Optimizers from scratch using JAX

Owner

Shreyansh Singh

We have a dataset of user performances. The project is to develop a machine learning model that will predict the salaries of baseball players.

An open source framework that provides a simple, universal API for building distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library.

The easy way to combine mlflow, hydra and optuna into one machine learning pipeline.

PLUR is a collection of source code datasets suitable for graph-based machine learning.

Apple-voice-recognition - Machine Learning

Programming assignments and quizzes from all courses within the Machine Learning Engineering for Production (MLOps) specialization offered by deeplearning.ai

Scikit-Learn useful pre-defined Pipelines Hub

QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.

This jupyter notebook project was completed by me and my friend using the dataset from Kaggle

Management of exclusive GPU access for distributed machine learning workloads

Hypernets: A General Automated Machine Learning framework to simplify the development of End-to-end AutoML toolkits in specific domains.

A Streamlit demo to interactively visualize Uber pickups in New York City

Uplift modeling and causal inference with machine learning algorithms

Tools for diffing and merging of Jupyter notebooks.

AutoX是一个高效的自动化机器学习工具，它主要针对于表格类型的数据挖掘竞赛。它的特点包括: 效果出色、简单易用、通用、自动化、灵活。

Automated Machine Learning Pipeline for tabular data. Designed for predictive maintenance applications, failure identification, failure prediction, condition monitoring, etc.

Little Ball of Fur - A graph sampling extension library for NetworKit and NetworkX (CIKM 2020)

YouTube Spam Detection with python

CrayLabs and user contibuted examples of using SmartSim for various simulation and machine learning applications.

Python module for performing linear regression for data with measurement errors and intrinsic scatter

ML Optimizers from scratch using JAX

Related tags

Overview

ML Optimizers from scratch using JAX

Owner

Shreyansh Singh

We have a dataset of user performances. The project is to develop a machine learning model that will predict the salaries of baseball players.

An open source framework that provides a simple, universal API for building distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library.

The easy way to combine mlflow, hydra and optuna into one machine learning pipeline.

PLUR is a collection of source code datasets suitable for graph-based machine learning.

Apple-voice-recognition - Machine Learning

Programming assignments and quizzes from all courses within the Machine Learning Engineering for Production (MLOps) specialization offered by deeplearning.ai

Scikit-Learn useful pre-defined Pipelines Hub

QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.

This jupyter notebook project was completed by me and my friend using the dataset from Kaggle

Management of exclusive GPU access for distributed machine learning workloads

Hypernets: A General Automated Machine Learning framework to simplify the development of End-to-end AutoML toolkits in specific domains.

A Streamlit demo to interactively visualize Uber pickups in New York City

Uplift modeling and causal inference with machine learning algorithms

Tools for diffing and merging of Jupyter notebooks.

AutoX是一个高效的自动化机器学习工具，它主要针对于表格类型的数据挖掘竞赛。 它的特点包括: 效果出色、简单易用、通用、自动化、灵活。

Automated Machine Learning Pipeline for tabular data. Designed for predictive maintenance applications, failure identification, failure prediction, condition monitoring, etc.

Little Ball of Fur - A graph sampling extension library for NetworKit and NetworkX (CIKM 2020)

YouTube Spam Detection with python

CrayLabs and user contibuted examples of using SmartSim for various simulation and machine learning applications.

Python module for performing linear regression for data with measurement errors and intrinsic scatter

AutoX是一个高效的自动化机器学习工具，它主要针对于表格类型的数据挖掘竞赛。它的特点包括: 效果出色、简单易用、通用、自动化、灵活。