ML Optimizers from scratch using JAX

Last update: Jul 29, 2022

Overview

ML Optimizers from scratch using JAX

Implementations of some popular optimizers from scratch for a simple model i.e., Linear Regression on a dataset of 5 features. The goal of this project was to understand how these optimizers work under the hood and try to do a toy implementation myself. I also use a bit of JAX magic to perform the differentiation of the loss function w.r.t to the weights and the bias without explicitly writing their derivatives as a separate function. This can help to generalize this notebook for other types of loss functions as well.

The optimizers I have implemented are -

Batch Gradient Descent
Batch Gradient Descent + Momentum
Nesterov Accelerated Momentum
Adagrad
RMSprop
Adam
Adamax
Nadam
Adabelief

References -

Owner

Shreyansh Singh

AI Engineer at @Mastercard. Ex-Intern at @Samsung. CS Grad, IIT (BHU) Varanasi. CTF player with @abs0lut3pwn4g3

GitHub Repository

Bayesian Modeling and Computation in Python

Bayesian Modeling and Computation in Python Open access and Code This repository contains the open access version of the text and the code examples in

339 Jan 02, 2023

pure-predict: Machine learning prediction in pure Python

pure-predict speeds up and slims down machine learning prediction applications. It is a foundational tool for serverless inference or small batch prediction with popular machine learning frameworks l

84 Dec 29, 2022

PROTEIN EXPRESSION ANALYSIS FOR DOWN SYNDROME

PROTEIN-EXPRESSION-ANALYSIS-FOR-DOWN-SYNDROME Down syndrome (DS) is a chromosomal disorder where organisms have an extra chromosome 21, sometimes know

1 Jan 20, 2022

ETNA – time series forecasting framework

ETNA Time Series Library Predict your time series the easiest way Homepage | Documentation | Tutorials | Contribution Guide | Release Notes ETNA is an

675 Jan 08, 2023

AutoX是一个高效的自动化机器学习工具，它主要针对于表格类型的数据挖掘竞赛。它的特点包括: 效果出色、简单易用、通用、自动化、灵活。

English | 简体中文 AutoX是什么？ AutoX一个高效的自动化机器学习工具，它主要针对于表格类型的数据挖掘竞赛。它的特点包括: 效果出色: AutoX在多个kaggle数据集上，效果显著优于其他解决方案(见效果对比)。简单易用: AutoX的接口和sklearn类似，方便上手使用。

431 Dec 28, 2022

Self Organising Map (SOM) for clustering of atomistic samples through unsupervised learning.

Self Organising Map for Clustering of Atomistic Samples - V2 Description Self Organising Map (also known as Kohonen Network) implemented in Python for

0 Nov 16, 2021

使用数学和计算机知识投机倒把

偷鸡不成项目集锦坦率地讲，涉及金融市场的好策略如果公开，必然导致使用的人多，最后策略变差。所以这个仓库只收集我目前失败了的案例。加密货币组合套利中国体育彩票预测我赚不上钱的项目，也许可以帮助更有能力的人去赚钱。

28 Dec 29, 2022

A Collection of Conference & School Notes in Machine Learning 🦄📝🎉

Machine Learning Conference & Summer School Notes. 🦄📝🎉

558 Dec 28, 2022

Practical Time-Series Analysis, published by Packt

Practical Time-Series Analysis This is the code repository for Practical Time-Series Analysis, published by Packt. It contains all the supporting proj

325 Dec 23, 2022

A Lucid Framework for Transparent and Interpretable Machine Learning Models.

Currently a Beta-Version lucidmode is an open-source, low-code and lightweight Python framework for transparent and interpretable machine learning mod

15 Aug 12, 2022

AutoOED: Automated Optimal Experiment Design Platform

AutoOED is an optimal experiment design platform powered with automated machine learning to accelerate the discovery of optimal solutions. Our platform solves multi-objective optimization problems an

107 Jan 03, 2023

Python library which makes it possible to dynamically mask/anonymize data using JSON string or python dict rules in a PySpark environment.

pyspark-anonymizer Python library which makes it possible to dynamically mask/anonymize data using JSON string or python dict rules in a PySpark envir

6 Jun 30, 2022

A simple example of ML classification, cross validation, and visualization of feature importances

Simple-Classifier This is a basic example of how to use several different libraries for classification and ensembling, mostly with sklearn. Example as

2 Aug 25, 2022

The easy way to combine mlflow, hydra and optuna into one machine learning pipeline.

mlflow_hydra_optuna_the_easy_way The easy way to combine mlflow, hydra and optuna into one machine learning pipeline. Objective TODO Usage 1. build do

9 Sep 09, 2022

Ml based project which uses regression technique to predict the price.

Price-Predictor Ml based project which uses regression technique to predict the price. I have used various regression models and finds the model with

1 Jul 09, 2022

Distributed Deep learning with Keras & Spark

Elephas: Distributed Deep Learning with Keras & Spark Elephas is an extension of Keras, which allows you to run distributed deep learning models at sc

1.6k Dec 29, 2022

Hypernets: A General Automated Machine Learning framework to simplify the development of End-to-end AutoML toolkits in specific domains.

A General Automated Machine Learning framework to simplify the development of End-to-end AutoML toolkits in specific domains.

216 Dec 23, 2022

ML Optimizers from scratch using JAX

Related tags

Overview

ML Optimizers from scratch using JAX

Owner

Shreyansh Singh

Bayesian Modeling and Computation in Python

pure-predict: Machine learning prediction in pure Python

PROTEIN EXPRESSION ANALYSIS FOR DOWN SYNDROME

ETNA – time series forecasting framework

AutoX是一个高效的自动化机器学习工具，它主要针对于表格类型的数据挖掘竞赛。它的特点包括: 效果出色、简单易用、通用、自动化、灵活。

Self Organising Map (SOM) for clustering of atomistic samples through unsupervised learning.

使用数学和计算机知识投机倒把

A Collection of Conference & School Notes in Machine Learning 🦄📝🎉

Practical Time-Series Analysis, published by Packt

A Lucid Framework for Transparent and Interpretable Machine Learning Models.

AutoOED: Automated Optimal Experiment Design Platform

Python library which makes it possible to dynamically mask/anonymize data using JSON string or python dict rules in a PySpark environment.

A simple example of ML classification, cross validation, and visualization of feature importances

The easy way to combine mlflow, hydra and optuna into one machine learning pipeline.

Ml based project which uses regression technique to predict the price.

Distributed Deep learning with Keras & Spark

Hypernets: A General Automated Machine Learning framework to simplify the development of End-to-end AutoML toolkits in specific domains.

Karate Club: An API Oriented Open-source Python Framework for Unsupervised Learning on Graphs (CIKM 2020)

It is a forest of random projection trees

database for artificial intelligence/machine learning data

ML Optimizers from scratch using JAX

Related tags

Overview

ML Optimizers from scratch using JAX

Owner

Shreyansh Singh

Bayesian Modeling and Computation in Python

pure-predict: Machine learning prediction in pure Python

PROTEIN EXPRESSION ANALYSIS FOR DOWN SYNDROME

ETNA – time series forecasting framework

AutoX是一个高效的自动化机器学习工具，它主要针对于表格类型的数据挖掘竞赛。 它的特点包括: 效果出色、简单易用、通用、自动化、灵活。

Self Organising Map (SOM) for clustering of atomistic samples through unsupervised learning.

使用数学和计算机知识投机倒把

A Collection of Conference & School Notes in Machine Learning 🦄📝🎉

Practical Time-Series Analysis, published by Packt

A Lucid Framework for Transparent and Interpretable Machine Learning Models.

AutoOED: Automated Optimal Experiment Design Platform

Python library which makes it possible to dynamically mask/anonymize data using JSON string or python dict rules in a PySpark environment.

A simple example of ML classification, cross validation, and visualization of feature importances

The easy way to combine mlflow, hydra and optuna into one machine learning pipeline.

Ml based project which uses regression technique to predict the price.

Distributed Deep learning with Keras & Spark

Hypernets: A General Automated Machine Learning framework to simplify the development of End-to-end AutoML toolkits in specific domains.

Karate Club: An API Oriented Open-source Python Framework for Unsupervised Learning on Graphs (CIKM 2020)

It is a forest of random projection trees

database for artificial intelligence/machine learning data

AutoX是一个高效的自动化机器学习工具，它主要针对于表格类型的数据挖掘竞赛。它的特点包括: 效果出色、简单易用、通用、自动化、灵活。