Implemented fully documented Particle Swarm Optimization algorithm (basic model with few advanced features) using Python programming language

Overview

Enhanced Particle Swarm Optimization (PSO) with Python

GitHub license GitHub issues

Implemented fully documented Particle Swarm Optimization (PSO) algorithm in Python which includes a basic model along with few advanced features such as updating inertia weight, cognitive, social learning coefficients and maximum velocity of the particle.

Dependencies

  • Numpy
  • matplotlib

Utilities

Once the installation is finished (download or cloning), go the pso folder and follow the below simple guidelines to execute PSO effectively (either write the code in command line or in a python editor).

>>> from pso import PSO

Next, a fitness function (or cost function) is required. I have included four different fitness functions for example purposes namely fitness_1, fitness_2, fitness_3, and fitness_4.

Fitness-1 (Himmelblau's Function)

Minimize: f(x) = (x2 + y - 11)2 + (x + y2 - 7)2

Optimum solution: x = 3 ; y = 2

Fitness-2 (Booth's Function)

Minimize: f(x) = (x + 2y - 7)2 + (2x + y - 5)2

Optimum solution: x = 1 ; y = 3

Fitness-3 (Beale's Function)

Minimize: f(x) = (1.5 - x - xy)2 + (2.25 - x + xy2)2 + (2.625 - x + xy3)2

Optimum solution: x = 3 ; y = 0.5

Fitness-4

Maximize: f(x) = 2xy + 2x - x2 - 2y2

Optimum solution: x = 2 ; y = 1

>>> from fitness import fitness_1, fitness_2, fitness_3, fitness_4

Now, if you want, you can provide an initial position X0 and bound value for all the particles (not mandatory) and optimize (minimize or maximize) the fitness function using PSO:

NOTE: a bool variable min=True (default value) for MINIMIZATION PROBLEM and min=False for MAXIMIZATION PROBLEM

>>> PSO(fitness=fitness_1, X0=[1,1], bound=[(-4,4),(-4,4)]).execute()

You will see the following similar output:

OPTIMUM SOLUTION
  > [3.0000078, 1.9999873]

OPTIMUM FITNESS
  > 0.0

When fitness_4 is used, observe that min=False since it is a Maximization problem.

>>> PSO(fitness=fitness_4, X0=[1,1], bound=[(-4,4),(-4,4)], min=False).execute()

You will see the following similar output:

OPTIMUM SOLUTION
  > [2.0, 1.0]

OPTIMUM FITNESS
  > 2.0

Incase you want to print the fitness value for each iteration, then set verbose=True (here Tmax=50 is the maximum iteration)

>>> PSO(fitness=fitness_2, Tmax=50, verbose=True).execute()

You will see the following similar output:

Iteration:   0  | best global fitness (cost): 18.298822
Iteration:   1  | best global fitness (cost): 1.2203953
Iteration:   2  | best global fitness (cost): 0.8178153
Iteration:   3  | best global fitness (cost): 0.5902262
Iteration:   4  | best global fitness (cost): 0.166928
Iteration:   5  | best global fitness (cost): 0.0926638
Iteration:   6  | best global fitness (cost): 0.0926638
Iteration:   7  | best global fitness (cost): 0.0114517
Iteration:   8  | best global fitness (cost): 0.0114517
Iteration:   9  | best global fitness (cost): 0.0114517
Iteration:   10 | best global fitness (cost): 0.0078867
Iteration:   11 | best global fitness (cost): 0.0078867
Iteration:   12 | best global fitness (cost): 0.0078867
Iteration:   13 | best global fitness (cost): 0.0078867
Iteration:   14 | best global fitness (cost): 0.0069544
Iteration:   15 | best global fitness (cost): 0.0063058
Iteration:   16 | best global fitness (cost): 0.0063058
Iteration:   17 | best global fitness (cost): 0.0011039
Iteration:   18 | best global fitness (cost): 0.0011039
Iteration:   19 | best global fitness (cost): 0.0011039
Iteration:   20 | best global fitness (cost): 0.0011039
Iteration:   21 | best global fitness (cost): 0.0007225
Iteration:   22 | best global fitness (cost): 0.0005875
Iteration:   23 | best global fitness (cost): 0.0001595
Iteration:   24 | best global fitness (cost): 0.0001595
Iteration:   25 | best global fitness (cost): 0.0001595
Iteration:   26 | best global fitness (cost): 0.0001595
Iteration:   27 | best global fitness (cost): 0.0001178
Iteration:   28 | best global fitness (cost): 0.0001178
Iteration:   29 | best global fitness (cost): 0.0001178
Iteration:   30 | best global fitness (cost): 0.0001178
Iteration:   31 | best global fitness (cost): 0.0001178
Iteration:   32 | best global fitness (cost): 0.0001178
Iteration:   33 | best global fitness (cost): 0.0001178
Iteration:   34 | best global fitness (cost): 0.0001178
Iteration:   35 | best global fitness (cost): 0.0001178
Iteration:   36 | best global fitness (cost): 0.0001178
Iteration:   37 | best global fitness (cost): 2.91e-05
Iteration:   38 | best global fitness (cost): 1.12e-05
Iteration:   39 | best global fitness (cost): 1.12e-05
Iteration:   40 | best global fitness (cost): 1.12e-05
Iteration:   41 | best global fitness (cost): 1.12e-05
Iteration:   42 | best global fitness (cost): 1.12e-05
Iteration:   43 | best global fitness (cost): 1.12e-05
Iteration:   44 | best global fitness (cost): 1.12e-05
Iteration:   45 | best global fitness (cost): 1.12e-05
Iteration:   46 | best global fitness (cost): 1.12e-05
Iteration:   47 | best global fitness (cost): 2.4e-06
Iteration:   48 | best global fitness (cost): 2.4e-06
Iteration:   49 | best global fitness (cost): 2.4e-06
Iteration:   50 | best global fitness (cost): 2.4e-06

OPTIMUM SOLUTION
  > [1.0004123, 2.9990281]

OPTIMUM FITNESS
  > 2.4e-06

Now, incase you want to plot the fitness value for each iteration, then set plot=True (here Tmax=50 is the maximum iteration)

>>> PSO(fitness=fitness_2, Tmax=50, plot=True).execute()

You will see the following similar output:

OPTIMUM SOLUTION
  > [1.0028365, 2.9977422]

OPTIMUM FITNESS
  > 1.45e-05

Fitness

Finally, in case you want to use the advanced features as mentioned above (say you want to update the weight inertia parameter w), simply use update_w=True and thats it. Similarly you can use update_c1=True (to update individual cognitive parameter c1), update_c2=True (to update social learning parameter c2), and update_vmax=True (to update maximum limited velocity of the particle vmax)

>>> PSO(fitness=fitness_1, update_w=True, update_c1=True).execute()

References:

[1] Almeida, Bruno & Coppo leite, Victor. (2019). Particle swarm optimization: a powerful technique for solving engineering problems. 10.5772/intechopen.89633.

[2] He, Yan & Ma, Wei & Zhang, Ji. (2016). The parameters selection of pso algorithm influencing on performance of fault diagnosis. matec web of conferences. 63. 02019. 10.1051/matecconf/20166302019.

[3] Clerc, M., and J. Kennedy. The particle swarm — explosion, stability, and convergence in a multidimensional complex space. ieee transactions on evolutionary computation 6, no. 1 (february 2002): 58–73.

[4] Y. H. Shi and R. C. Eberhart, “A modified particle swarm optimizer,” in proceedings of the ieee international conferences on evolutionary computation, pp. 69–73, anchorage, alaska, usa, may 1998.

[5] G. Sermpinis, K. Theofilatos, A. Karathanasopoulos, E. F. Georgopoulos, & C. Dunis, Forecasting foreign exchange rates with adaptive neural networks using radial-basis functions and particle swarm optimization, european journal of operational research.

[6] Particle swarm optimization (pso) visually explained (https://towardsdatascience.com/particle-swarm-optimization-visually-explained-46289eeb2e14)

[7] Rajib Kumar Bhattacharjya, Introduction to Particle Swarm Optimization (http://www.iitg.ac.in/rkbc/ce602/ce602/particle%20swarm%20algorithms.pdf)

Code for ICLR 2020 paper "VL-BERT: Pre-training of Generic Visual-Linguistic Representations".

VL-BERT By Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, Jifeng Dai. This repository is an official implementation of the paper VL-BERT:

Weijie Su 698 Dec 18, 2022
This repository contains the code used for Predicting Patient Outcomes with Graph Representation Learning (https://arxiv.org/abs/2101.03940).

Predicting Patient Outcomes with Graph Representation Learning This repository contains the code used for Predicting Patient Outcomes with Graph Repre

Emma Rocheteau 76 Dec 22, 2022
IEEE-CIS Technical Challenge on Predict+Optimize for Renewable Energy Scheduling

IEEE-CIS Technical Challenge on Predict+Optimize for Renewable Energy Scheduling This is my code, data and approach for the IEEE-CIS Technical Challen

3 Sep 18, 2022
Deep Markov Factor Analysis (NeurIPS2021)

Deep Markov Factor Analysis (DMFA) Codes and experiments for deep Markov factor analysis (DMFA) model accepted for publication at NeurIPS2021: A. Farn

Sarah Ostadabbas 2 Dec 16, 2022
Complex Answer Generation For Conversational Search Systems.

Complex Answer Generation For Conversational Search Systems. Code for Does Structure Matter? Leveraging Data-to-Text Generation for Answering Complex

Hanane Djeddal 0 Dec 06, 2021
Install alphafold on the local machine, get out of docker.

AlphaFold This package provides an implementation of the inference pipeline of AlphaFold v2.0. This is a completely new model that was entered in CASP

Kui Xu 73 Dec 13, 2022
The 2nd place solution of 2021 google landmark retrieval on kaggle.

Leaderboard, taxonomy, and curated list of few-shot object detection papers.

229 Dec 13, 2022
ConformalLayers: A non-linear sequential neural network with associative layers

ConformalLayers: A non-linear sequential neural network with associative layers ConformalLayers is a conformal embedding of sequential layers of Convo

Prograf-UFF 5 Sep 28, 2022
This is the workbook I created while I was studying for the Qiskit Associate Developer exam. I hope this becomes useful to others as it was for me :)

A Workbook for the Qiskit Developer Certification Exam Hello everyone! This is Bartu, a fellow Qiskitter. I have recently taken the Certification exam

Bartu Bisgin 66 Dec 10, 2022
A no-BS, dead-simple training visualizer for tf-keras

A no-BS, dead-simple training visualizer for tf-keras TrainingDashboard Plot inter-epoch and intra-epoch loss and metrics within a jupyter notebook wi

Vibhu Agrawal 3 May 28, 2021
Sandbox for training deep learning networks

Deep learning networks This repo is used to research convolutional networks primarily for computer vision tasks. For this purpose, the repo contains (

Oleg Sémery 2.7k Jan 01, 2023
Learning to Prompt for Continual Learning

Learning to Prompt for Continual Learning (L2P) Official Jax Implementation L2P is a novel continual learning technique which learns to dynamically pr

Google Research 207 Jan 06, 2023
Machine learning framework for both deep learning and traditional algorithms

NeoML is an end-to-end machine learning framework that allows you to build, train, and deploy ML models. This framework is used by ABBYY engineers for

NeoML 704 Dec 27, 2022
Code and Data for NeurIPS2021 Paper "A Dataset for Answering Time-Sensitive Questions"

Time-Sensitive-QA The repo contains the dataset and code for NeurIPS2021 (dataset track) paper Time-Sensitive Question Answering dataset. The dataset

wenhu chen 35 Nov 14, 2022
Official Pytorch Implementation of 'Learning Action Completeness from Points for Weakly-supervised Temporal Action Localization' (ICCV-21 Oral)

Learning-Action-Completeness-from-Points Official Pytorch Implementation of 'Learning Action Completeness from Points for Weakly-supervised Temporal A

Pilhyeon Lee 67 Jan 03, 2023
PyTorch implementation of the cross-modality generative model that synthesizes dance from music.

Dancing to Music PyTorch implementation of the cross-modality generative model that synthesizes dance from music. Paper Hsin-Ying Lee, Xiaodong Yang,

NVIDIA Research Projects 485 Dec 26, 2022
Viperdb - A tiny log-structured key-value database written in pure Python

ViperDB 🐍 ViperDB is a lightweight embedded key-value store written in pure Pyt

17 Oct 17, 2022
Digital Twin Mobility Profiling: A Spatio-Temporal Graph Learning Approach

Digital Twin Mobility Profiling: A Spatio-Temporal Graph Learning Approach This is the implementation of traffic prediction code in DTMP based on PyTo

chenxin 1 Dec 19, 2021
Code for the paper "Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness"

DU-VAE This is the pytorch implementation of the paper "Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness" Acknowledgement

Dazhong Shen 4 Oct 19, 2022
Neural network pruning for finding a sparse computational model for controlling a biological motor task.

MothPruning Scientific Overview Originally inspired by biological nervous systems, deep neural networks (DNNs) are powerful computational tools for mo

Olivia Thomas 0 Dec 14, 2022