visualize_ML is a python package made to visualize some of the steps involved while dealing with a Machine Learning problem

Overview

visualize_ML

visualize_ML is a python package made to visualize some of the steps involved while dealing with a Machine Learning problem. It is build on libraries like matplotlib for visualization and sklean,scipy for statistical computations.

PyPI version

Table of content:

Requirement

  • python 2.x or python 3.x

Install

Install dependencies needed for matplotlib

sudo apt-get build-dep python-matplotlib

Install it using pip

pip install visualize_ML

Let's Code

While dealing with a Machine Learning problem some of the initial steps involved are data exploration,analysis followed by feature selection.Below are the modules for these tasks.

1) Data Exploration

At this stage, we explore variables one by one using Uni-variate Analysis which depends on whether the variable type is categorical or continuous .To deal with this we have the explore module.

>>> explore module

visualize_ML.explore.plot(data_input,categorical_name=[],drop=[],PLOT_COLUMNS_SIZE=4,bin_size=20,
bar_width=0.2,wspace=0.5,hspace=0.8)

Continuous Variables : In case of continous variables it plots the Histogram for every variable and gives descriptive statistics for them.

Categorical Variables : In case on categorical variables with 2 or more classes it plots the Bar chart for every variable and gives descriptive statistics for them.

Parameters Type Description
data_input Dataframe This is the input Dataframe with all data.(Right now the input can be only be a dataframe input.)
categorical_name list (default=[ ]) Names of all categorical variable columns with more than 2 classes, to distinguish them with the continuous variablesEmply list implies that there are no categorical features with more than 2 classes.
drop list default=[ ] Names of columns to be dropped.
PLOT_COLUMNS_SIZE int (default=4) Number of plots to display vertically in the display window.The row size is adjusted accordingly.
bin_size int (default="auto") Number of bins for the histogram displayed in the categorical vs categorical category.
wspace float32 (default = 0.5) Horizontal padding between subplot on the display window.
hspace float32 (default = 0.8) Vertical padding between subplot on the display window.

Code Snippet

/* The data set is taken from famous Titanic data(Kaggle)*/

import pandas as pd
from visualize_ML import explore
df = pd.read_csv("dataset/train.csv")
explore.plot(df,["Survived","Pclass","Sex","SibSp","Ticket","Embarked"],drop=["PassengerId","Name"])

Alt text

see the dataset

Note: While plotting all the rows with NaN values and columns with Character values are removed(except if values are True and False ),only numeric data is plotted.

2) Feature Selection

This is one of the challenging task to deal with for a ML task.Here we have to do Bi-variate Analysis to find out the relationship between two variables. Here, we look for association and disassociation between variables at a pre-defined significance level.

relation module helps in visualizing the analysis done on various combination of variables and see relation between them.

>>> relation module

visualize_ML.relation.plot(data_input,target_name="",categorical_name=[],drop=[],bin_size=10)

Continuous vs Continuous variables: To do the Bi-variate analysis scatter plots are made as their pattern indicates the relationship between variables. To indicates the strength of relationship amongst them we use Correlation between them.

The graph displays the correlation coefficient along with other information.

Correlation = Covariance(X,Y) / SQRT( Var(X)*Var(Y))
  • -1: perfect negative linear correlation
  • +1:perfect positive linear correlation and
  • 0: No correlation

Categorical vs Categorical variables: Stacked Column Charts are made to visualize the relation.Chi square test is used to derive the statistical significance of relationship between the variables. It returns probability for the computed chi-square distribution with the degree of freedom. For more information on Chi Test see this

Probability of 0: It indicates that both categorical variable are dependent

Probability of 1: It shows that both variables are independent.

The graph displays the p_value along with other information. If it is leass than 0.05 it states that the variables are dependent.

Categorical vs Continuous variables: To explore the relation between categorical and continuous variables,box plots re drawn at each level of categorical variables. If levels are small in number, it will not show the statistical significance. ANOVA test is used to derive the statistical significance of relationship between the variables.

The graph displays the p_value along with other information. If it is leass than 0.05 it states that the variables are dependent.

For more information on ANOVA test see this

Parameters Type Description
data_input Dataframe This is the input Dataframe with all data.(Right now the input can be only be a dataframe input.)
target_name String The name of the target column.
categorical_name list (default=[ ]) Names of all categorical variable columns with more than 2 classes, to distinguish them with the continuous variablesEmply list implies that there are no categorical features with more than 2 classes.
drop list default=[ ] Names of columns to be dropped.
PLOT_COLUMNS_SIZE int (default=4) Number of plots to display vertically in the display window.The row size is adjusted accordingly.
bin_size int (default="auto") Number of bins for the histogram displayed in the categorical vs categorical category.
wspace float32 (default = 0.5) Horizontal padding between subplot on the display window.
hspace float32 (default = 0.8) Vertical padding between subplot on the display window.

Code Snippet

/* The data set is taken from famous Titanic data(Kaggle)*/
import pandas as pd
from visualize_ML import relation
df = pd.read_csv("dataset/train.csv")
relation.plot(df,"Survived",["Survived","Pclass","Sex","SibSp","Ticket","Embarked"],drop=["PassengerId","Name"],bin_size=10)

Alt text

see the dataset

Note: While plotting all the rows with NaN values and columns with Non numeric values are removed only numeric data is plotted.Only categorical taget variable with string values are allowed.

Contribute

If you want to contribute and add new feature feel free to send Pull request here

This project is still under development so to report any bugs or request new features, head over to the Issues page

Tasks To Do

  • Make input compatible with other formats like Numpy.

  • Visualize best fit lines and decision boundaries for various models to make Parameter Tuning task easy.

    and many others!

Licence

Licensed under The MIT License (MIT).

Copyright

ayush1997(c) 2016

You might also like...
Import, visualize, and analyze SpiderFoot OSINT data in Neo4j, a graph database
Import, visualize, and analyze SpiderFoot OSINT data in Neo4j, a graph database

SpiderFoot Neo4j Tools Import, visualize, and analyze SpiderFoot OSINT data in Neo4j, a graph database Step 1: Installation NOTE: This installs the sf

Extract and visualize information from Gurobi log files
Extract and visualize information from Gurobi log files

GRBlogtools Extract information from Gurobi log files and generate pandas DataFrames or Excel worksheets for further processing. Also includes a wrapp

Extract data from ThousandEyes REST API and visualize it on your customized Grafana Dashboard.
Extract data from ThousandEyes REST API and visualize it on your customized Grafana Dashboard.

ThousandEyes Grafana Dashboard Extract data from the ThousandEyes REST API and visualize it on your customized Grafana Dashboard. Deploy Grafana, Infl

This is  a web application to visualize various famous technical indicators and stocks tickers from user
This is a web application to visualize various famous technical indicators and stocks tickers from user

Visualizing Technical Indicators Using Python and Plotly. Currently facing issues hosting the application on heroku. As soon as I am able to I'll like

Visualize the training curve from the *.csv file (tensorboard format).
Visualize the training curve from the *.csv file (tensorboard format).

Training-Curve-Vis Visualize the training curve from the *.csv file (tensorboard format). Feature Custom labels Curve smoothing Support for multiple c

Visualize your pandas data with one-line code
Visualize your pandas data with one-line code

PandasEcharts 简介 基于pandas和pyecharts的可视化工具 安装 pip 安装 $ pip install pandasecharts 源码安装 $ git clone https://github.com/gamersover/pandasecharts $ cd pand

 Flame Graphs visualize profiled code
Flame Graphs visualize profiled code

Flame Graphs visualize profiled code

Visualize data of Vietnam's regions with interactive maps.
Visualize data of Vietnam's regions with interactive maps.

Plotting Vietnam Development Map This is my personal project that I use plotly to analyse and visualize data of Vietnam's regions with interactive map

 Epagneul is a tool to visualize and investigate windows event logs
Epagneul is a tool to visualize and investigate windows event logs

epagneul Epagneul is a tool to visualize and investigate windows event logs. Dep

Comments
  • Can't get graphs to space right

    Can't get graphs to space right

    Not sure what is going on tried looking at the code.. I'm using Jupyter notebook if that is messing stuff up? data: state region age gender race marital_status ptype status-grp 0 IA 3 73 M W M Patient NaN 1 IL 2 57 M W S Patient NaN 2 WI 2 32 F W U Patient NaN 3 WI 2 54 F W U Patient NaN 4 IL 2 56 F W M Patient NaN 5 WI 2 31 F W S Patient

    input line: explore.plot(df2,['state','region','age','gender','race','marital_status','ptype','status-grp'],PLOT_COLUMNS_SIZE=2,bin_size=20, bar_width=0.2,wspace=.75,hspace=.75) result: vizml

    opened by dartdog 6
  • Just installed but it required and executed a downgrade of MPL

    Just installed but it required and executed a downgrade of MPL

    The PIP install downgraded MPL from 1.5.1 to 1.4.2 and also required the installation of "sudo apt-get install blt-dev" for freetype to build,, I had not previously run into that before? Any advice on how to preserve Matplotlib at 1.5.1 and of course MPL 2.0 is about to drop soon as well? The package looks quite useful with some nice ideas!

    opened by dartdog 2
Releases(0.2.2)
Owner
Ayush Singh
Machine Learning | Computer Vision | Data Science | Python
Ayush Singh
Editor and Presenter for Manim Generated Content.

Editor and Presenter for Manim Generated Content. Take a look at the Working Example. More information can be found on the documentation. These Browse

Manim Community 149 Dec 29, 2022
Generate visualizations of GitHub user and repository statistics using GitHub Actions.

GitHub Stats Visualization Generate visualizations of GitHub user and repository statistics using GitHub Actions. This project is currently a work-in-

Aditya Thakekar 1 Jan 11, 2022
Visualize tensors in a plain Python REPL using Sparklines

Visualize tensors in a plain Python REPL using Sparklines

Shawn Presser 43 Sep 03, 2022
Functions for easily making publication-quality figures with matplotlib.

Data-viz utils 📈 Functions for data visualization in matplotlib 📚 API Can be installed using pip install dvu and then imported with import dvu. You

Chandan Singh 16 Sep 15, 2022
Simple CLI python app to show a stocks graph performance. Made with Matplotlib and Tiingo.

stock-graph-python Simple CLI python app to show a stocks graph performance. Made with Matplotlib and Tiingo. Tiingo API Key You will need to add your

Toby 3 May 14, 2022
Visualization of numerical optimization algorithms

Visualization of numerical optimization algorithms

Zhengxia Zou 46 Dec 01, 2022
This GitHub Repository contains Data Analysis projects that I have completed so far! While most of th project are focused on Data Analysis, some of them are also put here to show off other skills that I have learned.

Welcome to my Data Analysis projects page! This GitHub Repository contains Data Analysis projects that I have completed so far! While most of th proje

Kyle Dini 1 Jan 31, 2022
This plugin plots the time you spent on a tag as a histogram.

This plugin plots the time you spent on a tag as a histogram.

Tom Dörr 7 Sep 09, 2022
A minimalistic wrapper around PyOpenGL to save development time

glpy glpy is pyOpenGl wrapper which lets you work with pyOpenGl easily.It is not meant to be a replacement for pyOpenGl but runs on top of pyOpenGl to

Abhinav 9 Apr 02, 2022
ipyvizzu - Jupyter notebook integration of Vizzu

ipyvizzu - Jupyter notebook integration of Vizzu. Tutorial · Examples · Repository About The Project ipyvizzu is the Jupyter Notebook integration of V

Vizzu 729 Jan 08, 2023
NumPy and Pandas interface to Big Data

Blaze translates a subset of modified NumPy and Pandas-like syntax to databases and other computing systems. Blaze allows Python users a familiar inte

Blaze 3.1k Jan 01, 2023
finds grocery stores and stuff next to route (gpx)

Route-Report Route report is a command-line utility that can be used to locate points-of-interest near your planned route (gpx). The results are based

Clemens Mosig 5 Oct 10, 2022
Decision Border Visualizer for Classification Algorithms

dbv Decision Border Visualizer for Classification Algorithms Project description A python package for Machine Learning Engineers who want to visualize

Sven Eschlbeck 1 Nov 01, 2021
Visualize and compare datasets, target values and associations, with one line of code.

In-depth EDA (target analysis, comparison, feature analysis, correlation) in two lines of code! Sweetviz is an open-source Python library that generat

Francois Bertrand 2.3k Jan 05, 2023
Small project to recursively calculate and plot each successive order of the Hilbert Curve

hilbert-curve Small project to recursively calculate and plot each successive order of the Hilbert Curve. After watching 3Blue1Brown's video on Hilber

Stefan Mejlgaard 2 Nov 15, 2021
Here I plotted data for the average test scores across schools and class sizes across school districts.

HW_02 Here I plotted data for the average test scores across schools and class sizes across school districts. Average Test Score by Race This graph re

7 Oct 27, 2021
Generate a 3D Skyline in STL format and a OpenSCAD file from Gitlab contributions

Your Gitlab's contributions in a 3D Skyline gitlab-skyline is a Python command to generate a skyline figure from Gitlab contributions as Github did at

Félix Gómez 70 Dec 22, 2022
PyFlow is a general purpose visual scripting framework for python

PyFlow is a general purpose visual scripting framework for python. State Base structure of program implemented, such things as packages disco

1.8k Jan 07, 2023
Sci palettes for matplotlib/seaborn

sci palettes for matplotlib/seaborn Installation python3 -m pip install sci-palettes Usage import seaborn as sns import matplotlib.pyplot as plt impor

Qingdong Su 2 Jun 07, 2022
Dimensionality reduction in very large datasets using Siamese Networks

ivis Implementation of the ivis algorithm as described in the paper Structure-preserving visualisation of high dimensional single-cell datasets. Ivis

beringresearch 284 Jan 01, 2023