visualize_ML is a python package made to visualize some of the steps involved while dealing with a Machine Learning problem

Overview

visualize_ML

visualize_ML is a python package made to visualize some of the steps involved while dealing with a Machine Learning problem. It is build on libraries like matplotlib for visualization and sklean,scipy for statistical computations.

PyPI version

Table of content:

Requirement

  • python 2.x or python 3.x

Install

Install dependencies needed for matplotlib

sudo apt-get build-dep python-matplotlib

Install it using pip

pip install visualize_ML

Let's Code

While dealing with a Machine Learning problem some of the initial steps involved are data exploration,analysis followed by feature selection.Below are the modules for these tasks.

1) Data Exploration

At this stage, we explore variables one by one using Uni-variate Analysis which depends on whether the variable type is categorical or continuous .To deal with this we have the explore module.

>>> explore module

visualize_ML.explore.plot(data_input,categorical_name=[],drop=[],PLOT_COLUMNS_SIZE=4,bin_size=20,
bar_width=0.2,wspace=0.5,hspace=0.8)

Continuous Variables : In case of continous variables it plots the Histogram for every variable and gives descriptive statistics for them.

Categorical Variables : In case on categorical variables with 2 or more classes it plots the Bar chart for every variable and gives descriptive statistics for them.

Parameters Type Description
data_input Dataframe This is the input Dataframe with all data.(Right now the input can be only be a dataframe input.)
categorical_name list (default=[ ]) Names of all categorical variable columns with more than 2 classes, to distinguish them with the continuous variablesEmply list implies that there are no categorical features with more than 2 classes.
drop list default=[ ] Names of columns to be dropped.
PLOT_COLUMNS_SIZE int (default=4) Number of plots to display vertically in the display window.The row size is adjusted accordingly.
bin_size int (default="auto") Number of bins for the histogram displayed in the categorical vs categorical category.
wspace float32 (default = 0.5) Horizontal padding between subplot on the display window.
hspace float32 (default = 0.8) Vertical padding between subplot on the display window.

Code Snippet

/* The data set is taken from famous Titanic data(Kaggle)*/

import pandas as pd
from visualize_ML import explore
df = pd.read_csv("dataset/train.csv")
explore.plot(df,["Survived","Pclass","Sex","SibSp","Ticket","Embarked"],drop=["PassengerId","Name"])

Alt text

see the dataset

Note: While plotting all the rows with NaN values and columns with Character values are removed(except if values are True and False ),only numeric data is plotted.

2) Feature Selection

This is one of the challenging task to deal with for a ML task.Here we have to do Bi-variate Analysis to find out the relationship between two variables. Here, we look for association and disassociation between variables at a pre-defined significance level.

relation module helps in visualizing the analysis done on various combination of variables and see relation between them.

>>> relation module

visualize_ML.relation.plot(data_input,target_name="",categorical_name=[],drop=[],bin_size=10)

Continuous vs Continuous variables: To do the Bi-variate analysis scatter plots are made as their pattern indicates the relationship between variables. To indicates the strength of relationship amongst them we use Correlation between them.

The graph displays the correlation coefficient along with other information.

Correlation = Covariance(X,Y) / SQRT( Var(X)*Var(Y))
  • -1: perfect negative linear correlation
  • +1:perfect positive linear correlation and
  • 0: No correlation

Categorical vs Categorical variables: Stacked Column Charts are made to visualize the relation.Chi square test is used to derive the statistical significance of relationship between the variables. It returns probability for the computed chi-square distribution with the degree of freedom. For more information on Chi Test see this

Probability of 0: It indicates that both categorical variable are dependent

Probability of 1: It shows that both variables are independent.

The graph displays the p_value along with other information. If it is leass than 0.05 it states that the variables are dependent.

Categorical vs Continuous variables: To explore the relation between categorical and continuous variables,box plots re drawn at each level of categorical variables. If levels are small in number, it will not show the statistical significance. ANOVA test is used to derive the statistical significance of relationship between the variables.

The graph displays the p_value along with other information. If it is leass than 0.05 it states that the variables are dependent.

For more information on ANOVA test see this

Parameters Type Description
data_input Dataframe This is the input Dataframe with all data.(Right now the input can be only be a dataframe input.)
target_name String The name of the target column.
categorical_name list (default=[ ]) Names of all categorical variable columns with more than 2 classes, to distinguish them with the continuous variablesEmply list implies that there are no categorical features with more than 2 classes.
drop list default=[ ] Names of columns to be dropped.
PLOT_COLUMNS_SIZE int (default=4) Number of plots to display vertically in the display window.The row size is adjusted accordingly.
bin_size int (default="auto") Number of bins for the histogram displayed in the categorical vs categorical category.
wspace float32 (default = 0.5) Horizontal padding between subplot on the display window.
hspace float32 (default = 0.8) Vertical padding between subplot on the display window.

Code Snippet

/* The data set is taken from famous Titanic data(Kaggle)*/
import pandas as pd
from visualize_ML import relation
df = pd.read_csv("dataset/train.csv")
relation.plot(df,"Survived",["Survived","Pclass","Sex","SibSp","Ticket","Embarked"],drop=["PassengerId","Name"],bin_size=10)

Alt text

see the dataset

Note: While plotting all the rows with NaN values and columns with Non numeric values are removed only numeric data is plotted.Only categorical taget variable with string values are allowed.

Contribute

If you want to contribute and add new feature feel free to send Pull request here

This project is still under development so to report any bugs or request new features, head over to the Issues page

Tasks To Do

  • Make input compatible with other formats like Numpy.

  • Visualize best fit lines and decision boundaries for various models to make Parameter Tuning task easy.

    and many others!

Licence

Licensed under The MIT License (MIT).

Copyright

ayush1997(c) 2016

You might also like...
Import, visualize, and analyze SpiderFoot OSINT data in Neo4j, a graph database
Import, visualize, and analyze SpiderFoot OSINT data in Neo4j, a graph database

SpiderFoot Neo4j Tools Import, visualize, and analyze SpiderFoot OSINT data in Neo4j, a graph database Step 1: Installation NOTE: This installs the sf

Extract and visualize information from Gurobi log files
Extract and visualize information from Gurobi log files

GRBlogtools Extract information from Gurobi log files and generate pandas DataFrames or Excel worksheets for further processing. Also includes a wrapp

Extract data from ThousandEyes REST API and visualize it on your customized Grafana Dashboard.
Extract data from ThousandEyes REST API and visualize it on your customized Grafana Dashboard.

ThousandEyes Grafana Dashboard Extract data from the ThousandEyes REST API and visualize it on your customized Grafana Dashboard. Deploy Grafana, Infl

This is  a web application to visualize various famous technical indicators and stocks tickers from user
This is a web application to visualize various famous technical indicators and stocks tickers from user

Visualizing Technical Indicators Using Python and Plotly. Currently facing issues hosting the application on heroku. As soon as I am able to I'll like

Visualize the training curve from the *.csv file (tensorboard format).
Visualize the training curve from the *.csv file (tensorboard format).

Training-Curve-Vis Visualize the training curve from the *.csv file (tensorboard format). Feature Custom labels Curve smoothing Support for multiple c

Visualize your pandas data with one-line code
Visualize your pandas data with one-line code

PandasEcharts 简介 基于pandas和pyecharts的可视化工具 安装 pip 安装 $ pip install pandasecharts 源码安装 $ git clone https://github.com/gamersover/pandasecharts $ cd pand

 Flame Graphs visualize profiled code
Flame Graphs visualize profiled code

Flame Graphs visualize profiled code

Visualize data of Vietnam's regions with interactive maps.
Visualize data of Vietnam's regions with interactive maps.

Plotting Vietnam Development Map This is my personal project that I use plotly to analyse and visualize data of Vietnam's regions with interactive map

 Epagneul is a tool to visualize and investigate windows event logs
Epagneul is a tool to visualize and investigate windows event logs

epagneul Epagneul is a tool to visualize and investigate windows event logs. Dep

Comments
  • Can't get graphs to space right

    Can't get graphs to space right

    Not sure what is going on tried looking at the code.. I'm using Jupyter notebook if that is messing stuff up? data: state region age gender race marital_status ptype status-grp 0 IA 3 73 M W M Patient NaN 1 IL 2 57 M W S Patient NaN 2 WI 2 32 F W U Patient NaN 3 WI 2 54 F W U Patient NaN 4 IL 2 56 F W M Patient NaN 5 WI 2 31 F W S Patient

    input line: explore.plot(df2,['state','region','age','gender','race','marital_status','ptype','status-grp'],PLOT_COLUMNS_SIZE=2,bin_size=20, bar_width=0.2,wspace=.75,hspace=.75) result: vizml

    opened by dartdog 6
  • Just installed but it required and executed a downgrade of MPL

    Just installed but it required and executed a downgrade of MPL

    The PIP install downgraded MPL from 1.5.1 to 1.4.2 and also required the installation of "sudo apt-get install blt-dev" for freetype to build,, I had not previously run into that before? Any advice on how to preserve Matplotlib at 1.5.1 and of course MPL 2.0 is about to drop soon as well? The package looks quite useful with some nice ideas!

    opened by dartdog 2
Releases(0.2.2)
Owner
Ayush Singh
Machine Learning | Computer Vision | Data Science | Python
Ayush Singh
Bar Chart of the number of Senators from each party who are up for election in the next three General Elections

Congress-Analysis Bar Chart of the number of Senators from each party who are up for election in the next three General Elections This bar chart shows

11 Oct 26, 2021
Arras.io Highest Scores Over Time Bar Chart Race

Arras.io Highest Scores Over Time Bar Chart Race This repo contains a python script (make_racing_bar_chart.py) that can generate a csv file which can

Road 2 Jan 16, 2022
A Python toolbox for gaining geometric insights into high-dimensional data

"To deal with hyper-planes in a 14 dimensional space, visualize a 3D space and say 'fourteen' very loudly. Everyone does it." - Geoff Hinton Overview

Contextual Dynamics Laboratory 1.8k Dec 29, 2022
A curated list of awesome Dash (plotly) resources

Awesome Dash A curated list of awesome Dash (plotly) resources Dash is a productive Python framework for building web applications. Written on top of

Luke Singham 1.7k Dec 26, 2022
This project is created to visualize the system statistics such as memory usage, CPU usage, memory accessible by process and much more using Kibana Dashboard with Elasticsearch.

System Stats Visualizer This project is created to visualize the system statistics such as memory usage, CPU usage, memory accessible by process and m

Vishal Teotia 5 Feb 06, 2022
Create artistic visualisations with your exercise data (Python version)

strava_py Create artistic visualisations with your exercise data (Python version). This is a port of the R strava package to Python. Examples Facets A

Marcus Volz 53 Dec 28, 2022
Data Visualizations for the #30DayChartChallenge

The #30DayChartChallenge This repository contains all the charts made for the #30DayChartChallenge during the month of April. This project aims to exp

Isaac Arroyo 7 Sep 20, 2022
A program that analyzes data from inertia measurement units installed in aircraft and generates g-exceedance curves.

A program that analyzes data from inertia measurement units installed in aircraft and generates g-exceedance curves.

Pooya 1 Dec 02, 2021
Blender addon that creates a temporary window of any type from the 3D View.

CreateTempWindow2.8 Blender addon that creates a temporary window of any type from the 3D View. Features Can the following window types: 3D View Graph

3 Nov 27, 2022
🧇 Make Waffle Charts in Python.

PyWaffle PyWaffle is an open source, MIT-licensed Python package for plotting waffle charts. It provides a Figure constructor class Waffle, which coul

Guangyang Li 528 Jan 02, 2023
Active Transport Analytics Model (ATAM) is a new strategic transport modelling and data visualization framework for Active Transport as well as emerging micro-mobility modes

{ATAM} Active Transport Analytics Model Active Transport Analytics Model (“ATAM”) is a new strategic transport modelling and data visualization framew

Peter Stephan 0 Jan 12, 2022
A comprehensive tutorial for plotting focal mechanism

Focal_Mechanisms_Demo A comprehensive tutorial for plotting focal mechanism "beach-balls" using the PyGMT package for Python. (Resulting map of this d

3 Dec 13, 2022
A concise grammar of interactive graphics, built on Vega.

Vega-Lite Vega-Lite provides a higher-level grammar for visual analysis that generates complete Vega specifications. You can find more details, docume

Vega 4k Jan 08, 2023
Import, visualize, and analyze SpiderFoot OSINT data in Neo4j, a graph database

SpiderFoot Neo4j Tools Import, visualize, and analyze SpiderFoot OSINT data in Neo4j, a graph database Step 1: Installation NOTE: This installs the sf

Black Lantern Security 42 Dec 26, 2022
Compute and visualise incidence (reworking of the original incidence package)

incidence2 incidence2 is an R package that implements functions and classes to compute, handle and visualise incidence from linelist data. It refocuss

15 Nov 22, 2022
Some problems of SSLC ( High School ) before outputs and after outputs

Some problems of SSLC ( High School ) before outputs and after outputs 1] A Python program and its output (output1) while running the program is given

Fayas Noushad 3 Dec 01, 2021
Handout for the tutorial "Creating publication-quality figures with matplotlib"

Handout for the tutorial "Creating publication-quality figures with matplotlib"

JB Mouret 1.9k Jan 02, 2023
BGraph is a tool designed to generate dependencies graphs from Android.bp soong files.

BGraph BGraph is a tool designed to generate dependencies graphs from Android.bp soong files. Overview BGraph (for Build-Graphs) is a project aimed at

Quarkslab 10 Dec 19, 2022
CLAHE Contrast Limited Adaptive Histogram Equalization

A simple code to process images using contrast limited adaptive histogram equalization. Image processing is becoming a major part of data processig.

Happy N. Monday 4 May 18, 2022
Example Code Notebooks for Data Visualization in Python

This repository contains sample code scripts for creating awesome data visualizations from scratch using different python libraries (such as matplotli

Javed Ali 27 Jan 04, 2023