PanGraphViewer -- show panenome graph in an easy way

Overview

PanGraphViewer -- show panenome graph in an easy way

PyPI - Python Version GitHub release (latest by date) GitHub GitHub all releases

Table of Contents


Versions and dependences

Here we provide two application versions:

● Desktop-based application
● Web browser-based application

Overall, Python3 is needed to run this software and we highly recommend using miniconda3 to install all python3 libraries.

● On Windows system, you can download miniconda3 at 
  https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe

● On macOS system, you can download miniconda3 at 
  https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh  

● On Linux system,  you can download miniconda3 at 
  https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

After the installation of miniconda3, you can follow the steps below to ensure panGraphViewer can be executed.


Desktop-based panGraphViewer

Library installation for the desktop-based version

Steps on different systems

  • If you use Windows system, you may need to find or search Anaconda Powershell Prompt (miniconda3) first and then open it.

  • If you use macOS or Linux system, you may open Terminal first and then type the command line below

    $ export PATH=/full/path/to/miniconda3/bin:$PATH # modify the path based on your ENV
    

After the steps above, you can install the python3 libraries by typing:

conda config --add channels conda-forge
conda config --add channels bioconda
conda install pyqt pyqtwebengine configparser pandas bokeh==2.2.3 dna_features_viewer natsort attrdict networkx 

If you use pip, you can install the python3 libraries like:

pip install PyQt5 PyQtWebEngine configparser pandas bokeh==2.2.3 dna_features_viewer natsort attrdict networkx

or you can use pip to install like (need to go to the panGraphViewerApp directory first)

pip install -r requirement.txt ## On Linux or macOS system
pip install -r requirement_windows.txt ## On Windows system

Note:

  1. On Linux or macOS system, pysam is needed. You may install this package using

    $ conda install pysam 
    
  2. On Windows platforms, as pysam is not available, we use a windows-version samtools package instead. Additional libraries below are needed and can be installed using

    > conda install m2-base pyfaidx
    

Start the desktop-based version

  1. On Linux or macOS system, you may use the command line below in Terminal to open the software.

    $ cd /full/path/to/panGraphViewer/panGraphViewerApp # modify the path based on your ENV
    $ python panGraphViewerApp.py
    
  2. On Windows system, you may search and open Anaconda Prompt (miniconda3) first and then move to the panGraphViewer directory. For example, if you have put panGraphViewer on your Desktop and the opened Anaconda Prompt (miniconda3) is in your C drive, you may use the command line below to start the program:

    > cd C:\Users\%USERNAME%\Desktop\panGraphViewer\panGraphViewerApp
    > python panGraphViewerApp.py
    

    If you have put panGraphViewer on other drive, you may need to move to the target drive first. For instance, the target drive is D, you can move to the drive by typing D: in Anaconda Prompt (miniconda3) and then move to the panGraphViewer directory to execute panGraphViewerApp.py.

    Please NOTE that on Windows system, you need to use backslash \ rather than the common slash / to move to the target directory.

  3. The logging information will show in Anaconda Prompt (miniconda3) or Terminal depending on the system you use (Will be good for you to monitor the status of the application).


Web-based panGraphViewer

To meet different requirments, we have also created a web-based panGraphViewer. Basically, most functions provided in the Desktop-based version have been implemented in the Web browser-based version. Users can install this version locally or directly deploy this online. The web browser-based verison offers administrative functions to help create accounts for different users.

Library installation for the web-based version

Depending on the systems used, users can use pip directly to install the needed python3 libraries after moving to the panGraphViewerWeb directory.

pip install -r requirement.txt ## On Linux or macOS system
pip install -r requirement_windows.txt ## On Windows system

As mentioned in the desktop-based version, pysam cannot be installed on Windows systems, users need to install alternatives on Windows by using

> conda install m2-base pyfaidx

For Linux or macOS users, pysam can be installed directly using

$ conda install pysam

Start the web-based version

After the installation above, users can move to the panGraphViewerWeb directory by referring to the steps mentioned in the desktop version through Terminal or Anaconda Prompt (miniconda3).

Note that the folder needed here is panGraphViewerWeb.

Once moving to the panGraphViewerWeb directory, users can start the application by typing

python manage.py runserver   ## on local machine the IPaddress can be: localhost:8004

or users can use the CMD below to start the Web browser-based version

$ bash run.sh   ## On linux or macOS system.
> run.bat ## On Windows system

Note: the IP 0.0.0.0 in run.sh can be modified accordingly

Once the words Starting development server at http://localhost:8004/ or similar infomation is shown, user can open a browser to open the web-based panGraphViewer.

The admin page is http://localhost:8004/admin and the inital admin info is:

Account: admin
password: abcd1234

Note: please use the go back button provided by the web browser to move back rather than directly clicking the corresponding functions in the web page to perform analyses.


The Files needed in the application

The rGFA file

  1. If you have multiple high-quality genome assemblies from different individuals, you may use minigraph (Linux preferred) to generate a reference GFA (rGFA) file.

    Before the running, the header of the fasta file needs modifying. For example, if you have a fasta file from Sample1 with a header like:

    >chr1
    AAAAAGCCGCGCGCGCTTGCGC
    

    You may modify the header to:

    >Sample1||chr1
    AAAAAGCCGCGCGCGCTTGCGC
    

    On Linux, the command lines that can be used to achieve this are:

    />${sample}||/g" $fasta > ${name}.headerModified.fasta ">
    $ sample="" ## the name of the sample. For instance: Sample1
    $ fasta="" ## full path to the fasta file
    $ name=`echo $fasta | rev | cut -d"." -f2-| rev`
    $ sed -e "s/>/>${sample}||/g" $fasta > ${name}.headerModified.fasta
    

    We also provide a python script renameFastaHeader.py to help this conversion. The script can be found in the scripts folder under panGraphViewer --> panGraphViewerApp. Or users can use the UI to convert by clicking Tools --> Format Conversion --> Modify FASTA Header.

    usage: renameFastaHeader.py [-h] [--version] [-f FASTA] [-n NAME] [-o OUTPUT]
    
    rename the header of a given fasta file
    
    optional arguments:
      -h, --help  show this help message and exit
      --version   show program's version number and exit
      -f FASTA    a fasta format file
      -n NAME     name of the sample
      -d DELIM    delimiter. Default: '||'
      -o OUTPUT   the output directory
    

    Please NOTE that:

    I). If you do not modify the header of your fasta file and directly use minigraph to generate the rGFA file, panGraphViewer can still read the file, while many features, such as where the node comes from would not show in detail. A warning message will display in both UI and the opened Terminal or powershell.

    II). For the sample name, please DO NOT include ||.

  2. If you don't have an rGFA file, but a GFA file, you may try to follow the standard here to convert your GFA file into an rGFA file. After generating an rGFA file, you can use this software to visualise the graph of interest.


The VCF file

We also accept a VCF file to show the graph. Basically, a reference FASTA file is optional if the VCF is a standard one. The program will automatically check the input VCF file and evaluate if the VCF file meets the requirement. If not, a message will show.

VCF filtration is highly recommended before plot the graph.

We also provide a method to help convert a VCF file to an rGFA file. Users can perform the conversion directly through the interface provided in the application or directly use vcf2rGFA.py under the panGraphViewer --> panGraphViewerApp --> scripts folder.

Note: If there are many variations in the VCF file, we recommend using vcf2rGFA.py directly to convert by chromosomes rather than converting entirely. This will save a lot of computing resource when plot graphs.

The usage of vcf2rGFA.py is shown below. Both Windows and Linux/macOS users can directly use this script to convert a VCF file to an rGFA file.

usage: vcf2rGFA.py [-h] [--version] [-f FASTA] [-b BACKBONE] [-v VCF] [-o OUTPUT] [-c [CHR [CHR ...]]] [-n NTHREAD]
    
Convert a vcf file to an rGFA file
    
optional arguments:
    -h, --help          show this help message and exit
    --version           show program's version number and exit
    -f FASTA            a fasta format file that from the backbone sample
    -b BACKBONE         the name of the backbone sample
    -v VCF              the vcf file
    -o OUTPUT           the output directory
    -c [CHR [CHR ...]]  the name of the chromosome(s) [default: all chroms]
    -n NTHREAD          number of threads [default: 4]

The BED file

Basically, the BED file should contain the annoation information from the backbone sample. There should be at least 6 columns in the BED file.

Column Information
1 Chromosome ID
2 Gene start position
3 Gene end position
4 Gene ID
5 Score (or others; the program does not use the info in this column)
6 Orientation

Users can load the BED file to check the overlaps between variations and genes. By default, genes overlapping with more than 2 nodes will be shown in the dropdown menu. A gene list will be saved in the output directory after parsing the BED file.


Q&A:

The minimum computing resource needed

The minimum computing resource needed for running the application

Memory:  1Gb
Threads: 2

Which application should I use

For the desktop-based application, it is optimized on Windows 10 and macOS Big Sur. Ubuntu 18.04.5 is also tested. For Linux operating system version below Ubuntu 18.04.5 or equivalent, such as Ubuntu 16.04, PyQtWebEngine may not work properly. For other versions of operating systems, the desktop-based application may still work, however, the layout of the application may differ.

For the web browser-based version, we suggest running in Linux or macOS environment. If users want to run on Windows systems, Windows 10 or above is recommended. Users can also use docker to run the web browser-based version. However, WSL is needed to run the docker version on Windows 10 or above.


The backbone sample

The backbone sample is the one used as the main sequence provider to produce the pangenome graph or the reference sample to produce the VCF file. In the pangenome graph, most of the nodes are from the backbone sample (shared by all) with some nodes (variations) from other samples.


The colors showed in the graph

Each sample uses one particular colour and the most frequent colour should be the one used for the backbone sample. The colours are randomly selected by the program from a desgined colour palletes.


The type of graphs

We provides two kinds of graph plots in the program to achieve a good performance and visualisation. By default, if the number of checked nodes <= 200, vis.js based graph will show. Otherwise, a cytoscape.js based graph will show. Users can change the settings in the desktop-based application.


The shapes showed in the graph

If you use a VCF file to show graphs, we use different nodes shapes to represent different kinds of variants. For instance, in the default settings for the vis.js based graph, dot represent SNP, triangle represents deletion, triangleDown reprsents insertion, database represents duplication, text shows inversion and star represent translocation. Users can change the corresponding settings to select preferred node shapes to represent different variations on the desktop-based application.


How to use the program

For more detailed steps to run panGrapViewer, please refer to the Manual


Different variations

If users use a VCF file to generate a graph genome, when moving the mouse to the graph node, the program will automatically show the variation types, such as SNP(single nucleotide polymorphism), INS (insertion), INV (inversion) and DUP (duplication). The corresponding nodes from the backbone sample will also be linked and shown.


Enjoy using panGraphViewer!

You might also like...
HiPlot makes understanding high dimensional data easy
HiPlot makes understanding high dimensional data easy

HiPlot - High dimensional Interactive Plotting HiPlot is a lightweight interactive visualization tool to help AI researchers discover correlations and

Python library that makes it easy for data scientists to create charts.
Python library that makes it easy for data scientists to create charts.

Chartify Chartify is a Python library that makes it easy for data scientists to create charts. Why use Chartify? Consistent input data format: Spend l

HiPlot makes understanding high dimensional data easy
HiPlot makes understanding high dimensional data easy

HiPlot - High dimensional Interactive Plotting HiPlot is a lightweight interactive visualization tool to help AI researchers discover correlations and

Python library that makes it easy for data scientists to create charts.
Python library that makes it easy for data scientists to create charts.

Chartify Chartify is a Python library that makes it easy for data scientists to create charts. Why use Chartify? Consistent input data format: Spend l

This is a Cross-Platform Plot Manager for Chia Plotting that is simple, easy-to-use, and reliable.
This is a Cross-Platform Plot Manager for Chia Plotting that is simple, easy-to-use, and reliable.

Swar's Chia Plot Manager A plot manager for Chia plotting: https://www.chia.net/ Development Version: v0.0.1 This is a cross-platform Chia Plot Manage

Yata is a fast, simple and easy Data Visulaization tool, running on python dash
Yata is a fast, simple and easy Data Visulaization tool, running on python dash

Yata is a fast, simple and easy Data Visulaization tool, running on python dash. The main goal of Yata is to provide a easy way for persons with little programming knowledge to visualize their data easily.

An easy to use burndown chart generator for GitHub Project Boards.

Burndown Chart for GitHub Projects An easy to use burndown chart generator for GitHub Project Boards. Table of Contents Features Installation Assumpti

Regress.me is an easy to use data visualization tool powered by Dash/Plotly.

Regress.me Regress.me is an easy to use data visualization tool powered by Dash/Plotly. Regress.me.-.Google.Chrome.2022-05-10.15-58-59.mp4 Get Started

Generate a roam research like Network Graph view from your Notion pages.
Generate a roam research like Network Graph view from your Notion pages.

Notion Graph View Export Notion pages to a Roam Research like graph view.

Comments
  • Plot graph's backbone as straight line

    Plot graph's backbone as straight line

    When there were lots of node, they will coiled together like a ball of thread. Can you add an option before plot, which will plot graph's backbone as straight line and other nodes treated as bubble?

    opened by starskyzheng 4
Releases(v1.0.2)
Matplotlib tutorial for beginner

matplotlib is probably the single most used Python package for 2D-graphics. It provides both a very quick way to visualize data from Python and publication-quality figures in many formats. We are goi

Nicolas P. Rougier 2.6k Dec 28, 2022
Tidy data structures, summaries, and visualisations for missing data

naniar naniar provides principled, tidy ways to summarise, visualise, and manipulate missing data with minimal deviations from the workflows in ggplot

Nicholas Tierney 611 Dec 22, 2022
Tandem Mass Spectrum Prediction with Graph Transformers

MassFormer This is the original implementation of MassFormer, a graph transformer for small molecule MS/MS prediction. Check out the preprint on arxiv

Röst Lab 13 Oct 27, 2022
Plot-configurations for scientific publications, purely based on matplotlib

TUEplots Plot-configurations for scientific publications, purely based on matplotlib. Usage Please have a look at the examples in the example/ directo

Nicholas Krämer 487 Jan 08, 2023
mysql relation charts

sqlcharts 自动生成数据库关联关系图 复制settings.py.example 重命名为settings.py 将数据库配置信息填入settings.DATABASE,目前支持mysql和postgresql 执行 python build.py -b,-b是读取数据库表结构,如果只更新匹

6 Aug 22, 2022
Dimensionality reduction in very large datasets using Siamese Networks

ivis Implementation of the ivis algorithm as described in the paper Structure-preserving visualisation of high dimensional single-cell datasets. Ivis

beringresearch 284 Jan 01, 2023
Python+Numpy+OpenGL: fast, scalable and beautiful scientific visualization

Python+Numpy+OpenGL: fast, scalable and beautiful scientific visualization

Glumpy 1.1k Jan 05, 2023
Declarative statistical visualization library for Python

Altair http://altair-viz.github.io Altair is a declarative statistical visualization library for Python. With Altair, you can spend more time understa

Altair 8k Jan 05, 2023
Main repository for Vispy

VisPy: interactive scientific visualization in Python Main website: http://vispy.org VisPy is a high-performance interactive 2D/3D data visualization

vispy 3k Jan 03, 2023
A python script to visualise explain plans as a graph using graphviz

README Needs to be improved Prerequisites Need to have graphiz installed on the machine. Refer to https://graphviz.readthedocs.io/en/stable/manual.htm

Edward Mallia 1 Sep 28, 2021
Some method of processing point cloud

Point-Cloud Some method of processing point cloud inversion the completion pointcloud to incomplete point cloud Some model of encoding point cloud to

Tan 1 Nov 19, 2021
TensorDebugger (TDB) is a visual debugger for deep learning. It extends TensorFlow with breakpoints + real-time visualization of the data flowing through the computational graph

TensorDebugger (TDB) is a visual debugger for deep learning. It extends TensorFlow (Google's Deep Learning framework) with breakpoints + real-time visualization of the data flowing through the comput

Eric Jang 1.4k Dec 15, 2022
Create a visualization for Trump's Tweeted Words Using Python

Data Trump's Tweeted Words This plot illustrates twitter word occurences. We already did the coding I needed for this plot, so I was very inspired to

7 Mar 27, 2022
Epagneul is a tool to visualize and investigate windows event logs

epagneul Epagneul is a tool to visualize and investigate windows event logs. Dep

jurelou 190 Dec 13, 2022
Data parsing and validation using Python type hints

pydantic Data validation and settings management using Python type hinting. Fast and extensible, pydantic plays nicely with your linters/IDE/brain. De

Samuel Colvin 12.1k Jan 06, 2023
Jupyter notebook and datasets from the pandas Q&A video series

Python pandas Q&A video series Read about the series, and view all of the videos on one page: Easier data analysis in Python with pandas. Jupyter Note

Kevin Markham 2k Jan 05, 2023
A Jupyter - Three.js bridge

pythreejs A Python / ThreeJS bridge utilizing the Jupyter widget infrastructure. Getting Started Installation Using pip: pip install pythreejs And the

Jupyter Widgets 844 Dec 27, 2022
Attractors is a package for simulation and visualization of strange attractors.

attractors Attractors is a package for simulation and visualization of strange attractors. Installation The simplest way to install the module is via

Vignesh M 45 Jul 31, 2022
Python code for solving 3D structural problems using the finite element method

3DFEM Python 3D finite element code This python code allows for solving 3D structural problems using the finite element method. New features will be a

Rémi Capillon 6 Sep 29, 2022
Easily configurable, chart dashboards from any arbitrary API endpoint. JSON config only

Flask JSONDash Easily configurable, chart dashboards from any arbitrary API endpoint. JSON config only. Ready to go. This project is a flask blueprint

Chris Tabor 3.3k Dec 31, 2022