Hierarchical Uniform Manifold Approximation and Projection

Overview

pypi_version pypi_downloads

HUMAP exploration on Fashion MNIST dataset

HUMAP

Hierarchical Manifold Approximation and Projection (HUMAP) is a technique based on UMAP for hierarchical non-linear dimensionality reduction. HUMAP allows to:

  1. Focus on important information while reducing the visual burden when exploring whole datasets;
  2. Drill-down the hierarchy according to information demand.

The details of the algorithm can be found in our paper on ArXiv.

Installation

HUMAP was written in C++ for performance purposes, and it has an intuitive Python interface. It depends upon common machine learning libraries, such as scikit-learn and NumPy. It also needs the pybind11 due to the interface between C++ and Python.

Requirements:

  • Python 3.6 or greater
  • numpy
  • scipy
  • scikit-learn
  • pybind11
  • Eigen (C++)

If you have these requirements installed, use PyPI:

pip install humap

For Windows users:

The Eigen library does not have to be installed. Just add the files to C:Eigen or use the manual installation to change Eigen location.

Manual installation:

For manually installing HUMAP, download the project and proceed as follows:

python setup.py bdist_wheel
pip install dist/humap*.whl

Usage examples

HUMAP package follows the same idea of sklearn classes, in which you need to fit and transform data.

Fitting the hierarchy

import humap
from sklearn.datasets import fetch_openml


X, y = fetch_openml('mnist_784', version=1, return_X_y=True)

hUmap = humap.HUMAP()
hUmap.fit(X, y)

HUMAP embedding of top-level MNIST digits

By now, you can control six parameters related to the hierarchy construction and the embedding performed by UMAP.

  • levels: Controls the number of hierarchical levels + the first one (whole dataset). This parameter also controls how many data points are in each hierarchical level. The default is [0.2, 0.2], meaning the HUMAP will produce three levels: The first one with the whole dataset, the second one with 20% of the first level, and the third with 20% of the second level.
  • n_neighbors: This parameter controls the number of neighbors for approximating the manifold structures. Larger values produce embedding that preserves more of the global relations. In HUMAP, we recommend and set the default value to be 100.
  • min_dist: This parameter, used in UMAP dimensionality reduction, controls the allowance to cluster data points together. According to UMAP documentation, larger values allow evenly distributed embeddings, while smaller values encode the local structures better. We set this parameter as 0.15 as default.
  • knn_algorithm: Controls which knn approximation will be used, in which NNDescent is the default. Another option is ANNOY or FLANN if you have Python installations of these algorithms at the expense of slower run-time executions than NNDescent.
  • init: Controls the method for initing the low-dimensional representation. We set Spectral as default since it yields better global structure preservation. You can also use random initialization.
  • verbose: Controls the verbosity of the algorithm.

Embedding a hierarchical level

After fitting the dataset, you can generate the embedding for a hierarchical level by specifying the level.

embedding_l2 = hUmap.transform(2)
y_l2 = hUmap.labels(2)

Notice that the .labels() method only works for levels equal or greater than one.

Drilling down the hierarchy by embedding a subset of data points based on indices

Embedding data subsets throughout HUMAP hierarchy

When interested in a set of data samples, HUMAP allows for drilling down the hierarchy for those samples.

embedding, y, indices = hUmap.transform(2, indices=indices_of_interest)

This method returns the embedding coordinates, the labels (y), and the data points' indices in the current level. Notice that the current level is now level 1 since we used the hierarchy level 2 for drilling down operation.

Drilling down the hierarchy by embedding a subset of data points based on labels

You can apply the same concept as above to embed data points based on labels.

embedding, y, indices = hUmap.transform(2, indices=np.array([4, 9]), class_based=True)

C++ UMAP implementation

You can also fit a one-level HUMAP hierarchy, which essentially corresponds to a UMAP projection.

umap_reducer = humap.HUMAP(np.array([]))
umap_reducer.fit(X, y)

embedding = umap_reducer.transform(0)

Citation

Please, use the following reference to cite HUMAP in your work:

@misc{marciliojr_humap2021,
  title={HUMAP: Hierarchical Uniform Manifold Approximation and Projection},
  author={Wilson E. Marcílio-Jr and Danilo M. Eler and Fernando V. Paulovich and Rafael M. Martins},
  year={2021},
  eprint={2106.07718},
  archivePrefix={arXiv},
  primaryClass={cs.LG}
    }

License

HUMAP follows the 3-clause BSD license and it uses the open-source NNDescent implementation from EFANNA. It also uses a C++ implementation of UMAP for embedding hierarchy levels; this project would not be possible without UMAP's fantastic technique and package.

E-mail me (wilson_jr at outlook.com) if you like to contribute.


You might also like...
Finite-temperature variational Monte Carlo calculation of uniform electron gas using neural canonical transformation.

CoulombGas This code implements the neural canonical transformation approach to the thermodynamic properties of uniform electron gas. Building on JAX,

Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation. In CVPR 2022.
Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation. In CVPR 2022.

Nonuniform-to-Uniform Quantization This repository contains the training code of N2UQ introduced in our CVPR 2022 paper: "Nonuniform-to-Uniform Quanti

Curved Projection Reformation
Curved Projection Reformation

Description Assuming that we already know the image of the centerline, we want the lumen to be displayed on a plane, which requires curved projection

Implementation of
Implementation of "Bidirectional Projection Network for Cross Dimension Scene Understanding" CVPR 2021 (Oral)

Bidirectional Projection Network for Cross Dimension Scene Understanding CVPR 2021 (Oral) [ Project Webpage ] [ arXiv ] [ Video ] Existing segmentatio

my graduation project is about live human face augmentation by projection mapping by using CNN
my graduation project is about live human face augmentation by projection mapping by using CNN

Live-human-face-expression-augmentation-by-projection my graduation project is about live human face augmentation by projection mapping by using CNN o

Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology

Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology Sharon Zhou, Eric Zelikman

Code for Learning Manifold Patch-Based Representations of Man-Made Shapes, in ICLR 2021.
Code for Learning Manifold Patch-Based Representations of Man-Made Shapes, in ICLR 2021.

LearningPatches | Webpage | Paper | Video Learning Manifold Patch-Based Representations of Man-Made Shapes Dmitriy Smirnov, Mikhail Bessmeltsev, Justi

A Pytorch implementation of
A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generative Modeling" (ICCV 2021)

Manifold Matching via Deep Metric Learning for Generative Modeling A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generat

Research Artifact of USENIX Security 2022 Paper: Automated Side Channel Analysis of Media Software with Manifold Learning

Manifold-SCA Research Artifact of USENIX Security 2022 Paper: Automated Side Channel Analysis of Media Software with Manifold Learning The repo is org

Comments
  • [Packaging] Requesting conda-forge package

    [Packaging] Requesting conda-forge package

    Hi,

    Just putting it out there that you might want to consider putting up your package on conda-forge. Many other packages like numpy, scikit-learn, umap, are all available on conda-forge, and managing them through conda cli makes it easy to be up-to-date and not worry about dependencies like MKL, which pip doesn't handle well.

    As a bonus, I see that this package depends on Eigen, which needs to be manually configured on Windows. Conda-forge already has eigen available, which might make this much less error-prone for Windows users, which I assume will be a substantial chunk.

    Just as an FYI, here is a link for conda-forge submission process.

    Thanks!

    opened by stallam-unb 6
  • RuntimeError: Some rows contain fewer than n_neighbors distances

    RuntimeError: Some rows contain fewer than n_neighbors distances

    Problems when computing hierarchy for small datasets. I tried to execute HUMAP on Iris dataset using 100, 15, and 10 n_neighbors.

    RuntimeError: Some rows contain fewer than n_neighbors distances

    opened by wilsonjr 1
  • Transform with new data?

    Transform with new data?

    Semi-related to #4 , but my case is that I want to use HUMAP on a supervised data where I have a training data with labels, and I want to be able to project new test data with the same embeddings. UMAP supports this use case, I was wondering if this would be theoretically possible with HUMAP as well? Would be nice to be able to use HUMAP to interpret classifier decisions.

    opened by stallam-unb 0
  • Semi-supervised learning?

    Semi-supervised learning?

    Thanks for writing this awesome library, only recently discovered it. Do you have plans to support semi-supervised umap? From my first try outs of your library, this is the fastest (h)umap implementation which has nndescent. I would like to use it for semi-supervised learning, too.

    enhancement 
    opened by KnutJaegersberg 6
Releases(v0.2.1)
Owner
Wilson Estécio Marcílio Júnior
PhD Candidate in Computer Science. Interested in ML and Explainability.
Wilson Estécio Marcílio Júnior
An implementation for the loss function proposed in Decoupled Contrastive Loss paper.

Decoupled-Contrastive-Learning This repository is an implementation for the loss function proposed in Decoupled Contrastive Loss paper. Requirements P

Ramin Nakhli 71 Dec 04, 2022
A particular navigation route using satellite feed and can help in toll operations & traffic managemen

How about adding some info that can quanitfy the stress on a particular navigation route using satellite feed and can help in toll operations & traffic management The current analysis is on the satel

Ashish Pandey 1 Feb 14, 2022
Pixray is an image generation system

Pixray is an image generation system

pixray 883 Jan 07, 2023
NeurIPS'21 Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows

NeurIPS'21 Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows This repo contains the code for the paper Tractable Densit

Layer6 Labs 4 Dec 12, 2022
This program generates a random 12 digit/character password (upper and lowercase) and stores it in a file along with your username and app/website.

PasswordGeneratorAndVault This program generates a random 12 digit/character password (upper and lowercase) and stores it in a file along with your us

Chris 1 Feb 26, 2022
Official PyTorch implementation of "Evolving Search Space for Neural Architecture Search"

Evolving Search Space for Neural Architecture Search Usage Install all required dependencies in requirements.txt and replace all ..path/..to in the co

Yuanzheng Ci 10 Oct 24, 2022
The Official PyTorch Implementation of "VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models" (ICLR 2021 spotlight paper)

Official PyTorch implementation of "VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models" (ICLR 2021 Spotlight Paper) Zhisheng

NVIDIA Research Projects 45 Dec 26, 2022
Qlib is an AI-oriented quantitative investment platform

Qlib is an AI-oriented quantitative investment platform, which aims to realize the potential, empower the research, and create the value of AI technologies in quantitative investment.

Microsoft 10.1k Dec 30, 2022
CS50's Introduction to Artificial Intelligence Test Scripts

CS50's Introduction to Artificial Intelligence Test Scripts 🤷‍♂️ What's this? 🤷‍♀️ This repository contains Python scripts to automate tests for mos

Jet Kan 2 Dec 28, 2022
face_recognization (FaceNet) + TFHE (HNP) + hand_face_detection (Mediapipe)

SuperControlSystem Face_Recognization (FaceNet) 面部识别 (FaceNet) Fully Homomorphic Encryption over the Torus (HNP) 环面全同态加密 (TFHE) Hand_Face_Detection (M

liziyu0104 2 Dec 30, 2021
An unofficial implementation of "Unpaired Image Super-Resolution using Pseudo-Supervision." CVPR2020

UnpairedSR An unofficial implementation of "Unpaired Image Super-Resolution using Pseudo-Supervision." CVPR2020 turn RCAN(modified) -- xmodel(xilinx

JiaKui Hu 10 Oct 28, 2022
CasualHealthcare's Pneumonia detection with Artificial Intelligence (Convolutional Neural Network)

CasualHealthcare's Pneumonia detection with Artificial Intelligence (Convolutional Neural Network) This is PneumoniaDiagnose, an artificially intellig

Azhaan 2 Jan 03, 2022
Cossim - Sharpened Cosine Distance implementation in PyTorch

Sharpened Cosine Distance PyTorch implementation of the Sharpened Cosine Distanc

Istvan Fehervari 10 Mar 22, 2022
Fully-automated scripts for collecting AI-related papers

AI-Paper-collector Fully-automated scripts for collecting AI-related papers List of Conferences to crawel ACL: 21-19 (including findings) EMNLP: 21-19

Gordon Lee 776 Jan 08, 2023
Tools for investing in Python

InvestOps Original repository on GitHub Original author is Magnus Erik Hvass Pedersen Introduction This is a Python package with simple and effective

24 Nov 26, 2022
Implements an infinite sum of poisson-weighted convolutions

An infinite sum of Poisson-weighted convolutions Kyle Cranmer, Aug 2018 If viewing on GitHub, this looks better with nbviewer: click here Consider a v

Kyle Cranmer 26 Dec 07, 2022
《Lerning n Intrinsic Grment Spce for Interctive Authoring of Grment Animtion》

Learning an Intrinsic Garment Space for Interactive Authoring of Garment Animation Overview This is the demo code for training a motion invariant enco

YuanBo 213 Dec 14, 2022
Companion repository to the paper accepted at the 4th ACM SIGSPATIAL International Workshop on Advances in Resilient and Intelligent Cities

Transfer learning approach to bicycle sharing systems station location planning using OpenStreetMap Companion repository to the paper accepted at the

Politechnika Wrocławska - repozytorium dla informatyków 4 Oct 24, 2022
This is a re-implementation of TransGAN: Two Pure Transformers Can Make One Strong GAN (CVPR 2021) in PyTorch.

TransGAN: Two Transformers Can Make One Strong GAN [YouTube Video] Paper Authors: Yifan Jiang, Shiyu Chang, Zhangyang Wang CVPR 2021 This is re-implem

Ahmet Sarigun 79 Jan 05, 2023
Official git for "CTAB-GAN: Effective Table Data Synthesizing"

CTAB-GAN This is the official git paper CTAB-GAN: Effective Table Data Synthesizing. The paper is published on Asian Conference on Machine Learning (A

30 Dec 26, 2022