Hierarchical Uniform Manifold Approximation and Projection

Overview

pypi_version pypi_downloads

HUMAP exploration on Fashion MNIST dataset

HUMAP

Hierarchical Manifold Approximation and Projection (HUMAP) is a technique based on UMAP for hierarchical non-linear dimensionality reduction. HUMAP allows to:

  1. Focus on important information while reducing the visual burden when exploring whole datasets;
  2. Drill-down the hierarchy according to information demand.

The details of the algorithm can be found in our paper on ArXiv.

Installation

HUMAP was written in C++ for performance purposes, and it has an intuitive Python interface. It depends upon common machine learning libraries, such as scikit-learn and NumPy. It also needs the pybind11 due to the interface between C++ and Python.

Requirements:

  • Python 3.6 or greater
  • numpy
  • scipy
  • scikit-learn
  • pybind11
  • Eigen (C++)

If you have these requirements installed, use PyPI:

pip install humap

For Windows users:

The Eigen library does not have to be installed. Just add the files to C:Eigen or use the manual installation to change Eigen location.

Manual installation:

For manually installing HUMAP, download the project and proceed as follows:

python setup.py bdist_wheel
pip install dist/humap*.whl

Usage examples

HUMAP package follows the same idea of sklearn classes, in which you need to fit and transform data.

Fitting the hierarchy

import humap
from sklearn.datasets import fetch_openml


X, y = fetch_openml('mnist_784', version=1, return_X_y=True)

hUmap = humap.HUMAP()
hUmap.fit(X, y)

HUMAP embedding of top-level MNIST digits

By now, you can control six parameters related to the hierarchy construction and the embedding performed by UMAP.

  • levels: Controls the number of hierarchical levels + the first one (whole dataset). This parameter also controls how many data points are in each hierarchical level. The default is [0.2, 0.2], meaning the HUMAP will produce three levels: The first one with the whole dataset, the second one with 20% of the first level, and the third with 20% of the second level.
  • n_neighbors: This parameter controls the number of neighbors for approximating the manifold structures. Larger values produce embedding that preserves more of the global relations. In HUMAP, we recommend and set the default value to be 100.
  • min_dist: This parameter, used in UMAP dimensionality reduction, controls the allowance to cluster data points together. According to UMAP documentation, larger values allow evenly distributed embeddings, while smaller values encode the local structures better. We set this parameter as 0.15 as default.
  • knn_algorithm: Controls which knn approximation will be used, in which NNDescent is the default. Another option is ANNOY or FLANN if you have Python installations of these algorithms at the expense of slower run-time executions than NNDescent.
  • init: Controls the method for initing the low-dimensional representation. We set Spectral as default since it yields better global structure preservation. You can also use random initialization.
  • verbose: Controls the verbosity of the algorithm.

Embedding a hierarchical level

After fitting the dataset, you can generate the embedding for a hierarchical level by specifying the level.

embedding_l2 = hUmap.transform(2)
y_l2 = hUmap.labels(2)

Notice that the .labels() method only works for levels equal or greater than one.

Drilling down the hierarchy by embedding a subset of data points based on indices

Embedding data subsets throughout HUMAP hierarchy

When interested in a set of data samples, HUMAP allows for drilling down the hierarchy for those samples.

embedding, y, indices = hUmap.transform(2, indices=indices_of_interest)

This method returns the embedding coordinates, the labels (y), and the data points' indices in the current level. Notice that the current level is now level 1 since we used the hierarchy level 2 for drilling down operation.

Drilling down the hierarchy by embedding a subset of data points based on labels

You can apply the same concept as above to embed data points based on labels.

embedding, y, indices = hUmap.transform(2, indices=np.array([4, 9]), class_based=True)

C++ UMAP implementation

You can also fit a one-level HUMAP hierarchy, which essentially corresponds to a UMAP projection.

umap_reducer = humap.HUMAP(np.array([]))
umap_reducer.fit(X, y)

embedding = umap_reducer.transform(0)

Citation

Please, use the following reference to cite HUMAP in your work:

@misc{marciliojr_humap2021,
  title={HUMAP: Hierarchical Uniform Manifold Approximation and Projection},
  author={Wilson E. Marcílio-Jr and Danilo M. Eler and Fernando V. Paulovich and Rafael M. Martins},
  year={2021},
  eprint={2106.07718},
  archivePrefix={arXiv},
  primaryClass={cs.LG}
    }

License

HUMAP follows the 3-clause BSD license and it uses the open-source NNDescent implementation from EFANNA. It also uses a C++ implementation of UMAP for embedding hierarchy levels; this project would not be possible without UMAP's fantastic technique and package.

E-mail me (wilson_jr at outlook.com) if you like to contribute.


You might also like...
Finite-temperature variational Monte Carlo calculation of uniform electron gas using neural canonical transformation.

CoulombGas This code implements the neural canonical transformation approach to the thermodynamic properties of uniform electron gas. Building on JAX,

Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation. In CVPR 2022.
Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation. In CVPR 2022.

Nonuniform-to-Uniform Quantization This repository contains the training code of N2UQ introduced in our CVPR 2022 paper: "Nonuniform-to-Uniform Quanti

Curved Projection Reformation
Curved Projection Reformation

Description Assuming that we already know the image of the centerline, we want the lumen to be displayed on a plane, which requires curved projection

Implementation of
Implementation of "Bidirectional Projection Network for Cross Dimension Scene Understanding" CVPR 2021 (Oral)

Bidirectional Projection Network for Cross Dimension Scene Understanding CVPR 2021 (Oral) [ Project Webpage ] [ arXiv ] [ Video ] Existing segmentatio

my graduation project is about live human face augmentation by projection mapping by using CNN
my graduation project is about live human face augmentation by projection mapping by using CNN

Live-human-face-expression-augmentation-by-projection my graduation project is about live human face augmentation by projection mapping by using CNN o

Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology

Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology Sharon Zhou, Eric Zelikman

Code for Learning Manifold Patch-Based Representations of Man-Made Shapes, in ICLR 2021.
Code for Learning Manifold Patch-Based Representations of Man-Made Shapes, in ICLR 2021.

LearningPatches | Webpage | Paper | Video Learning Manifold Patch-Based Representations of Man-Made Shapes Dmitriy Smirnov, Mikhail Bessmeltsev, Justi

A Pytorch implementation of
A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generative Modeling" (ICCV 2021)

Manifold Matching via Deep Metric Learning for Generative Modeling A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generat

Research Artifact of USENIX Security 2022 Paper: Automated Side Channel Analysis of Media Software with Manifold Learning

Manifold-SCA Research Artifact of USENIX Security 2022 Paper: Automated Side Channel Analysis of Media Software with Manifold Learning The repo is org

Comments
  • [Packaging] Requesting conda-forge package

    [Packaging] Requesting conda-forge package

    Hi,

    Just putting it out there that you might want to consider putting up your package on conda-forge. Many other packages like numpy, scikit-learn, umap, are all available on conda-forge, and managing them through conda cli makes it easy to be up-to-date and not worry about dependencies like MKL, which pip doesn't handle well.

    As a bonus, I see that this package depends on Eigen, which needs to be manually configured on Windows. Conda-forge already has eigen available, which might make this much less error-prone for Windows users, which I assume will be a substantial chunk.

    Just as an FYI, here is a link for conda-forge submission process.

    Thanks!

    opened by stallam-unb 6
  • RuntimeError: Some rows contain fewer than n_neighbors distances

    RuntimeError: Some rows contain fewer than n_neighbors distances

    Problems when computing hierarchy for small datasets. I tried to execute HUMAP on Iris dataset using 100, 15, and 10 n_neighbors.

    RuntimeError: Some rows contain fewer than n_neighbors distances

    opened by wilsonjr 1
  • Transform with new data?

    Transform with new data?

    Semi-related to #4 , but my case is that I want to use HUMAP on a supervised data where I have a training data with labels, and I want to be able to project new test data with the same embeddings. UMAP supports this use case, I was wondering if this would be theoretically possible with HUMAP as well? Would be nice to be able to use HUMAP to interpret classifier decisions.

    opened by stallam-unb 0
  • Semi-supervised learning?

    Semi-supervised learning?

    Thanks for writing this awesome library, only recently discovered it. Do you have plans to support semi-supervised umap? From my first try outs of your library, this is the fastest (h)umap implementation which has nndescent. I would like to use it for semi-supervised learning, too.

    enhancement 
    opened by KnutJaegersberg 6
Releases(v0.2.1)
Owner
Wilson Estécio Marcílio Júnior
PhD Candidate in Computer Science. Interested in ML and Explainability.
Wilson Estécio Marcílio Júnior
The implementation our EMNLP 2021 paper "Enhanced Language Representation with Label Knowledge for Span Extraction".

LEAR The implementation our EMNLP 2021 paper "Enhanced Language Representation with Label Knowledge for Span Extraction". **The code is in the "master

杨攀 93 Jan 07, 2023
A general and strong 3D object detection codebase that supports more methods, datasets and tools (debugging, recording and analysis).

ALLINONE-Det ALLINONE-Det is a general and strong 3D object detection codebase built on OpenPCDet, which supports more methods, datasets and tools (de

Michael.CV 5 Nov 03, 2022
[CVPR'21] Projecting Your View Attentively: Monocular Road Scene Layout Estimation via Cross-view Transformation

Projecting Your View Attentively: Monocular Road Scene Layout Estimation via Cross-view Transformation Weixiang Yang, Qi Li, Wenxi Liu, Yuanlong Yu, Y

118 Dec 26, 2022
Official Python implementation of the FuzionCoin protocol

PyFuzc Official Python implementation of the FuzionCoin protocol WARNING: Under construction. Use at your own risk. Some functions may not work. Setup

FuzionCoin 3 Jul 07, 2022
Classify music genre from a 10 second sound stream using a Neural Network.

MusicGenreClassification Academic research in the field of Deep Learning (Deep Neural Networks) and Sound Processing, Tel Aviv University. Featured in

Matan Lachmish 453 Dec 27, 2022
StarGAN-ZSVC: Unofficial PyTorch Implementation

This repository is an unofficial PyTorch implementation of StarGAN-ZSVC by Matthew Baas and Herman Kamper. This repository provides both model architectures and the code to inference or train them.

Jirayu Burapacheep 11 Aug 28, 2022
Hunt down social media accounts by username across social networks

Hunt down social media accounts by username across social networks Installation | Usage | Docker Notes | Contributing Installation # clone the repo $

1 Dec 14, 2021
Privacy-Preserving Machine Learning (PPML) Tutorial Presented at PyConDE 2022

PPML: Machine Learning on Data you cannot see Repository for the tutorial on Privacy-Preserving Machine Learning (PPML) presented at PyConDE 2022 Abst

Valerio Maggio 10 Aug 16, 2022
A CROSS-MODAL FUSION NETWORK BASED ON SELF-ATTENTION AND RESIDUAL STRUCTURE FOR MULTIMODAL EMOTION RECOGNITION

CFN-SR A CROSS-MODAL FUSION NETWORK BASED ON SELF-ATTENTION AND RESIDUAL STRUCTURE FOR MULTIMODAL EMOTION RECOGNITION The audio-video based multimodal

skeleton 15 Sep 26, 2022
A command line simple note taking app

Why yet another note taking program? note was designed with a very specific target in mind: me, and my 2354 scraps of paper. It runs from the command

64 Nov 20, 2022
Public Models considered for emotion estimation from EEG

Emotion-EEG Set of models for emotion estimation from EEG. Composed by the combination of two deep-learing models learning together (RNN and CNN) with

Victor Delvigne 21 Dec 23, 2022
Generative Query Network (GQN) in PyTorch as described in "Neural Scene Representation and Rendering"

Update 2019/06/24: A model trained on 10% of the Shepard-Metzler dataset has been added, the following notebook explains the main features of this mod

Jesper Wohlert 313 Dec 27, 2022
Edison AT is software Depression Assistant personal.

Edison AT Edison AT is software / program Depression Assistant personal. Feature: Analyze emotional real-time from face. Audio Edison(Comingsoon relea

Ananda Rauf 2 Apr 24, 2022
Vehicle speed detection with python

Vehicle-speed-detection In the project simulate the tracker.py first then simulate the SpeedDetector.py. Finally, a new window pops up and the output

3 Dec 15, 2022
Using machine learning to predict undergrad college admissions.

College-Prediction Project- Overview: Many have tried, many have failed. Few trailblazers are ambitious enought to chase acceptance into the top 15 un

John H Klinges 1 Jan 05, 2022
A modified version of DeepMind's Alphafold2 to divide CPU part (MSA and template searching) and GPU part (prediction model)

ParallelFold Author: Bozitao Zhong This is a modified version of DeepMind's Alphafold2 to divide CPU part (MSA and template searching) and GPU part (p

Bozitao Zhong 77 Dec 22, 2022
Official Pytorch implementation of the paper "Action-Conditioned 3D Human Motion Synthesis with Transformer VAE", ICCV 2021

ACTOR Official Pytorch implementation of the paper "Action-Conditioned 3D Human Motion Synthesis with Transformer VAE", ICCV 2021. Please visit our we

Mathis Petrovich 248 Dec 23, 2022
[PyTorch] Official implementation of CVPR2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency". https://arxiv.org/abs/2103.05465

PointDSC repository PyTorch implementation of PointDSC for CVPR'2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency",

153 Dec 14, 2022
Official PyTorch implementation of the paper Image-Based CLIP-Guided Essence Transfer.

TargetCLIP- official pytorch implementation of the paper Image-Based CLIP-Guided Essence Transfer This repository finds a global direction in StyleGAN

Hila Chefer 221 Dec 13, 2022
Fully Convolutional DenseNets for semantic segmentation.

Introduction This repo contains the code to train and evaluate FC-DenseNets as described in The One Hundred Layers Tiramisu: Fully Convolutional Dense

485 Nov 26, 2022