Domain Connectivity Analysis Tools to analyze aggregate connectivity patterns across a set of domains during security investigations

Overview

DomainCAT (Domain Connectivity Analysis Tool)

Domain Connectivity Analysis Tool is used to analyze aggregate connectivity patterns across a set of domains during security investigations

This project was a collaborative effort between myself and Matthew Pahl

Introduction

When analyzing pivots during threat hunting, most people approach it from the perspective of “what can a single pivot tell you?” But often actors will set their domains up to use commodity hosting infrastructure, so the number of entities associated with a given pivot are so big they don’t really give you any useful information.

This is where DomainCAT can help. Actors make decisions around domain registration and hosting options when setting up their malicious infrastructure. These can be considered behavioral choices.

  • What registrar(s) do they use?
  • What TLDs do they prefer?
  • What hosting provider(s) do they like?
  • What TLS cert authority do they use?

All of these decisions, together, makeup part of that actor’s infrastructure tools, tactics and procedures (TTPs), and we can analyze them as a whole to look for patterns across a set of domains.

DomainCAT is a tool written in Jupyter Notebooks, a web-based interactive environment that lets you combine text, code, data, and interactive visualizations into your threat hunting toolbelt. The tool analyzes aggregate connectivity patterns across a set of domains looking at every pivot for every domain, asking; what are the shared pivots across these domains, how many shared pivots between each domain, do they have a small pivot count or a really large one? All of these aspects are taken into consideration as it builds out a connectivity graph that models how connected all the domains in an Iris search are to each other.

Example Visualizations:

3D visualization of domain to domain connections based on shared infrastructure, registration and naming patterns

SegmentLocal

2D visualization of domain to domain connection

domain_graph2d.png

DomainCat Tutorial

Click here for the DomainCAT Tutorial documentation

Installation Steps: Docker (recommended)

Note: building the container takes a bit of RAM to compile the resources for the jupyterlab-plotly extension. Bump up your RAM in Docker preferences to around 4Gb while building the container. Then afterwards you can drop it back down to your normal level to run the container

Steps:

Clone the git repository locally

$ git clone https://github.com/DomainTools/DomainCAT.git

Change directory to the domaincat folder

$ cd domaincat

Build the jupyter notebook container

$ docker build --tag domaincat .

Run the jupyter notebook

$ docker run -p 9999:9999 --name domaincat domaincat

Installation Steps: Manual (cross your fingers)

Note: this project uses JupyterLab Widgets, which requires nodejs >= 12.0.0 to be installed...which is on you

Steps:

Clone the git repository locally

$ git clone https://github.com/DomainTools/DomainCAT.git

Change directory to the domaincat folder

$ cd domaincat

Install python libraries

$ pip install -r requirements.txt

JupyterLab widgets extension

$ jupyter labextension install [email protected] --no-build
$ jupyter labextension install @jupyter-widgets/jupyterlab-manager --no-build
$ jupyter labextension install [email protected] --no-build
$ jupyter lab build

Run the jupyter notebook

$ jupyter lab

Plotly Bug: in the 2D visualization of the domain graph there is a weird bug in Plotly Visualization library where if your cursor is directly over the center of a node, the node's tool tip with the domain's name will disappear and if you click the node, it unselects all nodes. So only click on a node if you see it's tool tip

Owner
DomainTools
DomainTools
plotly scatterplots which show molecule images on hover!

molplotly Plotly scatterplots which show molecule images on hovering over the datapoints! Required packages: pandas rdkit jupyter_dash ➡️ See example.

150 Dec 28, 2022
🎨 Python3 binding for `@AntV/G2Plot` Plotting Library .

PyG2Plot 🎨 Python3 binding for @AntV/G2Plot which an interactive and responsive charting library. Based on the grammar of graphics, you can easily ma

hustcc 990 Jan 05, 2023
Automate the case review on legal case documents and find the most critical cases using network analysis

Automation on Legal Court Cases Review This project is to automate the case review on legal case documents and find the most critical cases using netw

Yi Yin 7 Dec 28, 2022
OpenStats is a library built on top of streamlit that extracts data from the Github API and shows the main KPIs

Open Stats Discover and share the KPIs of your OpenSource project. OpenStats is a library built on top of streamlit that extracts data from the Github

Pere Miquel Brull 4 Apr 03, 2022
A minimalistic wrapper around PyOpenGL to save development time

glpy glpy is pyOpenGl wrapper which lets you work with pyOpenGl easily.It is not meant to be a replacement for pyOpenGl but runs on top of pyOpenGl to

Abhinav 9 Apr 02, 2022
Visualizing weather changes across the world using third party APIs and Python.

WEATHER FORECASTING ACROSS THE WORLD Overview Python scripts were created to visualize the weather for over 500 cities across the world at varying di

G Johnson 0 Jun 12, 2021
This is a Cross-Platform Plot Manager for Chia Plotting that is simple, easy-to-use, and reliable.

Swar's Chia Plot Manager A plot manager for Chia plotting: https://www.chia.net/ Development Version: v0.0.1 This is a cross-platform Chia Plot Manage

Swar Patel 1.3k Dec 13, 2022
Small binja plugin to import header file to types

binja-import-header (v1.0.0) Author: matteyeux Import header file to Binary Ninja types view Description: Binary Ninja plugin to import types from C h

matteyeux 15 Dec 10, 2022
Open-questions - Open questions for Bellingcat technical contributors

Open questions for Bellingcat technical contributors These are difficult, long-term projects that would contribute to open source investigations at Be

Bellingcat 234 Dec 31, 2022
patchwork for matplotlib

patchworklib patchwork for matplotlib test code Preparation of example plots import seaborn as sns import numpy as np import pandas as pd #Bri

Mori Hideto 185 Jan 06, 2023
Draw datasets from within Jupyter.

drawdata This small python app allows you to draw a dataset in a jupyter notebook. This should be very useful when teaching machine learning algorithm

vincent d warmerdam 505 Nov 27, 2022
Parse Robinhood 1099 Tax Document from PDF into CSV

Robinhood 1099 Parser This project converts Robinhood Securities 1099 tax document from PDF to CSV file. This tool will be helpful for those who need

Keun Tae (Kevin) Park 52 Jun 10, 2022
A collection of 100 Deep Learning images and visualizations

A collection of Deep Learning images and visualizations. The project has been developed by the AI Summer team and currently contains almost 100 images.

AI Summer 65 Sep 12, 2022
Ana's Portfolio

Ana's Portfolio ✌️ Welcome to my Portfolio! You will find here different Projects I have worked on (from scratch) 💪 Projects 💻 1️⃣ Hangman game (Mad

Ana Katherine Cortes Sobrino 9 Mar 15, 2022
A custom qq-plot for two sample data comparision

QQ-Plot 2 Sample Just a gist to include the custom code to draw a qq-plot in python when dealing with a "two sample problem". This means when u try to

1 Dec 20, 2021
Make sankey, alluvial and sankey bump plots in ggplot

The goal of ggsankey is to make beautiful sankey, alluvial and sankey bump plots in ggplot2

David Sjoberg 156 Jan 03, 2023
Interactive plotting for Pandas using Vega-Lite

pdvega: Vega-Lite plotting for Pandas Dataframes pdvega is a library that allows you to quickly create interactive Vega-Lite plots from Pandas datafra

Altair 342 Oct 26, 2022
Histogramming for analysis powered by boost-histogram

Hist Hist is an analyst-friendly front-end for boost-histogram, designed for Python 3.7+ (3.6 users get version 2.4). See what's new. Installation You

Scikit-HEP Project 97 Dec 25, 2022
Python ts2vg package provides high-performance algorithm implementations to build visibility graphs from time series data.

ts2vg: Time series to visibility graphs The Python ts2vg package provides high-performance algorithm implementations to build visibility graphs from t

Carlos Bergillos 26 Dec 17, 2022
A Python package that provides evaluation and visualization tools for the DexYCB dataset

DexYCB Toolkit DexYCB Toolkit is a Python package that provides evaluation and visualization tools for the DexYCB dataset. The dataset and results wer

NVIDIA Research Projects 107 Dec 26, 2022