HashDB API hash lookup plugin for IDA Pro

Overview

GitHub release Chat

HashDB IDA Plugin

Malware string hash lookup plugin for IDA Pro. This plugin connects to the OALABS HashDB Lookup Service.

Adding New Hash Algorithms

The hash algorithm database is open source and new algorithms can be added on GitHub here. Pull requests are mostly automated and as long as our automated tests pass the new algorithm will be usable on HashDB within minutes.

Using HashDB

HashDB can be used to look up strings that have been hashed in malware by right-clicking on the hash constant in the IDA disassembly view and launching the HashDB Lookup client.

Settings

Before the plugin can be used to look up hashes the HashDB settings must be configured. The settings window can be launched from the plugins menu Edit->Plugins->HashDB.

Screen Shot 2021-09-24 at 4 23 19 PM

Hash Algorithms

Click Refresh Algorithms to pull a list of supported hash algorithms from the HashDB API, then select the algorithm used in the malware you are analyzing.

Optional XOR

There is also an option to enable XOR with each hash value as this is a common technique used by malware authors to further obfuscate hashes.

API URL

The default API URL for the HashDB Lookup Service is https://hashdb.openanalysis.net/. If you are using your own internal server this URL can be changed to point to your server.

Enum Name

When a new hash is identified by HashDB the hash and its associated string are added to an enum in IDA. This enum can then be used to convert hash constants in IDA to their corresponding enum name. The enum name is configurable from the settings in the event that there is a conflict with an existing enum.

Hash Lookup

Once the plugin settings have been configured you can right-click on any constant in the IDA disassembly window and look up the constant as a hash. The right-click also provides a quick way to set the XOR value if needed.

Screen Shot 2021-09-24 at 4 23 47 PM

Bulk Import

If a hash is part of a module a prompt will ask if you want to import all the hashes from that module. This is a quick way to pull hashes in bulk. For example, if one of the hashes identified is Sleep from the kernel32 module, HashDB can then pull all the hashed exports from kernel32.

Screen Shot 2021-09-24 at 4 24 06 PM

Installing HashDB

Before using the plugin you must install the python requests module in your IDA environment. The simplest way to do this is to use pip from a shell outside of IDA.
pip install requests

Once you have the requests module installed simply copy the latest release of hashdb.py into your IDA plugins directory and you are ready to start looking up hashes!

Compatibility Issues

The HashDB plugin has been developed for use with the IDA 7+ and Python 3 it is not backwards compatible.

Comments
  • Unable to create enum of strings with special characters

    Unable to create enum of strings with special characters

    An enum member is not populated properly when the resulting string has a special character - only tested with a preceding - (e.g. -path), unsure if position matters or if other characters cause the issue. Assuming it is IDA rejecting the string as a valid enum member name.

    Sample (BlackMatter v2): https://www.virustotal.com/gui/file/520bd9ed608c668810971dbd51184c6a29819674280b018dc4027bc38fc42e57

    Function 0x411464 contains hashes for command line parameter strings that reproduce the issue. The algorithm is add_ror13. The hashes are correctly identified by HashDB, but the string is not added to the hashdb_strings enum.

    image

    bug ida-plugin 
    opened by Demonslay335 5
  • HashDB Lookup kills IDA on M1 macs

    HashDB Lookup kills IDA on M1 macs

    Tested versions: main branch at 4d73263cffac975e1e47e67eb7fb55bfb85416a2 and release version 1.4.0

    IDA Version: Version 7.6.210427 macOS arm64

    Python: 3.9.7

    Steps to reproduce:

    1. Load malshare sample (from OALabs YT Video) https://malshare.com/sample.php?action=detail&hash=132fa71af952927e1961f735e68ae38a3305e7ae8d7197c170d071f74db60d1c

    2. Right click on hash 992892395 in sub_4010D7 Select "HashDB Lookup"

    Result: IDA crashes with "Oops! internal error 40217 occurred. Screenshot 2021-10-23 at 16 08 50 Screenshot 2021-10-23 at 16 08 43 "

    bug good first issue ida-plugin 
    opened by Lichtlos 5
  • Asynchronous requests

    Asynchronous requests

    When performing requests the UI (especially with large IAT scans) the UI is frozen and IDA is unusable while the requests are performed. To fix this we should use asyncio's Futures.

    Example combined with add_done_callback: image image

    More elegant alternatives would include the use of async libraries such as:

    enhancement help wanted ida-plugin 
    opened by anthonyprintup 5
  • TypeError: hunt_algorithm_done() takes from 0 to 1 positional arguments but 2 were given

    TypeError: hunt_algorithm_done() takes from 0 to 1 positional arguments but 2 were given

    Popped up when there's more than 1 hash table that matched the hash.

    HashDB: Hunting for a hash algorithm, please wait! Timeout: 15 seconds.
    CRITICAL:root:hunt_algorithm_request errored: Traceback (most recent call last):
      File "C:/Users/sang/Downloads/IDA Pro 7.5.20.1028 SP3 Portable + All decompilers (Windows)/IDA Pro 7.5.20.1028 SP3 Portable/plugins/hashdb.py", line 333, in __wrapper
        self.__done_callback(*result)
    TypeError: hunt_algorithm_done() takes from 0 to 1 positional arguments but 2 were given
    
    ERROR: HashDB hash scan failed: Traceback (most recent call last):
      File "C:/Users/sang/Downloads/IDA Pro 7.5.20.1028 SP3 Portable + All decompilers (Windows)/IDA Pro 7.5.20.1028 SP3 Portable/plugins/hashdb.py", line 333, in __wrapper
        self.__done_callback(*result)
    TypeError: hunt_algorithm_done() takes from 0 to 1 positional arguments but 2 were given
    
    ERROR:root:Caught an unexpected exception: TypeError, raising.
    Traceback (most recent call last):
      File "C:/Users/sang/Downloads/IDA Pro 7.5.20.1028 SP3 Portable + All decompilers (Windows)/IDA Pro 7.5.20.1028 SP3 Portable/plugins/hashdb.py", line 333, in __wrapper
        self.__done_callback(*result)
    TypeError: hunt_algorithm_done() takes from 0 to 1 positional arguments but 2 were given
    Exception in thread Thread-2:
    Traceback (most recent call last):
      File "C:\Users\sang\AppData\Local\Programs\Python\Python37\Lib\threading.py", line 926, in _bootstrap_inner
        self.run()
      File "C:/Users/sang/Downloads/IDA Pro 7.5.20.1028 SP3 Portable + All decompilers (Windows)/IDA Pro 7.5.20.1028 SP3 Portable/plugins/hashdb.py", line 385, in run
        super().run()
      File "C:\Users\sang\AppData\Local\Programs\Python\Python37\Lib\threading.py", line 870, in run
        self._target(*self._args, **self._kwargs)
      File "C:/Users/sang/Downloads/IDA Pro 7.5.20.1028 SP3 Portable + All decompilers (Windows)/IDA Pro 7.5.20.1028 SP3 Portable/plugins/hashdb.py", line 344, in __wrapper
        raise exception
      File "C:/Users/sang/Downloads/IDA Pro 7.5.20.1028 SP3 Portable + All decompilers (Windows)/IDA Pro 7.5.20.1028 SP3 Portable/plugins/hashdb.py", line 333, in __wrapper
        self.__done_callback(*result)
    TypeError: hunt_algorithm_done() takes from 0 to 1 positional arguments but 2 were given
    
    opened by sas0125 4
  • TypeError: unsupported operand type(s) for |: 'type' and 'type'

    TypeError: unsupported operand type(s) for |: 'type' and 'type'

    Hi,

    Thank you for sharing this fantastic plugin!

    I was using hashdb in latest version of ida pro on win 7 x64 python 3.8 the following line of code threw the above error.

    HASHDB_REQUEST_TIMEOUT: int | float = 15 # Limit to 15 seconds

    I realised its because python 3.8 doesn't support use of '|' as an Union operator.

    I guess the work around is to modify the code to fall back to "Union". as I prefer to work with win 7 and the last of python it supports is 3.8 :(

    Just wanted to let you guys know in case someone else faced the same issue.

    bug ida-plugin 
    opened by peta909 4
  • Fix #34

    Fix #34

    This PR fixes issue #34.

    Detailed description:

    • added a new import ida_typeinf for typeinf related APIs,
    • added a new helper function get_existing_enum_values which checks if an enum already exists and parses its values using the IDA API (ida_typeinf.get_named_type, ida_typeinf.idc_print_type),
    • add_enums will now check for existing enum values before parsing the provided hash list,
    • add_enums now manually corrects strings with dots in their names (e.g. ntdll.dll -> ntdll_dll),
    • add_enums now uses ida_name.get_name_ea to check if a name already exists in the database (previously used ida_enum.add_enum_member which checked this automatically)
    • add_enums no longer expects a 3rd function parameter because the enum size now depends on the database type or the algorithm size (64-bit databases will always generate 64-bit enums),
    • add_enums: replaced the method used when creating an enum (massive performance improvements); using ida_typeinf.idc_parse_types and ida_typeinf.import_type.

    Possible (but unlikely) bugs:

    • get_existing_enum_values will not parse values properly on expected integer bases (e.g. if for some reason the values aren't in decimal or hexadecimal form),
    • because we no longer use ida_enum.add_enum_member to append enum members we lose some internal corrections that IDA performs on the enum names (e.g. replacing dots with underscores); this has to be corrected manually from now on.

    Disclaimer:

    I've only tested the plugin on two samples (one 32-bit and one 64-bit). Strings with invalid names (e.g. -path) haven't been tested yet, but should work (look at #31). TLDR; more testing required before we can push a release.

    enhancement ida-plugin bugfix 
    opened by anthonyprintup 2
  • [BUG]:

    [BUG]:

    Steps to reproduce:

    1. Just placed the script at the plugins and ran refresh nothing special

    Stack trace:

    {"user_data": {"platform": "win32", "python_version": "3.8.2", "plugin_version": "1.8.0", "ida": {"kernel_version": "7.6", "bits": 32}}, "exception_data": {"exception_type": "UnboundLocalError", "exception_value": "local variable 'algorithms' referenced before assignment"}, "frames": [{"frame_index": 2, "line_number": 570, "function_name": "OnBtnRefresh", "line": "sorted_algorithms = sorted(algorithms, key = lambda algorithm: algorithm[0].lower())", "locals": {"self": "<__plugins__hashdb.hashdb_settings_t object at 0x000002BA20DAA2B0>", "code": "0", "api_url": "'https://hashdb.openanalysis.net'"}}]}
    

    Error Log

      bytes   pages size description
    --------- ----- ---- --------------------------------------------
       524288    64 8192 allocating memory for b-tree...
       278528    34 8192 allocating memory for virtual array...
       262144    32 8192 allocating memory for name pointers...
    -----------------------------------------------------------------
      1064960            total memory allocated
    
    Loading processor module C:\Program Files\IDA Pro 7.6\procs\pc.dll for metapc...Initializing processor module metapc...OK
    Loading type libraries...
    Autoanalysis subsystem has been initialized.
    ....
    
      Please check the Edit/Plugins menu for more informaton.
    ================================================================================
       _   _           _    ____________ 
      | | | |         | |   |  _  \ ___ \ 
      | |_| | __ _ ___| |__ | | | | |_/ /
      |  _  |/ _` / __| '_ \| | | | ___ \ 
      | | | | (_| \__ \ | | | |/ /| |_/ /
      \_| |_/\__,_|___/_| |_|___/ \____/ 
    
    
    HashDB v1.8.0 by @herrcore
    
    HashDB search shortcut key is Alt+`
    ================================================================================
    No saved HashDB configuration
    ---------------------------------------------------------------------------------------
    Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64)] 
    IDAPython v7.4.0 final (serial 0) (c) The IDAPython Team <[email protected]>
    ---------------------------------------------------------------------------------------
    ERROR: HashDB API request failed: HTTPSConnectionPool(host='hashdb.openanalysis.net', port=443): Max retries exceeded with url: /hash (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1108)')))
    Traceback (most recent call last):
      File "_ctypes/callbacks.c", line 237, in 'calling callback function'
      File "C:\Program Files\IDA Pro 7.6\python\3\ida_kernwin.py", line 9792, in helper_cb
        r = self.handler(button_code)
      File "C:/Program Files/IDA Pro 7.6/plugins/hashdb.py", line 570, in OnBtnRefresh
        sorted_algorithms = sorted(algorithms, key = lambda algorithm: algorithm[0].lower())
    UnboundLocalError: local variable 'algorithms' referenced before assignment
    ERROR: HashDB API request failed: HTTPSConnectionPool(host='hashdb.openanalysis.net', port=443): Max retries exceeded with url: /hash (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1108)')))
    HashDB settings saved
    HashDB: HashDB terminated...
    Traceback (most recent call last):
      File "_ctypes/callbacks.c", line 237, in 'calling callback function'
      File "C:\Program Files\IDA Pro 7.6\python\3\ida_kernwin.py", line 9792, in helper_cb
        r = self.handler(button_code)
      File "C:/Program Files/IDA Pro 7.6/plugins/hashdb.py", line 570, in OnBtnRefresh
        sorted_algorithms = sorted(algorithms, key = lambda algorithm: algorithm[0].lower())
    UnboundLocalError: local variable 'algorithms' referenced before assignment
    HashDB configuration cancelled!
    
    opened by orchechik 2
  • Modified the `add_enums` to avoid collisions

    Modified the `add_enums` to avoid collisions

    • add_enums now takes one additional argument which has the default value of hdb,
    • if the suffix isn't empty it's appended to the member_name.

    fixes #23

    ida-plugin bugfix 
    opened by anthonyprintup 2
  • Fixed compatibility with Python 3.6, bumped version

    Fixed compatibility with Python 3.6, bumped version

    Previously we used features from PEP 604, which means our minimum version support was Py 3.10. I've replaced all of the instances of union types with typing.Union, and checked the the language support with the vermin module. At the moment the minimum required version is Py 3.6 (because of fstrings).

    opened by anthonyprintup 2
  • Added support for 64-bit enum values

    Added support for 64-bit enum values

    This PR is in reference to issue #13. For detailed information please read the commit messages.

    Known bugs:

    • 32-bit IDA can't add 64-bit hash values to enums (ida_enum.add_enum_member is broken, I'll contact Hex-Rays about it).
    opened by anthonyprintup 2
  • Added support for dynamic IAT hash scanning for 64-bit hashes

    Added support for dynamic IAT hash scanning for 64-bit hashes

    Previously, when scanning for hashed dynamic imports we would check if the database is 32 or 64-bit, which could break hash resolution if the binary was 64-bit, but the hash was 32-bit (see OALabs/hashdb-ida#5).

    To fix this issue I've added/removed/changed the following:

    • The plugin now has a new constant (HASHDB_ALGORITHM_SIZE) to keep track of the size of the hashing algorithm,
    • the hashdb_settings_t and hunt_result_form_t dialogues now show the size of the hashing algorithm in bits;
    • Removed code duplicates when setting the HASHDB_ALGORITHM to avoid errors in the future - set_algorithm contains all the necessary checks,
    • added a check for an edge-case where we would print an empty string in the output window if the API returned an empty string (see an example here) - this would break code generation (enums, etc.),
    • added a few TODOs regarding error handling and Python 3 syntax usage due to multiple uncertainties (@herrcore please resolve).

    Unfortunately at the moment the plugin does a second API request when executing hunt_algorithm, as the API only responds with the algorithm's name (perhaps we can fix this) - we would also need the size and parse it using the added determine_algorithm_size helper function.

    The plugin has been tested with the following samples (provided by @herrcore):

    • 32-bit: 1, 2, 3
    • 64-bit: 1, a private sample to test 64-bit hashes when scanning for a hashed IAT.
    opened by anthonyprintup 2
  • add binaries for unit-testing

    add binaries for unit-testing

    Create a set of manual (ugh) tests that developers can use to test their changes. Since these tests have to be manual we should try to standardize the steps as best as possible.

    Example.

    • Download binary (Malshare link)
    • Open binary in IDA
      • ✅ Did plugin load properly?
    • In disassembly window jump to address xxxxxxx and add immediate value xxxxx as XOR key
      • ✅ Was the XOR key correctly set?
    • In disassembly window jump to address xxxxxxx and right click on immediate value xxxxxx and run Hash Hunt Algorithm
      • ✅ Was the algorithm xxxxx correctly returned from HashDB?
    • In disassembly window jump to address xxxxxxx and select bytes from address xxxxxxx to xxxxxxx
    • Right click on selection and choose IAT Scan
      • ✅ Was each DWORD in the selected range correctly identified in HashDB and converted to an enum?
    enhancement ida-plugin 
    opened by herrcore 2
  • handle negative hashes

    handle negative hashes

    Currently we use get_highlight to pull the selected constant from IDA but this is an issue when the constant is negative. The negative sign is not highlighted in IDA so we don't grab it and misinterpret the constant as positive.

    Screen Shot 2021-09-26 at 9 40 22 PM

    This can be handled elegantly in the disassembly by checking the operands at the selected address for both negative and positive values of the selected constant to determine which one to use... however this doesn't really work when the value is selected in the pseudocode window.

    TLDR Issue When we use get_highlight in the pseudocode window to select a constant how can we match that constant with the actual value in the IDA microcode/pseudocode representation? This is my lack of understanding of how to manipulate the pseudocode from python... example code would be much appreciated 🙏

    enhancement ida-plugin 
    opened by herrcore 4
  • Automatically convert constant into enum in IDA pseudocode view

    Automatically convert constant into enum in IDA pseudocode view

    I'm not sure how to use ida python to convert a constant to an enum in the IDA pseudocode view... easy enough to do in the disassembly view but I'm not sure how to find the constant in the decompiled code?

    Help appreciated 🙏

    enhancement ida-plugin 
    opened by herrcore 2
Owner
OALabs
OALabs
Finite Volume simulation of the Raleigh-Taylor Instability

finitevolume2-python Finite Volume simulation of the Raleigh-Taylor Instability Create Your Own Finite Volume Fluid Simulation (With Python): Part 2 B

Philip Mocz 12 Sep 01, 2022
pwncat module that automatically exploits CVE-2021-4034 (pwnkit)

pwncat_pwnkit Introduction The purpose of this module is to attempt to exploit CVE-2021-4034 (pwnkit) on a target when using pwncat. There is no need

Dana Epp 33 Jul 01, 2022
集成crawlergo、xray、dirsearch、nmap等工具的src漏洞挖掘工具,使用docker封装运行;

tools下有几个工具,所以项目文件比较大,如果下载总是中断的话建议拆开下载各个项目然后直接拷贝dockefile和recon.py即可 0x01 hscan介绍 hscan是什么 hscan是一款旨在使用一条命令替代渗透前的多条扫描命令,通过集成crawlergo扫描和xray扫描、dirsear

102 Jan 04, 2023
A python script written for lazy people to hack their school systen ;D

F-ck-the-system A python script written for lazy people to hack their school systen ;D Python voice notes writer This is a python script to record you

Sachit 2 Jan 09, 2022
This script allows you to make a onion host instantly.

Installation It only works in Debian based Linux distros. Clone the repo: git clone https://github.com/0xStevenson/Auto-Tor-Host.git Go to the direct

Steven 4 Feb 22, 2022
This python script will automate the testing for the Log4J vulnerability for HTTP and HTTPS connections.

Log4J-Huntress-Automate-Script This python script will automate the testing for the Log4J vulnerability for HTTP and HTTPS connections. Pre-Requisits

1 Dec 16, 2021
An experimental script to perform bulk parsing of arbitrary file features with YARA and console logging.

RonnieColemanYARAParser This script is named after Ronnie Coleman, and peforms bulk lifts on arbitary file features using YARA console logging. Requir

Steve 20 Dec 13, 2022
Seamless deployment and management of cybersecurity solutions 🏗️

Description 🖼️ Background 👴🏼 Vision 📜 Concepts 💬 Solutions' Lifecycle. Operations ⭕ Functionalities 🚀 Supported Cybersecurity Solutions 📦 Insta

MutableSecurity 36 Nov 10, 2022
Simple script for looping a Denial Of Service (DoS) attack over one single mac address in range

Bluetooth Simple Denial Of Service (DoS) Legal Note This project is made only for educational purposes and for helping in Proofs of Concept. The autho

1 Jan 09, 2022
Ini membuat tema berbasis bendera Indonesia with Python + Linux.py

tema Ubah Tema Termux Menjadi Linux Ubah Font Termux Jadi Linux dibuat oleh wahyudioputra INSTALL pkg update && pkg upgrade pkg install python pkg ins

wahyudioputra 2 Nov 30, 2021
Learning to compose soft prompts for compositional zero-shot learning.

Compositional Soft Prompting (CSP) Compositional soft prompting (CSP), a parameter-efficient learning technique to improve the zero-shot compositional

Bats Research 32 Jan 02, 2023
INFO 3350/6350, Spring 2022, Cornell

Information Science 3350/6350 Text mining for history and literature Staff and sections Instructor: Matthew Wilkens Graduate TAs: Federica Bologna, Ro

Wilkens Teaching 6 Feb 21, 2022
Exploiting CVE-2021-42278 and CVE-2021-42287

noPac Exploiting CVE-2021-42278 and CVE-2021-42287 原项目noPac在实现上可能有点问题,导致在本地没有打通,于是参考sam-the-admin项目进行修改。 使用 pip3 install -r requirements.txt # GetShel

W4ter 2 Jun 23, 2022
A fully automated, accurate, and extensive scanner for finding vulnerable log4j hosts

log4j-scan A fully automated, accurate, and extensive scanner for finding vulnerable log4j hosts Features Support for lists of URLs. Fuzzing for more

Duc Linh Nguyen 4 Aug 08, 2022
Operational information regarding the vulnerability in the Log4j logging library.

Log4j Vulnerability (CVE-2021-44228) This repo contains operational information regarding the vulnerability in the Log4j logging library (CVE-2021-442

Nationaal Cyber Security Centrum (NCSC-NL) 1.9k Dec 26, 2022
AmiEviL - This program uses the Virus Total API to determine if your suspicious file is malicious or not

AmiEviL - This program uses the Virus Total API to determine if your suspicious file is malicious or not. The program requests the hash of the file and outputs information (if any). This version will

Kirk 1 Jan 03, 2022
Fuzzercorn - Bring libfuzzer to Unicorn

Fuzzercorn libfuzzer bindings for Unicorn. API // The main entry point of the fu

lazymio 23 Nov 17, 2022
Python tool for exploiting CVE-2021-35616

OracleOTM Python tool for exploiting CVE-2021-35616 The script works in modules, which I implemented in the following order: ► Username enumeration ►

11 Dec 06, 2022
A small Python Script To get all levels of subdomains from a list

getlevels A small Python Script To get all levels of subdomains Easily get 1st level, 2nd level, 3rd level, 4th level .... nth level subdomains Usag

9 Feb 15, 2022
Python implementation of the diceware password generating algorithm.

Diceware Password Generator - Generate High Entropy Passwords Please Note - This Program Do Not Store Passwords In Any Form And All The Passwords Are

Sameera Madushan 35 Dec 25, 2022