GDB python tool to pretty print and debug c++ xtensor containers

Overview

gdb_xt2np

GDB python tool to pretty print, examine, and debug c++ Xtensor containers. Xtensor is a c++ library for scientific computing using multidimensional arrays while providing a very python numpy-esque interface. Using the python GDB interface, this tool converts an xtensor to a numpy array and saves it to the GDB python interactive session allowing one to examine the xtensor array using matplotlib, printing numpy slices, etc..

Here is a screenshot of the gdb_xt2np helper in action. During a debugging session using eclipse IDE, in c++, I use xtensor to create a 2-d (100x200) array filled with uniformly random distributed integers. At a gdb breakpoint, I loaded the gdb_xt2np.py helper script.

Execute the gdb convenience function enabled by gdb_xt2np.py to examine the contents of the variable 'ar2' from the source code. (gdb) p $xt2np("ar2")

The xt2np function loads the xtensor container into a numpy array, then uses the native python numpy pretty printing to show a summary of the array contents, and then returns showing properties of the xtensor varible. In addition, to printing the 'ar2' variable summary, the variable is now available in the python interactive session within gdb. I use matplotlib.imshow to display the contents of the 'ar2' variable.

Eclipse debugging session with the gdb_xt2np helper

Version 0.1

FAQ

  • For modest sized arrays gdb complains about a max-size-value limit. Within gdb, set max-size-value unlimited will remove this warning, but beware you are not trying to bring too large of an array to numpy. Your system needs memory for holding multiple copies

AUTHORS

Christopher J. Burke. Inspiration for this came from several blogs and posts.

Dependencies & Setup

Well I'm not going to sugar coat it. It's a pain and I am not thrilled over all these hoops I needed to get the dependencies to work. The main problem is that I use a CentOS 7 linux machine that I don't have root access. I also am using gdb with eclipse IDE. I point out specific steps for my setup, and you will need to tailor them for your system, debugging IDE, etc..

CentOS 7 comes with ancient gcc 4.8.5 compiler. For c++17 standards (for xtensor) and more recent gdb one needs at least gcc 8+. It seems that in order to upgrade gcc without breaking things on CentOS, one needs to use 'Software Collection' devtoolset. Read about it here and the particular one they installed devtoolset-8 here. At a terminal, one invokes scl enable devtoolset-8 bash in order to enable the gcc 8+ tool chain. However, even starting eclipse IDE from a bash shell with devtoolset-8 enabled, the eclipse IDE was still using the gcc 4 compiler and old gdb. The only way I found to have eclipse adopt the gcc 8+ tool chain is to set a command alias in my .bashrc

alias eclipse='scl enable devtoolset-8 /eta-earth/cjburke/local/eclipse/eclipse

Starting eclipse IDE in this manner got the gcc 8+ tool chain available from within eclipse. Within IDE gdb session (gdb) show version reports 8.2 in my setup. Also within gdb you can show what python version is being used

(gdb) py
>import sys
>print(sys.version)
>end
2.7.5 (default, Nov 16 2020, 22:23:17) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]

NOTE: The version of python in gdb has nothing to do with any python you may have on your system. If you don't compile gdb yourself, but use gdb from a package manager, then gdb uses a python library version. In other words, the python used by gdb is self contained within gdb itself and the python version used was decided when gdb was compiled by the package managers. One can look in the gdb binary for the python symbols ldd $(which gdb) | grep python yields libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0 (0x00007f8026f60000) on my setup. Unfortunately, the version compiled in as part of the devtoolset-8 is a python 2 version. The question arises then how do I install numpy, matplotlib, and any other python package for use in the gdb version of python? The normal solution is install packages in the python path that the gdb python uses. One can use

(gdb) py
>import sys
>print(sys.path)
>end

to show the path where the gdb python version expects to find site-packages. Unfortunately, I don't have write permission to the location it was using as part of the devtools-8 install. My workaround was to use my preferred anaconda python manager and its enviroment support. I created a python 2.7 environment called py27 conda create -n py27 python=2.7. Then activate the py27 environment and I installed numpy and matplotlib which grabs the appropriate python 2.7 versions of these packages. conda activate py27 conda install numpy conda install matplotlib. Unfortunately, it was not enough to call eclipse IDE after activating the py27 environment. I needed to resort to setting shell environment variable $PYTHONPATH in the eclipse alias. I set the $GDBPYDIR to the head directory of the py27 files setup by the conda environment.

export GDBPYDIR=/eta-earth/cjburke/local/.conda/envs/py27
alias eclipse='export PYTHONPATH=${PYTHONPATH}:${GDBPYDIR}/lib/python2.7/site-packages; export LD_LIBRARY_PATH=${GDBPYDIR}/lib:${LD_LIBRARY_PATH}; scl enable devtoolset-8 /eta-earth/cjburke/local/eclipse/eclipse'

It was also necessary to add the python lib directory to LD_LIBRARY_PATH because there was a collision for a library that matplotlib needs in the system and the one that it needs from the environment.

Whew.. kinda sucky. I also haven't figured out why my ~/.gdbinit file is not working to automatically source the gdb_xt2np.py file at startup, so I have manually source it within eclipse IDE gdb version.

TODOs

  • Add support for memory ordering (i.e., column or row ordering). xt2np doesn't check the ordering of the xtensor container or set ordering when creating the numpy array
  • xt2np sets a python global variable with the same name as the variable. There is probably a safer way to do this and have the variable be available in the gdb python interpreter.
  • How to get working on a mac or windows machine and other Gui IDEs that use gdb.
Owner
Christopher Burke
Christopher Burke
Motion Detection Squid Game with OpenCV Python

*Motion Detection Squid Game with OpenCV Python i am newbie in python. In this project I made a simple game to follow the trend about the red light gr

Nayan 17 Nov 22, 2022
Scale-aware Automatic Augmentation for Object Detection (CVPR 2021)

SA-AutoAug Scale-aware Automatic Augmentation for Object Detection Yukang Chen, Yanwei Li, Tao Kong, Lu Qi, Ruihang Chu, Lei Li, Jiaya Jia [Paper] [Bi

Jia Research Lab 182 Dec 29, 2022
Handwritten_Text_Recognition

Deep Learning framework for Line-level Handwritten Text Recognition Short presentation of our project Introduction Installation 2.a Install conda envi

24 Jul 15, 2022
This project modify tensorflow object detection api code to predict oriented bounding boxes. It can be used for scene text detection.

This is an oriented object detector based on tensorflow object detection API. Most of the code is not changed except for those related to the need of

Dafang He 30 Oct 22, 2022
A novel region proposal network for more general object detection ( including scene text detection ).

DeRPN: Taking a further step toward more general object detection DeRPN is a novel region proposal network which concentrates on improving the adaptiv

Deep Learning and Vision Computing Lab, SCUT 151 Dec 12, 2022
Apply different text recognition services to images of handwritten documents.

Handprint The Handwritten Page Recognition Test is a command-line program that invokes HTR (handwritten text recognition) services on images of docume

Caltech Library 117 Jan 02, 2023
A document scanner application for laptops/desktops developed using python, Tkinter and OpenCV.

DcoumentScanner A document scanner application for laptops/desktops developed using python, Tkinter and OpenCV. Directly install the .exe file to inst

Harsh Vardhan Singh 1 Oct 29, 2021
A python program to block out your face

Readme This is a small program I threw together in about 6 hours to block out your face. It probably doesn't work very well, so be warned. By default,

1 Oct 17, 2021
ocroseg - This is a deep learning model for page layout analysis / segmentation.

ocroseg This is a deep learning model for page layout analysis / segmentation. There are many different ways in which you can train and run it, but by

NVIDIA Research Projects 71 Dec 06, 2022
Maze generator and solver with python

Procedural-Maze-Generator-Algorithms Check out my youtube channel : Auctux Ressources Thanks to Jamis Buck Book : Mazes for programmers Requirements P

Joseph 19 Dec 07, 2022
Fine tuning keras-ocr python package with custom synthetic dataset from scratch

OCR-Pipeline-with-Keras The keras-ocr package generally consists of two parts: a Detector and a Recognizer: Detector is responsible for creating bound

Eugene 1 Jan 05, 2022
Library used to deskew a scanned document

Deskew //Note: Skew is measured in degrees. Deskewing is a process whereby skew is removed by rotating an image by the same amount as its skew but in

Stéphane Brunner 273 Jan 06, 2023
OCRmyPDF adds an OCR text layer to scanned PDF files, allowing them to be searched

OCRmyPDF adds an OCR text layer to scanned PDF files, allowing them to be searched or copy-pasted. ocrmypdf # it's a scriptable c

jbarlow83 7.9k Jan 03, 2023
A python screen recorder for low-end computers, provides high quality video output.

RecorderX - v1.0 A screen recorder made in Python with the help of OpenCv, it has ability to record your screen in high quality. No matter what your P

Priyanshu Jindal 4 Nov 10, 2021
In this project we will be using the live feed coming from the webcam to create a virtual mouse with complete functionalities.

Virtual Mouse Using OpenCV In this project we will be using the live feed coming from the webcam to create a virtual mouse using hand tracking. Projec

Hassan Shahzad 8 Dec 20, 2022
scene-linear test images

Scene-Referred Image Collection A collection of OpenEXR Scene-Referred images, encoded as max 2048px width, DWAA 80 compression. All exrs are encoded

Gralk Klorggson 7 Aug 25, 2022
Generate text images for training deep learning ocr model

New version release:https://github.com/oh-my-ocr/text_renderer Text Renderer Generate text images for training deep learning OCR model (e.g. CRNN). Su

Qing 1.2k Jan 04, 2023
A curated list of promising OCR resources

Call for contributor(paper summary,dataset generation,algorithm implementation and any other useful resources) awesome-ocr A curated list of promising

wanghaisheng 1.6k Jan 04, 2023
This repository contains codes on how to handle mouse event using OpenCV

Handling-Mouse-Click-Events-Using-OpenCV This repository contains codes on how t

Happy N. Monday 3 Feb 15, 2022
Lightning Fast Language Prediction 🚀

whatthelang Lightning Fast Language Prediction 🚀 Dependencies The dependencies can be installed using the requirements.txt file: $ pip install -r req

Indix 152 Oct 16, 2022