Python codecs extension featuring CLI tools for encoding/decoding anything

Overview

CodExt Tweet

Encode/decode anything.

PyPi Read The Docs Build Status Coverage Status Python Versions Requirements Status Known Vulnerabilities DOI License

This library extends the native codecs library (namely for adding new custom encodings and character mappings) and provides a myriad of new encodings (static or parametrized, like rot or xor), hence its named combining CODecs EXTension.

$ pip install codext
Want to contribute a new codec ? Want to contribute a new macro ?
Check the documentation first
Then PR your new codec
PR your updated version of macros.json

πŸ” Demonstrations

Using CodExt from the command line

Using base tools from the command line

Using the debase command line tool

πŸ’» Usage (main CLI tool) Tweet on codext

$ codext -i test.txt encode dna-1
GTGAGCGGGTATGTGA

$ echo -en "test" | codext encode morse
- . ... -

$ echo -en "test" | codext encode braille
β žβ ‘β Žβ ž

$ echo -en "test" | codext encode base100
πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«

Chaining codecs

$ echo -en "Test string" | codext encode reverse
gnirts tseT

$ echo -en "Test string" | codext encode reverse morse
--. -. .. .-. - ... / - ... . -

$ echo -en "Test string" | codext encode reverse morse dna-2
AGTCAGTCAGTGAGAAAGTCAGTGAGAAAGTGAGTGAGAAAGTGAGTCAGTGAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTTAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTGAGAAAGTC

$ echo -en "Test string" | codext encode reverse morse dna-2 octal
101107124103101107124103101107124107101107101101101107124103101107124107101107101101101107124107101107124107101107101101101107124107101107124103101107124107101107101101101107124103101107101101101107124107101107124107101107124107101107101101101107124124101107101101101107124103101107101101101107124107101107124107101107124107101107101101101107124107101107101101101107124103

$ echo -en "AGTCAGTCAGTGAGAAAGTCAGTGAGAAAGTGAGTGAGAAAGTGAGTCAGTGAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTTAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTGAGAAAGTC" | codext -d dna-2 morse reverse
test string

Using macros

$ codext add-macro my-encoding-chain gzip base63 lzma base64

$ codext list macros
example-macro, my-encoding-chain

$ echo -en "Test string" | codext encode my-encoding-chain
CQQFAF0AAIAAABuTgySPa7WaZC5Sunt6FS0ko71BdrYE8zHqg91qaqadZIR2LafUzpeYDBalvE///ug4AA==

$ codext remove-macro my-encoding-chain

$ codext list macros
example-macro

πŸ’» Usage (base CLI tool) Tweet on debase

$ echo "Test string !" | base122
*.7!ft9οΏ½-f9Γ‚

$ echo "Test string !" | base91 
"ONK;WDZM%Z%xE7L

$ echo "Test string !" | base91 | base85
B2P|BJ6A+nO(j|-cttl%

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr
QVx5tvgjvCAkXaMSuKoQmCnjeCV1YyyR3WErUUErFf

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | base58-flickr -d | base36 -d | base85 -d | base91 -d
Test string !
$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | debase -m 3
Test string !

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | debase -f Test
Test string !

πŸ’» Usage (Python)

Getting the list of available codecs:

>> codext.encode("this is a test", "base58-ripple") 'jo9rA2LQwr44eBmZK7E' >>> codext.encode("this is a test", "base58-url") 'JN91Wzkpa1nnDbLyjtf' >>> codecs.encode("this is a test", "base100") 'πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«' >>> codecs.decode("πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«", "base100") 'this is a test' >>> for i in range(8): print(codext.encode("this is a test", "dna-%d" % (i + 1))) GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA CTCACGGACGGCCTATAGAACGGCCTATAGAACGACAGAACTCACGCCCTATCTCA ACAGATTGATTAACGCGTGGATTAACGCGTGGATGAGTGGACAGATAAACGCACAG AGACATTCATTAAGCGCTCCATTAAGCGCTCCATCACTCCAGACATAAAGCGAGAC TCTGTAAGTAATTCGCGAGGTAATTCGCGAGGTAGTGAGGTCTGTATTTCGCTCTG TGTCTAACTAATTGCGCACCTAATTGCGCACCTACTCACCTGTCTATTTGCGTGTC GAGTGCCTGCCGGATATCTTGCCGGATATCTTGCTGTCTTGAGTGCGGGATAGAGT CACTCGGTCGGCCATATGTTCGGCCATATGTTCGTCTGTTCACTCGCCCATACACT >>> codext.decode("GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA", "dna-1") 'this is a test' >>> codecs.encode("this is a test", "morse") '- .... .. ... / .. ... / .- / - . ... -' >>> codecs.decode("- .... .. ... / .. ... / .- / - . ... -", "morse") 'this is a test' >>> with open("morse.txt", 'w', encoding="morse") as f: f.write("this is a test") 14 >>> with open("morse.txt",encoding="morse") as f: f.read() 'this is a test' >>> codext.decode(""" = X : x n r y Y y p a ` n | a o h ` g o z """, "whitespace-after+before") 'CSC{not_so_invisible}' >>> print(codext.encode("An example test string", "baudot-tape")) ***.** . * ***.* * . .* * .* . * ** .* ***.** ** .** .* * . * *. * .* * *. * *. * * . * *. * *. * ***. *.* ***.* * .* ">
>>> import codext

>>> codext.list()
['ascii85', 'base85', 'base100', 'base122', ..., 'tomtom', 'dna', 'html', 'markdown', 'url', 'resistor', 'sms', 'whitespace', 'whitespace-after-before']

>>> codext.encode("this is a test", "base58-bitcoin")
'jo91waLQA1NNeBmZKUF'

>>> codext.encode("this is a test", "base58-ripple")
'jo9rA2LQwr44eBmZK7E'

>>> codext.encode("this is a test", "base58-url")
'JN91Wzkpa1nnDbLyjtf'

>>> codecs.encode("this is a test", "base100")
'πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«'

>>> codecs.decode("πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«", "base100")
'this is a test'

>>> for i in range(8):
        print(codext.encode("this is a test", "dna-%d" % (i + 1)))
GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA
CTCACGGACGGCCTATAGAACGGCCTATAGAACGACAGAACTCACGCCCTATCTCA
ACAGATTGATTAACGCGTGGATTAACGCGTGGATGAGTGGACAGATAAACGCACAG
AGACATTCATTAAGCGCTCCATTAAGCGCTCCATCACTCCAGACATAAAGCGAGAC
TCTGTAAGTAATTCGCGAGGTAATTCGCGAGGTAGTGAGGTCTGTATTTCGCTCTG
TGTCTAACTAATTGCGCACCTAATTGCGCACCTACTCACCTGTCTATTTGCGTGTC
GAGTGCCTGCCGGATATCTTGCCGGATATCTTGCTGTCTTGAGTGCGGGATAGAGT
CACTCGGTCGGCCATATGTTCGGCCATATGTTCGTCTGTTCACTCGCCCATACACT
>>> codext.decode("GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA", "dna-1")
'this is a test'

>>> codecs.encode("this is a test", "morse")
'- .... .. ... / .. ... / .- / - . ... -'

>>> codecs.decode("- .... .. ... / .. ... / .- / - . ... -", "morse")
'this is a test'

>>> with open("morse.txt", 'w', encoding="morse") as f:
	f.write("this is a test")
14

>>> with open("morse.txt",encoding="morse") as f:
	f.read()
'this is a test'

>>> codext.decode("""
      =            
              X         
   :            
      x         
  n  
    r 
        y   
      Y            
              y        
     p    
         a       
 `          
            n            
          |    
  a          
o    
       h        
          `            
          g               
           o 
   z      """, "whitespace-after+before")
'CSC{not_so_invisible}'

>>> print(codext.encode("An example test string", "baudot-tape"))
***.**
   . *
***.* 
*  .  
   .* 
*  .* 
   . *
** .* 
***.**
** .**
   .* 
*  .  
* *. *
   .* 
* *.  
* *. *
*  .  
* *.  
* *. *
***.  
  *.* 
***.* 
 * .* 

πŸ“ƒ List of codecs

BaseXX

  • ascii85: classical ASCII85 (Python3 only)
  • baseN: see base encodings (incl base32, 36, 45, 58, 62, 63, 64, 91, 100, 122)
  • base-genericN: see base encodings ; supports any possible base

Binary

  • baudot: supports CCITT-1, CCITT-2, EU/FR, ITA1, ITA2, MTK-2 (Python3 only), UK, ...
  • baudot-spaced: variant of baudot ; groups of 5 bits are whitespace-separated
  • baudot-tape: variant of baudot ; outputs a string that looks like a perforated tape
  • bcd: Binary Coded Decimal, encodes characters from their (zero-left-padded) ordinals
  • bcd-extended0: variant of bcd ; encodes characters from their (zero-left-padded) ordinals using prefix bits 0000
  • bcd-extended1: variant of bcd ; encodes characters from their (zero-left-padded) ordinals using prefix bits 1111
  • excess3: uses Excess-3 (aka Stibitz code) binary encoding to convert characters from their ordinals
  • gray: aka reflected binary code
  • manchester: XORes each bit of the input with 01
  • manchester-inverted: variant of manchester ; XORes each bit of the input with 10
  • rotateN: rotates characters by the specified number of bits (N belongs to [1, 7] ; Python 3 only)

Common

  • a1z26: keeps words whitespace-separated and uses a custom character separator
  • cases: set of case-related encodings (including camel-, kebab-, lower-, pascal-, upper-, snake- and swap-case, slugify, capitalize, title)
  • dummy: set of simple encodings (including reverse and word-reverse)
  • octal: dummy octal conversion (converts to 3-digits groups)
  • octal-spaced: variant of octal ; dummy octal conversion, handling whitespace separators
  • ordinal: dummy character ordinals conversion (converts to 3-digits groups)
  • ordinal-spaced: variant of ordinal ; dummy character ordinals conversion, handling whitespace separators

Compression

  • gzip: standard Gzip compression/decompression
  • lz77: compresses the given data with the algorithm of Lempel and Ziv of 1977
  • lz78: compresses the given data with the algorithm of Lempel and Ziv of 1978
  • pkzip_deflate: standard Zip-deflate compression/decompression
  • pkzip_bzip2: standard BZip2 compression/decompression
  • pkzip_lzma: standard LZMA compression/decompression

Cryptography

  • affine: aka Affine Cipher
  • atbash: aka Atbash Cipher
  • bacon: aka Baconian Cipher
  • barbie-N: aka Barbie Typewriter (N belongs to [1, 4])
  • citrix: aka Citrix CTX1 passord encoding
  • rotN: aka Caesar cipher (N belongs to [1,25])
  • scytaleN: encrypts using the number of letters on the rod (N belongs to [1,[)
  • shiftN: shift ordinals (N belongs to [1,255])
  • xorN: XOR with a single byte (N belongs to [1,255])

Languages

  • braille: well-known braille language (Python 3 only)
  • ipsum: aka lorem ipsum
  • leetspeak: based on minimalistic elite speaking rules
  • morse: uses whitespace as a separator
  • navajo: only handles letters (not full words from the Navajo dictionary)
  • radio: aka NATO or radio phonetic alphabet
  • southpark: converts letters to Kenny's language from Southpark (whitespace is also handled)
  • southpark-icase: case insensitive variant of southpark
  • tomtom: similar to morse, using slashes and backslashes

Others

  • dna: implements the 8 rules of DNA sequences (N belongs to [1,8])
  • html: implements entities according to this reference
  • letter-indices: encodes consonants and/or vowels with their corresponding indices
  • markdown: unidirectional encoding from Markdown to HTML
  • url: aka URL encoding

Steganography

  • klopf: aka Klopf code ; Polybius square with trivial alphabetical distribution
  • resistor: aka resistor color codes
  • sms: also called T9 code ; uses "-" as a separator for encoding, "-" or "_" or whitespace for decoding
  • whitespace: replaces bits with whitespaces and tabs
  • whitespace_after_before: variant of whitespace ; encodes characters as new characters with whitespaces before and after according to an equation described in the codec name (e.g. "whitespace+2*after-3*before")

πŸ‘ Supporters

Stargazers repo roster for @dhondta/python-codext

Forkers repo roster for @dhondta/python-codext

Back to top

Comments
  • using Codext guess / Codext rank gives an error

    using Codext guess / Codext rank gives an error

    When I run it on linux and try to use "codext guess" or "codext rank" I get an error message saying:

    # echo "3yZe7d" | codext rank
    Traceback (most recent call last):
      File "/usr/local/bin/codext", line 8, in <module>
        sys.exit(main())
      File "/usr/local/lib/python3.9/dist-packages/codext/__init__.py", line 184, in main
        args.include, args.exclude = __format_list(args.include), __format_list(args.exclude, False)
    AttributeError: 'Namespace' object has no attribute 'include'
    
    opened by GitSunburn 1
  • UU decoding raises an exception in some cases

    UU decoding raises an exception in some cases

    # cat raw.txt 
    1Oh6axLwfecHErbVpRbtNj8t5JACsSQrofdnnMdQtBmoU8cQj6EyLcVRsQLz1MyWfXbqQDIc9wGyyBuH7uV95lBVpFGn3syGIw0IVLx8wJr4ABsIH9Q71hBH4AvIgljx7XnfjfmafahBNrPMDkK3dsJF0r41nzyMnOf7l36NcllAOgRLoB6Qh0APotZu6plYpkSiRCAkDKowOFm0mybKp336TAJe4JiDecD9hNbl5fBDLkGNYhmSkzOQzLBH1aPumW4o
    
    # codext -i raw.txt decode base62
    begin 666 <data>
    M'XL( /H^)V("__-)SBB-<O7*+#;,K8PL\/ H\C#UR2LP= H)[email protected]\*"[email protected]
    M\"T-\*@(#\]V\<A,*R^W\"@I,W9Q-,LU\8XR=$XM*TYR3PG,2G)+RO P#_'.
    ?"LRJS#)US0X(KPJH<[email protected](<TGR  #3A(K<90      
     
    end
    
    # codext -i raw.txt decode base62 | codext decode UU
    Traceback (most recent call last):
      File "/usr/lib/python3.8/encodings/uu_codec.py", line 58, in uu_decode
        data = binascii.a2b_uu(s)
    binascii.Error: Illegal char
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/jeane/.local/bin/codext", line 8, in <module>
        sys.exit(main())
      File "/home/jeane/.local/lib/python3.8/site-packages/codext/__init__.py", line 124, in main
        c = getattr(codecs, ["encode", "decode"][args.command == "decode"])(c, encoding, args.errors)
      File "/home/jeane/.local/lib/python3.8/site-packages/codext/__common__.py", line 691, in decode
        return lookup(encoding).decode(obj, errors)[0]
      File "/usr/lib/python3.8/encodings/uu_codec.py", line 62, in uu_decode
        data = binascii.a2b_uu(s[:nbytes])
    binascii.Error: Illegal char
    
    opened by RomainJennes 1
  • Implement Rail Fence Cipher

    Implement Rail Fence Cipher

    Tests pass.

    RegEx might need a fix : rail-5-3- is recognized as a valid codec (ending dash is too much). Only rail-5-3 or rail-5-3-up should be recognized. Couldn't manage to get that right.

    When encoding, there might be trailing whitespaces (which is normal), but users might forget one when copy/pasting. I left them invisible to ease the integration with other tools.

    opened by smarbal 0
  • Add tap/knock language

    Add tap/knock language

    Tap encoding and decoding is implemented and added to the documentation. Couldn't make it work with add_map() due to problems with letter and word separations (single space/double space). Did my own encoding/decoding functions instead.

    opened by smarbal 0
  • [Snyk] Security upgrade markdown2 from 2.3.10 to 2.4.0

    [Snyk] Security upgrade markdown2 from 2.3.10 to 2.4.0

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 768/1000
    Why? Proof of Concept exploit, Recently disclosed, Has a fix available, CVSS 7.5 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-MARKDOWN2-1063233 | markdown2:
    2.3.10 -> 2.4.0
    | No | Proof of Concept

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.


    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    πŸ›  Adjust project settings

    πŸ“š Read more about Snyk's upgrade and patch logic

    opened by dhondta 0
  • pip 22.3 / python 3.10 warning on install

    pip 22.3 / python 3.10 warning on install

    Looks like there's a warning on the installation method via pip:

    pip install codext         
    Collecting codext
      Downloading codext-1.14.0.tar.gz (116 kB)
         ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.2/116.2 kB 1.7 MB/s eta 0:00:00
      Preparing metadata (setup.py) ... done
    Requirement already satisfied: six in ./venv/lib/python3.10/site-packages (from codext) (1.16.0)
    Collecting markdown2>=2.4.0
      Downloading markdown2-2.4.6-py2.py3-none-any.whl (37 kB)
    Installing collected packages: markdown2, codext
      DEPRECATION: codext is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
      Running setup.py install for codext ... done
    Successfully installed codext-1.14.0 markdown2-2.4.6
    
    opened by mikeatlas 0
Releases(1.10.1)
Owner
Alex
I'm a security professional, passionate about programming (especially Python) and cyber security.
Alex
A simple script to make the operation of AltServer-Linux more easier with cli

A simple script to make the operation of AltServer-Linux more easier with cli

powen 23 Dec 13, 2022
An anime command-line system information tool written in python.

Animefetch - v0.0.3 An anime command-line system information tool written in python. Description Animefetch is an anime command-line system informatio

Thadeuks 6 Jun 17, 2022
CLI for SQLite Databases with auto-completion and syntax highlighting

litecli Docs A command-line client for SQLite databases that has auto-completion and syntax highlighting. Installation If you already know how to inst

dbcli 1.8k Dec 31, 2022
Python package with library and CLI tool for analyzing SeaFlow data

Seaflowpy A Python package for SeaFlow flow cytometer data. Table of Contents Install Read EVT/OPP/VCT Files Command-line Interface Configuration Inte

<a href=[email protected]"> 3 Nov 03, 2021
CLI utility to search and download torrents from major torrent sites

CLI Torrent Downloader About CLI Torrent Downloader provides convenient and quick way to search torrent magnet links (and to run associated torrent cl

x0r0x 86 Dec 19, 2022
Projeto Reverse Shell For Python

Use com sabedoria!!! Modo de uso: Linux (inclui Android e Mac): - apt-get update - apt install python3 (ou "python" apenas) - git clone https://github

1 Jan 03, 2022
Terminal epub reader with inline images

nuber Inspired by epy, nuber is an Epub terminal reader with inline images written with Rust and Python using Überzug. Features Display images in term

Moshe Sherman 73 Oct 12, 2022
Free and Open-Source Command Line tool for Text Replacement

Sniplet Free and Open Source Text Replacement Tool Description: Sniplet is a work in progress CLI tool which can do text replacement globally in Linux

Veeraraghavan Narasimhan 13 Nov 28, 2022
A simple yet powerful timer and time tracker from the command line.

Focus Phase Focus Phase (FP) is a simple yet powerful timer and time tracker. It is a command-line application written in Python and can be installed

Ammar Alyousfi 13 Jan 13, 2022
Command Line For Truecaller Written In Python

Truecaller-CLI Command Line Version For Truecaller Written In Python Never Login With A Number Over And Over Or It Will Be Banned Because Program Is S

Sandaru Ashen Fernando 16 Nov 08, 2022
Tmux Based Dropdown Dashboard For Python

sextans It's a private configuration and an ongoing experiment while I use Archlinux. A simple drop down dashboard based on tmux. It includes followin

秋葉 4 Dec 22, 2021
A simple automation script that logs into your kra account and files your taxes with one command

EASY_TAX A simple automation script that logs into your kra account and files your taxes with one command Currently works for Chrome users. Will creat

leon koech 13 Sep 23, 2021
A minimal todo list for your terminal.

todo A minimal todo list for your terminal. Installation Run the following command. pip install git+https://github.com/piero-vic/todo.git Usage todo

Piero Lescano 7 Aug 08, 2022
A CLI Application to detect plagiarism in Source Code Files.

Plag Description A CLI Application to detect plagiarism in Source Code Files. Features Compare source code files for plagiarism. Extract code features

default=dev 2 Nov 10, 2022
This is my fetch, with ascii arts from neofetch and internet

deadfetch This is my fetch, with ascii arts from neofetch and internet Installation First what you need its python Fedora sudo dnf install python3 sud

DedSec 8 Jan 20, 2022
frogtrade9000 - a command-line Rich client for the freqtrade REST API

frogtrade9000 - a command-line Rich client for the freqtrade REST API I found FreqUI too cumbersome and slow on my Raspberry Pi 400 when running multi

Robert Davey 79 Dec 02, 2022
GanTTY - Project planning from the terminal

GanTTY - Project planning from the terminal

Timeo Sam Pochin 161 Dec 26, 2022
A simple CLI based any Download Tool, that find files and let you stream or download thorugh WebTorrent CLI or Aria or any command tool

Privateer A simple CLI based any Download Tool, that find files and let you stream or download thorugh WebTorrent CLI or Aria or any command tool How

Shreyash Chavan 2 Apr 04, 2022
Yts-cli-streamer - A CLI movie streaming client which works on yts.mx API written in python

YTSP It is a CLI movie streaming client which works on yts.mx API written in pyt

1 Feb 05, 2022
Turdshovel is an interactive CLI tool that allows users to dump objects from .NET memory dumps

Turdshovel Description Turdshovel is an interactive CLI tool that allows users to dump objects from .NET memory dumps without having to fully understa

Leron Gray 41 Jul 27, 2022