Python codecs extension featuring CLI tools for encoding/decoding anything

Related tags

Command-line Toolspythonencodingcryptographydecoderdnamorsebase58codecbraillebase62ctfencodingsbaseleetspeakctf-toolsbase36encoder-decodercodecsbase100base45
Overview

CodExt Tweet

Encode/decode anything.

PyPi Read The Docs Build Status Coverage Status Python Versions Requirements Status Known Vulnerabilities DOI License

This library extends the native codecs library (namely for adding new custom encodings and character mappings) and provides a myriad of new encodings (static or parametrized, like rot or xor), hence its named combining CODecs EXTension.

$ pip install codext
Want to contribute a new codec ? Want to contribute a new macro ?
Check the documentation first
Then PR your new codec		 PR your updated version of macros.json

πŸ” Demonstrations

Using CodExt from the command line

Using base tools from the command line

Using the debase command line tool

πŸ’» Usage (main CLI tool) Tweet on codext

$ codext -i test.txt encode dna-1
GTGAGCGGGTATGTGA

$ echo -en "test" | codext encode morse
- . ... -

$ echo -en "test" | codext encode braille
β žβ ‘β Žβ ž

$ echo -en "test" | codext encode base100
πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«

Chaining codecs

$ echo -en "Test string" | codext encode reverse
gnirts tseT

$ echo -en "Test string" | codext encode reverse morse
--. -. .. .-. - ... / - ... . -

$ echo -en "Test string" | codext encode reverse morse dna-2
AGTCAGTCAGTGAGAAAGTCAGTGAGAAAGTGAGTGAGAAAGTGAGTCAGTGAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTTAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTGAGAAAGTC

$ echo -en "Test string" | codext encode reverse morse dna-2 octal
101107124103101107124103101107124107101107101101101107124103101107124107101107101101101107124107101107124107101107101101101107124107101107124103101107124107101107101101101107124103101107101101101107124107101107124107101107124107101107101101101107124124101107101101101107124103101107101101101107124107101107124107101107124107101107101101101107124107101107101101101107124103

$ echo -en "AGTCAGTCAGTGAGAAAGTCAGTGAGAAAGTGAGTGAGAAAGTGAGTCAGTGAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTTAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTGAGAAAGTC" | codext -d dna-2 morse reverse
test string

Using macros

$ codext add-macro my-encoding-chain gzip base63 lzma base64

$ codext list macros
example-macro, my-encoding-chain

$ echo -en "Test string" | codext encode my-encoding-chain
CQQFAF0AAIAAABuTgySPa7WaZC5Sunt6FS0ko71BdrYE8zHqg91qaqadZIR2LafUzpeYDBalvE///ug4AA==

$ codext remove-macro my-encoding-chain

$ codext list macros
example-macro

πŸ’» Usage (base CLI tool) Tweet on debase

$ echo "Test string !" | base122
*.7!ft9οΏ½-f9Γ‚

$ echo "Test string !" | base91 
"ONK;WDZM%Z%xE7L

$ echo "Test string !" | base91 | base85
B2P|BJ6A+nO(j|-cttl%

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr
QVx5tvgjvCAkXaMSuKoQmCnjeCV1YyyR3WErUUErFf

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | base58-flickr -d | base36 -d | base85 -d | base91 -d
Test string !

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | debase -m 3
Test string !

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | debase -f Test
Test string !

πŸ’» Usage (Python)

Getting the list of available codecs:

>> codext.encode("this is a test", "base58-ripple") 'jo9rA2LQwr44eBmZK7E' >>> codext.encode("this is a test", "base58-url") 'JN91Wzkpa1nnDbLyjtf' >>> codecs.encode("this is a test", "base100") 'πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«' >>> codecs.decode("πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«", "base100") 'this is a test' >>> for i in range(8): print(codext.encode("this is a test", "dna-%d" % (i + 1))) GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA CTCACGGACGGCCTATAGAACGGCCTATAGAACGACAGAACTCACGCCCTATCTCA ACAGATTGATTAACGCGTGGATTAACGCGTGGATGAGTGGACAGATAAACGCACAG AGACATTCATTAAGCGCTCCATTAAGCGCTCCATCACTCCAGACATAAAGCGAGAC TCTGTAAGTAATTCGCGAGGTAATTCGCGAGGTAGTGAGGTCTGTATTTCGCTCTG TGTCTAACTAATTGCGCACCTAATTGCGCACCTACTCACCTGTCTATTTGCGTGTC GAGTGCCTGCCGGATATCTTGCCGGATATCTTGCTGTCTTGAGTGCGGGATAGAGT CACTCGGTCGGCCATATGTTCGGCCATATGTTCGTCTGTTCACTCGCCCATACACT >>> codext.decode("GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA", "dna-1") 'this is a test' >>> codecs.encode("this is a test", "morse") '- .... .. ... / .. ... / .- / - . ... -' >>> codecs.decode("- .... .. ... / .. ... / .- / - . ... -", "morse") 'this is a test' >>> with open("morse.txt", 'w', encoding="morse") as f: f.write("this is a test") 14 >>> with open("morse.txt",encoding="morse") as f: f.read() 'this is a test' >>> codext.decode(""" = X : x n r y Y y p a ` n | a o h ` g o z """, "whitespace-after+before") 'CSC{not_so_invisible}' >>> print(codext.encode("An example test string", "baudot-tape")) ***.** . * ***.* * . .* * .* . * ** .* ***.** ** .** .* * . * *. * .* * *. * *. * * . * *. * *. * ***. *.* ***.* * .* "> 
>>> import codext

>>> codext.list()
['ascii85', 'base85', 'base100', 'base122', ..., 'tomtom', 'dna', 'html', 'markdown', 'url', 'resistor', 'sms', 'whitespace', 'whitespace-after-before']

>>> codext.encode("this is a test", "base58-bitcoin")
'jo91waLQA1NNeBmZKUF'

>>> codext.encode("this is a test", "base58-ripple")
'jo9rA2LQwr44eBmZK7E'

>>> codext.encode("this is a test", "base58-url")
'JN91Wzkpa1nnDbLyjtf'

>>> codecs.encode("this is a test", "base100")
'πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«'

>>> codecs.decode("πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«", "base100")
'this is a test'

>>> for i in range(8):
        print(codext.encode("this is a test", "dna-%d" % (i + 1)))
GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA
CTCACGGACGGCCTATAGAACGGCCTATAGAACGACAGAACTCACGCCCTATCTCA
ACAGATTGATTAACGCGTGGATTAACGCGTGGATGAGTGGACAGATAAACGCACAG
AGACATTCATTAAGCGCTCCATTAAGCGCTCCATCACTCCAGACATAAAGCGAGAC
TCTGTAAGTAATTCGCGAGGTAATTCGCGAGGTAGTGAGGTCTGTATTTCGCTCTG
TGTCTAACTAATTGCGCACCTAATTGCGCACCTACTCACCTGTCTATTTGCGTGTC
GAGTGCCTGCCGGATATCTTGCCGGATATCTTGCTGTCTTGAGTGCGGGATAGAGT
CACTCGGTCGGCCATATGTTCGGCCATATGTTCGTCTGTTCACTCGCCCATACACT
>>> codext.decode("GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA", "dna-1")
'this is a test'

>>> codecs.encode("this is a test", "morse")
'- .... .. ... / .. ... / .- / - . ... -'

>>> codecs.decode("- .... .. ... / .. ... / .- / - . ... -", "morse")
'this is a test'

>>> with open("morse.txt", 'w', encoding="morse") as f:
	f.write("this is a test")
14

>>> with open("morse.txt",encoding="morse") as f:
	f.read()
'this is a test'

>>> codext.decode("""
      =            
              X         
   :            
      x         
  n  
    r 
        y   
      Y            
              y        
     p    
         a       
 `          
            n            
          |    
  a          
o    
       h        
          `            
          g               
           o 
   z      """, "whitespace-after+before")
'CSC{not_so_invisible}'

>>> print(codext.encode("An example test string", "baudot-tape"))
***.**
   . *
***.* 
*  .  
   .* 
*  .* 
   . *
** .* 
***.**
** .**
   .* 
*  .  
* *. *
   .* 
* *.  
* *. *
*  .  
* *.  
* *. *
***.  
  *.* 
***.* 
 * .*

πŸ“ƒ List of codecs

BaseXX

  • ascii85: classical ASCII85 (Python3 only)
  • baseN: see base encodings (incl base32, 36, 45, 58, 62, 63, 64, 91, 100, 122)
  • base-genericN: see base encodings ; supports any possible base

Binary

  • baudot: supports CCITT-1, CCITT-2, EU/FR, ITA1, ITA2, MTK-2 (Python3 only), UK, ...
  • baudot-spaced: variant of baudot ; groups of 5 bits are whitespace-separated
  • baudot-tape: variant of baudot ; outputs a string that looks like a perforated tape
  • bcd: Binary Coded Decimal, encodes characters from their (zero-left-padded) ordinals
  • bcd-extended0: variant of bcd ; encodes characters from their (zero-left-padded) ordinals using prefix bits 0000
  • bcd-extended1: variant of bcd ; encodes characters from their (zero-left-padded) ordinals using prefix bits 1111
  • excess3: uses Excess-3 (aka Stibitz code) binary encoding to convert characters from their ordinals
  • gray: aka reflected binary code
  • manchester: XORes each bit of the input with 01
  • manchester-inverted: variant of manchester ; XORes each bit of the input with 10
  • rotateN: rotates characters by the specified number of bits (N belongs to [1, 7] ; Python 3 only)

Common

  • a1z26: keeps words whitespace-separated and uses a custom character separator
  • cases: set of case-related encodings (including camel-, kebab-, lower-, pascal-, upper-, snake- and swap-case, slugify, capitalize, title)
  • dummy: set of simple encodings (including reverse and word-reverse)
  • octal: dummy octal conversion (converts to 3-digits groups)
  • octal-spaced: variant of octal ; dummy octal conversion, handling whitespace separators
  • ordinal: dummy character ordinals conversion (converts to 3-digits groups)
  • ordinal-spaced: variant of ordinal ; dummy character ordinals conversion, handling whitespace separators

Compression

  • gzip: standard Gzip compression/decompression
  • lz77: compresses the given data with the algorithm of Lempel and Ziv of 1977
  • lz78: compresses the given data with the algorithm of Lempel and Ziv of 1978
  • pkzip_deflate: standard Zip-deflate compression/decompression
  • pkzip_bzip2: standard BZip2 compression/decompression
  • pkzip_lzma: standard LZMA compression/decompression

Cryptography

  • affine: aka Affine Cipher
  • atbash: aka Atbash Cipher
  • bacon: aka Baconian Cipher
  • barbie-N: aka Barbie Typewriter (N belongs to [1, 4])
  • citrix: aka Citrix CTX1 passord encoding
  • rotN: aka Caesar cipher (N belongs to [1,25])
  • scytaleN: encrypts using the number of letters on the rod (N belongs to [1,[)
  • shiftN: shift ordinals (N belongs to [1,255])
  • xorN: XOR with a single byte (N belongs to [1,255])

Languages

  • braille: well-known braille language (Python 3 only)
  • ipsum: aka lorem ipsum
  • leetspeak: based on minimalistic elite speaking rules
  • morse: uses whitespace as a separator
  • navajo: only handles letters (not full words from the Navajo dictionary)
  • radio: aka NATO or radio phonetic alphabet
  • southpark: converts letters to Kenny's language from Southpark (whitespace is also handled)
  • southpark-icase: case insensitive variant of southpark
  • tomtom: similar to morse, using slashes and backslashes

Others

  • dna: implements the 8 rules of DNA sequences (N belongs to [1,8])
  • html: implements entities according to this reference
  • letter-indices: encodes consonants and/or vowels with their corresponding indices
  • markdown: unidirectional encoding from Markdown to HTML
  • url: aka URL encoding

Steganography

  • klopf: aka Klopf code ; Polybius square with trivial alphabetical distribution
  • resistor: aka resistor color codes
  • sms: also called T9 code ; uses "-" as a separator for encoding, "-" or "_" or whitespace for decoding
  • whitespace: replaces bits with whitespaces and tabs
  • whitespace_after_before: variant of whitespace ; encodes characters as new characters with whitespaces before and after according to an equation described in the codec name (e.g. "whitespace+2*after-3*before")

πŸ‘ Supporters

Stargazers repo roster for @dhondta/python-codext

Forkers repo roster for @dhondta/python-codext

Back to top

Comments
  • using Codext guess / Codext rank gives an error

    using Codext guess / Codext rank gives an error

    When I run it on linux and try to use "codext guess" or "codext rank" I get an error message saying:

    # echo "3yZe7d" | codext rank
Traceback (most recent call last):
  File "/usr/local/bin/codext", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/dist-packages/codext/__init__.py", line 184, in main
    args.include, args.exclude = __format_list(args.include), __format_list(args.exclude, False)
AttributeError: 'Namespace' object has no attribute 'include'
    opened by GitSunburn 1
  • UU decoding raises an exception in some cases

    UU decoding raises an exception in some cases 

    # cat raw.txt 
1Oh6axLwfecHErbVpRbtNj8t5JACsSQrofdnnMdQtBmoU8cQj6EyLcVRsQLz1MyWfXbqQDIc9wGyyBuH7uV95lBVpFGn3syGIw0IVLx8wJr4ABsIH9Q71hBH4AvIgljx7XnfjfmafahBNrPMDkK3dsJF0r41nzyMnOf7l36NcllAOgRLoB6Qh0APotZu6plYpkSiRCAkDKowOFm0mybKp336TAJe4JiDecD9hNbl5fBDLkGNYhmSkzOQzLBH1aPumW4o

# codext -i raw.txt decode base62
begin 666 <data>
M'XL( /H^)V("__-)SBB-<O7*+#;,K8PL\/ H\C#UR2LP= H)[email protected]\*"[email protected]
M\"T-\*@(#\]V\<A,*R^W\"@I,W9Q-,LU\8XR=$XM*TYR3PG,2G)+RO P#_'.
?"LRJS#)US0X(KPJH<[email protected](<TGR  #3A(K<90      
 
end

# codext -i raw.txt decode base62 | codext decode UU
Traceback (most recent call last):
  File "/usr/lib/python3.8/encodings/uu_codec.py", line 58, in uu_decode
    data = binascii.a2b_uu(s)
binascii.Error: Illegal char

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jeane/.local/bin/codext", line 8, in <module>
    sys.exit(main())
  File "/home/jeane/.local/lib/python3.8/site-packages/codext/__init__.py", line 124, in main
    c = getattr(codecs, ["encode", "decode"][args.command == "decode"])(c, encoding, args.errors)
  File "/home/jeane/.local/lib/python3.8/site-packages/codext/__common__.py", line 691, in decode
    return lookup(encoding).decode(obj, errors)[0]
  File "/usr/lib/python3.8/encodings/uu_codec.py", line 62, in uu_decode
    data = binascii.a2b_uu(s[:nbytes])
binascii.Error: Illegal char
    opened by RomainJennes 1
  • Implement Rail Fence Cipher

    Implement Rail Fence Cipher

    Tests pass.

    RegEx might need a fix : rail-5-3- is recognized as a valid codec (ending dash is too much). Only rail-5-3 or rail-5-3-up should be recognized. Couldn't manage to get that right.

    When encoding, there might be trailing whitespaces (which is normal), but users might forget one when copy/pasting. I left them invisible to ease the integration with other tools.

    opened by smarbal 0
  • Add tap/knock language

    Add tap/knock language

    Tap encoding and decoding is implemented and added to the documentation. Couldn't make it work with add_map() due to problems with letter and word separations (single space/double space). Did my own encoding/decoding functions instead.

    opened by smarbal 0
  • [Snyk] Security upgrade markdown2 from 2.3.10 to 2.4.0

    [Snyk] Security upgrade markdown2 from 2.3.10 to 2.4.0

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 768/1000
    Why? Proof of Concept exploit, Recently disclosed, Has a fix available, CVSS 7.5 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-MARKDOWN2-1063233 | markdown2:
    2.3.10 -> 2.4.0
    | No | Proof of Concept

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    πŸ›  Adjust project settings

    πŸ“š Read more about Snyk's upgrade and patch logic

    opened by dhondta 0
  • pip 22.3 / python 3.10 warning on install

    pip 22.3 / python 3.10 warning on install

    Looks like there's a warning on the installation method via pip:

    pip install codext         
Collecting codext
  Downloading codext-1.14.0.tar.gz (116 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.2/116.2 kB 1.7 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Requirement already satisfied: six in ./venv/lib/python3.10/site-packages (from codext) (1.16.0)
Collecting markdown2>=2.4.0
  Downloading markdown2-2.4.6-py2.py3-none-any.whl (37 kB)
Installing collected packages: markdown2, codext
  DEPRECATION: codext is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
  Running setup.py install for codext ... done
Successfully installed codext-1.14.0 markdown2-2.4.6
    opened by mikeatlas 0
Releases(1.10.1)
Owner
Alex
I'm a security professional, passionate about programming (especially Python) and cyber security.
Alex
GitHub Repository
The project help you to quickly build layouts in terminal，cross-platform

The project help you to quickly build layouts in terminal，cross-platform

gojuukaze 133 Nov 30, 2022
stonky is a simple command line dashboard for monitoring stocks.

stonky is a simple command line dashboard for monitoring stocks.

Jessy Williams 228 Dec 14, 2022
πŸƒ Python3 Solutions of All Problems in GCJ 2022 (In Progress)

GoogleCodeJam 2022 Python3 solutions of Google Code Jam 2022. Solution begins with * means it will get TLE in the largest data set. Total computation

kamyu 12 Dec 20, 2022
Command line client for Audience Insights

Dynamics 365 Audience Insights CLI The AuI CLI is a command line tool for Dynamics 365 Audience Insights. It is based on the customerinsights Python l

Microsoft 8 Jan 09, 2023
Chat with Rem in Terminal!

Chat with Rem in Terminal!

bariscodefx 1 Dec 19, 2021
Python API and CLI for the ikea IDΓ…SEN desk.

idasen This is a heavily modified fork of rhyst/idasen-controller. The IDΓ…SEN is an electric sitting standing desk with a Linak controller sold by ike

Alex 79 Dec 14, 2022
A Python package for Misty II development

Misty2py Misty2py is a Python 3 package for Misty II development using Misty's REST API. Read the full documentation here! Installation Poetry To inst

Chris Scarred 1 Mar 07, 2022
A very simple and lightweight ToDo app using python that can be used from the command line

A very simple and lightweight ToDo app using python that can be used from the command line

Nilesh Sengupta 2 Jul 20, 2022
Plumbum: Shell Combinators

Plumbum: Shell Combinators Ever wished the compactness of shell scripts be put into a real programming language? Say hello to Plumbum Shell Combinator

Tomer Filiba 2.5k Dec 28, 2022
QueraToCSV is a simple python CLI project to convert the Quera results file into CSV files.

Quera is an Iranian Learning management system (LMS) that has an online judge for programming languages. Some Iranian universities use it to automate the evaluation of programming assignments.

Amirmahdi Namjoo 16 Nov 11, 2022
A web shell client written in python.

Webshell client A webshell client written in python. Only works well for linux for the time being. Why? Because there are too many heavy webshells. So

tchar 1 Dec 07, 2021
A simple and easy-to-use CLI parse tool.

A simple and easy-to-use CLI parse tool.

AbsentM 1 Mar 04, 2022
gcptree - Like the unix tree command but for GCP Org Heirarchy

gcptree Like the unix tree command but for GCP Org Heirarchy. For a note on coloring, the org node is green, folders and blue, and projects that are n

Ryan Canty 25 Sep 06, 2022
Synchronization tool for external devices which does not support time stamps, e.g. over MTP.

MTP-Sync Tool to synchronize data to a slow device, e.g. a smartphone which is connected over MTP. A state file (.mtp_sync_state.json) is created in t

2 Jul 22, 2022
moviepy-cli: Command line interface for MoviePy.

Moviepy-cli is designed to apply several video editing in a single command with Moviepy as an alternative to Video-cli.

Kentaro Wada 23 Jun 29, 2022
A dashboard for your Terminal written in the Python 3 language,

termDash is a handy little program, written in the Python 3 language, and is a small little dashboard for your terminal, designed to be a utility to help people, as well as helping new users get used

Rebecca White 2 Dec 03, 2021
TerminalGV is a very simple client to display stats about your SNCF TGV/TER train in your terminal.

TerminalGV So I got bored in the train, TerminalGV is a very simple client to display stats about your SNCF TGV/TER train in your terminal. The "on-tr

Samuel 8 Dec 15, 2022
A simple discord slash command handler for for discord.py.

A simple discord slash command handler for discord.py About β¦Ώ Installation β¦Ώ Disclaimer β¦Ώ Examples β¦Ώ Documentation β¦Ώ Discussions Note that master bran

641 Jan 03, 2023
A Python script for finding a food-truck based on latitude and longitude coordinates that you can run in your shell

Food Truck Finder Project Description This repository contains a Python script for finding a food-truck based on latitude and longitude coordinates th

1 Jan 22, 2022
Custom function scheduler TUI (text-based user interface) in the console

Custom function scheduler TUI (text-based user interface) in the console

Luke 1 Oct 26, 2022