Async boto3 with Autogenerated Data Classes

Overview

awspydk

Async boto3 with Autogenerated JIT Data Classes


Motivation

This library is forked from an internal project that works with a lot of backend AWS APIs, and I got tired of having to constantly parse the returned responses before being able to work with them for (most of the time), a few seconds. Any API-driven application that uses boto3 tends to suffer from being syncronous only. So this library solves a few major problems:

  • Enables both sync and async from the same client.

  • Client can be called implicitly without needing initialization, i.e. you can directly use AwsClient without needing to initialize.

  • Dynamically generates class functions based on the boto3.client child functions, and allows you to call them. This is useful in repl or ipython environments where type-hints are always helpful, especially when the names are so long.

  • Automatically initializes the aws client if its not initialized from the defaults, simply by calling it.

  • Translates all results into Automatically Generated Dataclasses through lazycls.

    • Can be disabled by setting aws.config.AutoCls = False

Quickstart

pip install --upgrade awspydk
from aws import AwsClient

# Sync Method
buckets = AwsClient.v1.s3_list_buckets(as_cls=True)

# Async Method
buckets = await AwsClient.v1.async_s3_list_buckets(as_cls=True)

"""
Both yield the same results.
The underlying classes are auto-generated from Pydantic BaseModels, so anything you can do with Pydantic Models, you can do with these.

{
    'Buckets': [
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 8, 25, 16, 42, 46, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 17, 54, 56, tzinfo=tzutc()), Name='...',
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 3, 4, 20, 10, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 1, 20, 50, 33, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 4, 2, 28, tzinfo=tzutc()), Name='...')
    ],
    'Owner': AwsS3Owner(DisplayName='...', ID='...')
}
"""

## Change Regions
AwsClient.reset(region='us-west-1')

## Change the defaut clients created
from aws.config import DefaultClients

## Modify to only create ec2 client
DefaultClients = {
    'ec2': 'ec2'
}

## Reset implicitly
AwsClient.reset()


BotoKwargs = {
    'AWS_PROFILE': ...,
}

## Reset Explicitly
AwsClient.reset(clients=DefaultClients, boto_kwargs=BotoKwargs)

Client Defaults

These are found in aws.config

AwsRegion = envToStr('AWS_REGION', 'us-east-1')
AutoCls = envToBool('AWSSDK_AUTOCLS', 'true')

## These are the default clients that will be autogenerated.
## Key is the shorthand, value is the actual AWS API Name in boto3
DefaultClients = {
    'ec2': 'ec2', 
    'ecr': 'ecr', 
    'r53' :'route53', 
    'acm': 'acm',
    'elb': 'elb',
    'elbv2': 'elbv2',
    'asg': 'autoscaling',
    's3': 's3'
}

## These are the default resources that will be autogenerated.
## Key is the shorthand, value is the actual AWS Resource API Name in boto3
DefaultResources = {
    'Ec2': 'ec2',
    'S3': 's3',
    'Iam': 'iam'
}

# These are the default filter args for querying
DefaultFilterArgs = {
    'string_only': True,
    'remove_null': True
}
Owner
Chief Architect @ Growth Engine
allow windows programs to call dssp/mkdssp command from wsl; rework biopython on windows (PDB -> dssp -> fasta)

dssp-wsl Converting PDB (Protein Data Bank) file format to DSSP file format is required for generating datasets of peptides and their secondary struct

Taine Zhao 1 Feb 23, 2022
Python Markov Chain chatbot running on Telegram

Hanasubot Hanasubot (Japanese 話すボット, talking bot) is a Python chatbot running on Telegram. The bot is based on Markov Chains so it can learn your word

12 Dec 27, 2022
MusicBot is the original Discord music bot written for Python 3.5+, using the discord.py library

The original MusicBot for Discord (formerly SexualRhinoceros/MusicBot)

Just Some Bots 2.9k Jan 02, 2023
Twitch Points Miner for multiple accounts with Discord logging

Twitch Points Miner for multiple accounts with Discord logging Creator of the Twitch Miner -- PLEASE NOTE THIS IS PROBABLY BANNABLE -- Made on python

8 Apr 27, 2022
A simple Discord Mass-Ban that's still working with Member Scraper.

Mass-Ban [!] This was made for education / you can use for revenge. Please don't skid it. [!] If you want to use it, please use member scraper before

WoahThatsHot 1 Nov 20, 2021
This Server Cloner can clone the server you want with all the perms of roles in every particular channel.

Server-Cloner-with-perms 🚀 This Server Cloner can clone the server you want with all the perms of roles in every particular channel. Features Clone C

Gripz 0 Feb 17, 2022
Messing around with GitHub API to look at omicron build times

gh-workflow-runs This is a very simple tool to dump out basic information about workflow runs for a GitHub repo. The structure is based on gh-subscrip

David Pacheco 1 Nov 30, 2021
A Telegram Bot to Extract Various Types Of Archives

IDN Unzip Bot A Telegram Bot to Extract Various Types Of Archives Features Extract various types of archives like rar, zip, tar, 7z, tar.xz etc. Passw

IDNCoderX 8 Jul 25, 2022
Reddit comment bot emulating Telugu actor N. Bala Krishna.

Balayya-Bot Reddit comment bot emulating Telugu actor N. Bala Krishna. Project structure config.py contains Bot's higher level configuration. generate

Kari Lorince 2 Nov 05, 2021
This Python script will automate the process of uploading your project to GitHub.

ProjectToGithub This Python script will help you to upload your project to Github without having to type in any commands !!! Quick Start guide First C

Imira Randeniya 1 Sep 11, 2022
Reverse engineered connection to the TradingView ticker in Python

Tradingview-ticker Reverse engineered connection to the TradingView ticker in Python. Makes a websocket connection to the Tradeview website and receiv

Aaron 20 Dec 02, 2022
Wrapper for shh/rsync for use with OpenFOAM and blue bear

bbsync wrapper for shh/rsync for use with OpenFOAM and blue bear About The Project bbsync is a wrapper for shh/rsync for use with OpenFOAM and blue be

1 Dec 10, 2021
Gorrabot is a bot made to automate checks and processes in the development process.

Gorrabot is a Gitlab bot made to automate checks and processes in the Faraday development. Features Check that the CHANGELOG is modified By default, m

Faraday 7 Dec 14, 2022
A Python library for PagerDuty.

Pygerduty Python Library for PagerDuty's REST API and Events API. This library was originally written to support v1 and is currently being updated to

Dropbox 164 Dec 20, 2022
QR-Code-Grabber - A python script that allows a person to create a qr code token grabber

Qr Code Grabber Description Un script python qui permet a une personne de creer

5 Jun 28, 2022
The community bot for the Python Discord community

Python Utility Bot This project is a Discord bot specifically for use with the Python Discord server. It provides numerous utilities and other tools t

Python Discord 998 Jan 03, 2023
Python API wrapper library for Convex Value API

convex-value-python Python API wrapper library for Convex Value API. Further Links: Convex Value homepage @ConvexValue on Twitter JB on Twitter Authen

Aaron DeVera 2 May 11, 2022
Python wrapper for JeyyAPI

Async python wrapper for JeyyAPI

7 Dec 10, 2022
A script written in python3 for bruteforcing Gmail accounts.

GmailBruteforce Made for bruteforcing gmail accounts. It needs Less Secure Apps setting turned on in order to work. Installation For windows git clone

Shinero 4 Sep 16, 2022
NekoRobot-2 - Neko is An Anime themed advance Telegram group management bot.

NekoRobot A modular telegram Python bot running on python3 with an sqlalchemy, mongodb database. ╒═══「 Status 」 Maintained Support Group Included Free

Lovely Boy 19 Nov 12, 2022