Delayed iteration for polling and retries.

Overview

image image image image image image image image image

Does Python need yet another retry / poll library? It needs at least one that isn't coupled to decorators and functions. Decorators prevent the caller from customizing delay options, and organizing the code around functions hinders any custom handling of failures.

Waiter is built around iteration instead, because the foundation of retrying / polling is a slowly executing loop. The resulting interface is both easier to use and more flexible, decoupling the delay algorithms from the application logic.

Usage

creation

Supply a number of seconds to repeat endlessly, or any iterable of seconds.

from waiter import wait

wait(1)                 # 1, 1, 1, 1, ...
wait([1] * 3)           # 1, 1, 1
wait([0.5, 0.5, 60])    # circuit breaker

Iterable delays can express any waiting strategy, and constructors for common algorithms are also provided.

wait.count(1)           # incremental backoff 1, 2, 3, 4, 5, ...
wait(1) + 1             # alternate syntax 1, 2, 3, 4, 5, ...
wait.fibonacci(1)       # 1, 1, 2, 3, 5, ...
wait.polynomial(2)      # 0, 1, 4, 9, 16, ...

wait.exponential(2)     # exponential backoff 1, 2, 4, 8, ...
backoff = wait(1) * 2   # alternate syntax 1, 2, 4, 8, ...
backoff[:3]             # limit attempt count 1, 2, 4
backoff <= 5            # set maximum delay   1, 2, 4, 5, 5, 5, ...
backoff.random(-1, 1)   # add random jitter

iteration

Then simply use the wait object like any iterable, yielding the amount of elapsed time. Timeouts also supported of course.

from waiter import wait, suppress, first

for elapsed in wait(delays):            # first iteration is immediate
    with suppress(exception):           # then each subsequent iteration sleeps as necessary
        ...
        break

for _ in wait(delays, timeout):         # standard convention for ignoring a loop variable
    ...                                 # won't sleep past the timeout
    if ...:
        break

results = (... for _ in wait(delays))   # expressions are even easier
first(predicate, results[, default])    # filter for first true item
assert any(results)                     # perfect for tests too

functions

Yes, functional versions are provided, as well as being trivial to implement.

wait(...).throttle(iterable)                      # generate items from iterable
wait(...).repeat(func, *args, **kwargs)           # generate successive results
wait(...).retry(exception, func, *args, **kwargs) # return first success or re-raise exception
wait(...).poll(predicate, func, *args, **kwargs)  # return first success or raise StopIteration

The decorator variants are simply partial applications of the corresponding methods. Note decorator syntax doesn't support arbitrary expressions.

backoff = wait(0.1) * 2
@backoff.repeating
@backoff.retrying(exception)
@backoff.polling(predicate)

But in the real world:

  • the function may not exist or be succinctly written as a lambda
  • the predicate may not exist or be succinctly written as a lambda
  • logging may be required
  • there may be complex handling of different exceptions or results

So consider the block form, just as decorators don't render with blocks superfluous. Also note wait objects are re-iterable provided their original delays were.

async

Waiters also support async iteration. throttle optionally accepts an async iterable. repeat, retry, and poll optionally accept coroutine functions.

statistics

Waiter objects have a stats attribute for aggregating statistics about the calls made. The base implementation provides total and failure counts. The interface of the stats object itself is considered provisional for now, but can be extended by overriding the Stats class attribute. This also allows customization of the iterable values; elapsed time is the default.

Installation

% pip install waiter

Dependencies

  • multimethod

Tests

100% branch coverage.

% pytest [--cov]

Changes

dev

  • Python >=3.7 required

1.2

  • Python >=3.6 required

1.1

  • Stream from sized groups

1.0

  • Map a function across an iterable in batches

0.6

  • Extensible iterable values and statistics
  • Additional constructors: fibonacci, polynomial, accumulate

0.5

  • Asynchronous iteration

0.4

  • Decorators support methods
  • Iterables can be throttled

0.3

  • Waiters behave as iterables instead of iterators
  • Support for function decorators

0.2

  • suppress context manager for exception handling
  • repeat method for decoupled iteration
  • first function for convenient filtering
Age of Empires II recorded game parsing and summarization in Python 3.

mgz Age of Empires II recorded game parsing and summarization in Python 3. Supported Versions Age of Kings (.mgl) The Conquerors (.mgx) Userpatch 1.4

148 Dec 11, 2022
Lookup for interesting stuff in SMB shares

SMBSR - what is that? Well, SMBSR is a python script which given a CIDR/IP/IP_file/HOSTNAME(s) enumerates all the SMB services listening (445) among t

Vincenzo 112 Dec 15, 2022
A casual IDOR exploiter that provides .csv files of url and status code.

IDOR-for-the-casual Do you like to IDOR? Are you a Windows hax0r? Well have I got a tool for you... A casual IDOR exploiter that provides .csv files o

Ben Wildee 2 Jan 20, 2022
Script de monitoramento das teclas do teclado, salvando todos os dados digitados em um arquivo de log juntamente com os dados de rede.

listenerPython Script de monitoramento das teclas do teclado, salvando todos os dados digitados em um arquivo de log juntamente com os dados de rede.

Vinícius Azevedo 4 Nov 27, 2022
Ml-design-patterns - Source code accompanying O'Reilly book: Machine Learning Design Patterns

This is not an official Google product ml-design-patterns Source code accompanying O'Reilly book: Title: Machine Learning Design Patterns Authors: Val

Google Cloud Platform 1.5k Jan 05, 2023
Toppr Os Auto Class Joiner

Toppr Os Auto Class Joiner Toppr os is a irritating platform to work with especially for students it takes a while and is problematic most of the time

1 Dec 18, 2021
a bit of my project :) and I use some of them for my school lesson or study for an exam! but some of them just for myself.

Handy Project a bit of my project :) and I use some of them for my school lesson or study for an exam! but some of them just for myself. the handy pro

amirkasra esmaeilian 13 Jul 05, 2021
RangDev Notepad App With Python

RangDev Notepad-App-With-Python Take down quick and speedy notes! This is a small project of a notepad app built with Tkinter and SQLite3. Database cr

rangga.alrasya 1 Dec 01, 2021
Scripts for hosting urbit in production-ish

Urbit Sysops Contains some helpful scripts for hosting Urbit. There are two variants included in this repo: one using docker, and one using plain syst

Jōshin 12 Sep 25, 2022
Simple Python script I use to manage and build my Reflux themes.

Simple Python script I use to manage and build my Reflux themes. Built for personal use, but anyone can easily fork and tweak to suit thier needs.

Ire 3 Jan 25, 2022
Aerial Ace is a helper bot for poketwo which provide various functionalities on top of being a pokedex.

Aerial Ace is a helper bot for poketwo which provide various functionalities on top of being a pokedex.

Devanshu Mishra 1 Dec 01, 2021
A light library to build tiny websites

A light library to build tiny websites

BT.Q 1 Dec 23, 2021
The ROS publisher/subscriber example packaged as a snap

publisher-subscriber The ROS publisher/subscriber example packaged as a snap, based on ROS Noetic and Ubuntu Core 20. Strictly confined. This example

3 Dec 03, 2021
A compiler for ARM, X86, MSP430, xtensa and more implemented in pure Python

A compiler for ARM, X86, MSP430, xtensa and more implemented in pure Python

Windel Bouwman 277 Dec 26, 2022
Petuhlang is a joke-like language, based on Python.

Petuhlang is a joke-like language, based on Python. It updates builtins to make a new syntax based on operators rewrite.

DenyS 9 Jun 19, 2022
Pokemon catch events project to demonstrate data pipeline on AWS

Pokemon Catches Data Pipeline This is a sample project to practice end-to-end data project; Terraform is used to deploy infrastructure; Kafka is the t

Vitor Carra 4 Sep 03, 2021
Turn a raspberry pi into a Bluetooth Midi device

PiBluetoothMidSetup This will change serveral system wide packages/configurations Do not run this on your primary machine or anything you don't know h

MyLab6 40 Sep 19, 2022
mrcal is a generic toolkit to solve calibration and SFM-like problems originating at NASA/JPL

mrcal is a generic toolkit to solve calibration and SFM-like problems originating at NASA/JPL. Functionality related to these problems is exposed as a set of C and Python libraries and some commandli

Dima Kogan 102 Dec 23, 2022
A simple chatbot that I made for school project

Chatbot: Python A simple chatbot that I made for school Project. Tho this chatbot is dumb sometimes, but it's not too bad lol. Check it Out! FAQ How t

Prashant 2 Nov 13, 2021
A male and female dog names python package

A male and female dog names python package

Fayas Noushad 3 Dec 12, 2021