Delayed iteration for polling and retries.

Overview

image image image image image image image image image

Does Python need yet another retry / poll library? It needs at least one that isn't coupled to decorators and functions. Decorators prevent the caller from customizing delay options, and organizing the code around functions hinders any custom handling of failures.

Waiter is built around iteration instead, because the foundation of retrying / polling is a slowly executing loop. The resulting interface is both easier to use and more flexible, decoupling the delay algorithms from the application logic.

Usage

creation

Supply a number of seconds to repeat endlessly, or any iterable of seconds.

from waiter import wait

wait(1)                 # 1, 1, 1, 1, ...
wait([1] * 3)           # 1, 1, 1
wait([0.5, 0.5, 60])    # circuit breaker

Iterable delays can express any waiting strategy, and constructors for common algorithms are also provided.

wait.count(1)           # incremental backoff 1, 2, 3, 4, 5, ...
wait(1) + 1             # alternate syntax 1, 2, 3, 4, 5, ...
wait.fibonacci(1)       # 1, 1, 2, 3, 5, ...
wait.polynomial(2)      # 0, 1, 4, 9, 16, ...

wait.exponential(2)     # exponential backoff 1, 2, 4, 8, ...
backoff = wait(1) * 2   # alternate syntax 1, 2, 4, 8, ...
backoff[:3]             # limit attempt count 1, 2, 4
backoff <= 5            # set maximum delay   1, 2, 4, 5, 5, 5, ...
backoff.random(-1, 1)   # add random jitter

iteration

Then simply use the wait object like any iterable, yielding the amount of elapsed time. Timeouts also supported of course.

from waiter import wait, suppress, first

for elapsed in wait(delays):            # first iteration is immediate
    with suppress(exception):           # then each subsequent iteration sleeps as necessary
        ...
        break

for _ in wait(delays, timeout):         # standard convention for ignoring a loop variable
    ...                                 # won't sleep past the timeout
    if ...:
        break

results = (... for _ in wait(delays))   # expressions are even easier
first(predicate, results[, default])    # filter for first true item
assert any(results)                     # perfect for tests too

functions

Yes, functional versions are provided, as well as being trivial to implement.

wait(...).throttle(iterable)                      # generate items from iterable
wait(...).repeat(func, *args, **kwargs)           # generate successive results
wait(...).retry(exception, func, *args, **kwargs) # return first success or re-raise exception
wait(...).poll(predicate, func, *args, **kwargs)  # return first success or raise StopIteration

The decorator variants are simply partial applications of the corresponding methods. Note decorator syntax doesn't support arbitrary expressions.

backoff = wait(0.1) * 2
@backoff.repeating
@backoff.retrying(exception)
@backoff.polling(predicate)

But in the real world:

  • the function may not exist or be succinctly written as a lambda
  • the predicate may not exist or be succinctly written as a lambda
  • logging may be required
  • there may be complex handling of different exceptions or results

So consider the block form, just as decorators don't render with blocks superfluous. Also note wait objects are re-iterable provided their original delays were.

async

Waiters also support async iteration. throttle optionally accepts an async iterable. repeat, retry, and poll optionally accept coroutine functions.

statistics

Waiter objects have a stats attribute for aggregating statistics about the calls made. The base implementation provides total and failure counts. The interface of the stats object itself is considered provisional for now, but can be extended by overriding the Stats class attribute. This also allows customization of the iterable values; elapsed time is the default.

Installation

% pip install waiter

Dependencies

  • multimethod

Tests

100% branch coverage.

% pytest [--cov]

Changes

dev

  • Python >=3.7 required

1.2

  • Python >=3.6 required

1.1

  • Stream from sized groups

1.0

  • Map a function across an iterable in batches

0.6

  • Extensible iterable values and statistics
  • Additional constructors: fibonacci, polynomial, accumulate

0.5

  • Asynchronous iteration

0.4

  • Decorators support methods
  • Iterables can be throttled

0.3

  • Waiters behave as iterables instead of iterators
  • Support for function decorators

0.2

  • suppress context manager for exception handling
  • repeat method for decoupled iteration
  • first function for convenient filtering
Python MQTT v5.0 async client

gmqtt: Python async MQTT client implementation. Installation The latest stable version is available in the Python Package Index (PyPi) and can be inst

Gurtam 306 Jan 03, 2023
CoreSE - basic of social Engineering tool

Core Social Engineering basic of social Engineering tool. just for fun :) About First of all, I must say that I wrote such a project because of my int

Hamed Mohammadvand 7 Jun 10, 2022
Manually Install Python 2.7 pip without any problem !

Python2.7_install_pip Manually Install Python 2.7 pip without any problem ! Download installPip.py to your system and Run the code using this Command

Ali Jafari 1 Dec 09, 2021
Python based scripts for obtaining system information from Linux.

sysinfo Python based scripts for obtaining system information from Linux. Python2 and Python3 compatible Output in JSON format Simple scripts and exte

Petr Vavrin 70 Dec 20, 2022
Unified Distributed Execution

Unified Distributed Execution The framework supports multiple execution backends: Ray, Dask, MPI and MultiProcessing. To run tests you need to install

17 Dec 25, 2022
Python library for the Unmand APIs.

Unmand Python SDK This is a simple package to aid in consuming the Unmand APIs. For more help, see our docs. Getting Started Create virtual environmen

Unmand 4 Jul 22, 2022
Automatic certificate unpinning for Android apps

What is this? Script used to perform automatic certificate unpinning of an APK by adding a custom network security configuration that permits user-add

Antoine Neuenschwander 5 Jul 28, 2021
LSO, also known as Linux Swap Operator, is a software with both GUI and terminal versions that you can manage the Swap area for Linux operating systems.

LSO - Linux Swap Operator Türkçe - LSO Nedir? LSO, diğer adıyla Linux Swap Operator Linux işletim sistemleri için Swap alanını yönetebileceğiniz hem G

Eren İnce 4 Feb 09, 2022
A bot to use in a pump & dump event

A bot to use in a pump & dump event on Binance.com. Please note the bot is in heavy devleopment currently so be aware of errors. If you experience err

Freddie Jonas 189 Dec 24, 2022
Running a complete single-node all-in-one cluster instance of TIBCO ActiveMatrix™ BusinessWorks 6.8.0.

TIBCO ActiveMatrix™ BusinessWorks 6.8 Docker Image Image for running a complete single-node all-in-one cluster instance of TIBCO ActiveMatrix™ Busines

Federico Alpi 1 Dec 10, 2021
Graphene Metanode is a locally hosted node for one account and several trading pairs, which uses minimal RAM resources.

Graphene Metanode is a locally hosted node for one account and several trading pairs, which uses minimal RAM resources. It provides the necessary user stream data and order book data for trading in a

litepresence 5 May 08, 2022
Writeup of NilbinSec's participation in the Winja CTF for c0c0n 2021

Winja-CTF-c0c0n-2021-Writeup NilbinSec's participation in the Winja CTF for c0c0n 2021 This repo covers NilbinSec's participation in the Winja CTF dur

1 Nov 15, 2021
A 3-line lisp implementation

Nanolisp The download page of many a language harbors deep senses of forboding, of evil lurking in its native lair. You feel that the language is not

5 Jun 17, 2022
Python Multilingual Ucrel Semantic Analysis System

PymUSAS Python Multilingual Ucrel Semantic Analysis System, it currently is a rule based token level semantic tagger which can be added to any spaCy p

UCREL 13 Nov 18, 2022
Source for the Fedora Silverblue and Kinoite variants.

Source for the Fedora Silverblue and Kinoite variants.

Fedora Kinoite 7 Aug 20, 2022
dta Convert Dict To Attributes!

dta (Dict to Attributes) dta is very small dict (or json) to attributes converter. It is only have 1 files and applied to every python versions.

Rukchad Wongprayoon 0 Dec 31, 2021
Birthday program - A program that lookups a birthday txt file and compares to the current date to check for birthdays

Birthday Program This is a program that lookups a birthday txt file and compares

Daquiver 4 Feb 02, 2022
an elegant datasets factory

rawbuilder an elegant datasets factory Free software: MIT license Documentation: https://rawbuilder.readthedocs.io. Features Schema oriented datasets

Mina Farag 7 Nov 12, 2022
Backup dc registry - A simple POC that abuses Backup Operator privileges to remote dump SAM, SYSTEM, and SECURITY

Backup Operator Registry Backup to Domain Compromise A simple POC that abuses Ba

Horizon 3 AI Inc 57 Dec 18, 2022
Versión preliminar análisis general de Covid-19 en Colombia

Covid_Colombia_v09 Versión: Python 3.8.8 1/ La base de datos del Ministerio de Salud (Minsalud Colombia) está en https://www.datos.gov.co/Salud-y-Prot

Julián Gómez 1 Jan 30, 2022