a wrapper around pytest for executing tests to look for test flakiness and runtime regression

Overview

bubblewrap

a wrapper around pytest for assessing flakiness and runtime regressions

a cs implementations practice project

How to Run:

First, install depedencies

# from the bubblewrap root
# first, install depdencies:
$ pip3 install -r requirements.txt`
# then, invoke with:
$ ./bubblewrap path/to/code

e.g.

$ ./bubblewrap .

or, if you're using add bubblewrap to your $PATH and call with just bubblewrap path/to/code, e.g.:

$ PATH="$HOME/path/to/bubblewrap:$PATH"

Demos

This package also has a couple quick demos for testing against:

# from the bubblewrap root
$ ./bubblewrap examples/stable
# from the bubblewrap root
$ ./bubblewrap examples/flake

Usage Notes

As of right now, imports in test modules need to import the full path of the module, e.g.:

# this is alright: 
from example.one.A import apple
# but this is currently supported:
from A import apple

This'll be changed in the future.

Also, bubblewrap doesn't yet support regression analysis

Local Setup

This project uses Python3.9, pip to manage dependencies, and runs in a virtual environment. Install with the following:

# create + activate venv + install dependencies
# see docs:  https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/
$ python3 -m pip install --user virtualenv
$ python3 -m venv env
$ source env/bin/activate
$ pip install -r requirements.txt
# note: to deactivate venv, use
$ deactivate

Code Formatting

This project uses Black for code formatting:

$ python3 -m black .

The pyproject.toml file includes configuration for Black.

And, the project is linted with flake8

$ python3 -m flake8 path/to/code

Tests

Actual unit tests for this project are also run with with pytest. Having installed the pip dependencies (see prior section), run with the following:

# unit tests, ignoring ones in the demo dirs that are designed to flake
$ python3 -m pytest -vv -k 'not flake' .
# test coverage report
$ python3 -m coverage run -m pytest -k 'not flake' .
$ coverage report -m
Owner
Anna Nagy
software development engineer @twitter
Anna Nagy
Automated tests for OKAY websites in Python (Selenium) - user friendly version

Okay Selenium Testy Aplikace určená k testování produkčních webů společnosti OKAY s.r.o. Závislosti K běhu aplikace je potřeba mít v počítači nainstal

Viktor Bem 0 Oct 01, 2022
Just a small test with lists in cython

Test for lists in cython Algorithm create a list of 10^4 lists each with 10^4 floats values (namely: 0.1) - 2 nested for iterate each list and compute

Federico Simonetta 32 Jul 23, 2022
A configurable set of panels that display various debug information about the current request/response.

Django Debug Toolbar The Django Debug Toolbar is a configurable set of panels that display various debug information about the current request/respons

Jazzband 7.3k Jan 02, 2023
User-interest mock backend server implemnted using flask restful, and SQLAlchemy ORM confiugred with sqlite

Flask_Restful_SQLAlchemy_server User-interest mock backend server implemnted using flask restful, and SQLAlchemy ORM confiugred with sqlite. Backend b

Austin Weigel 1 Nov 17, 2022
a socket mock framework - for all kinds of socket animals, web-clients included

mocket /mɔˈkɛt/ A socket mock framework for all kinds of socket animals, web-clients included - with gevent/asyncio/SSL support ...and then MicroPytho

Giorgio Salluzzo 249 Dec 14, 2022
A collection of benchmarking tools.

Benchmark Utilities About A collection of benchmarking tools. PYPI Package Table of Contents Using the library Installing and using the library Manual

Kostas Georgiou 2 Jan 28, 2022
Set your Dynaconf environment to testing when running pytest

pytest-dynaconf Set your Dynaconf environment to testing when running pytest. Installation You can install "pytest-dynaconf" via pip from PyPI: $ pip

David Baumgold 3 Mar 11, 2022
A pytest plugin to run an ansible collection's unit tests with pytest.

pytest-ansible-units An experimental pytest plugin to run an ansible collection's unit tests with pytest. Description pytest-ansible-units is a pytest

Community managed Ansible repositories 9 Dec 09, 2022
:game_die: Pytest plugin to randomly order tests and control random.seed

pytest-randomly Pytest plugin to randomly order tests and control random.seed. Features All of these features are on by default but can be disabled wi

pytest-dev 471 Dec 30, 2022
Nokia SR OS automation

Nokia SR OS automation Nokia is one of the biggest vendors of the telecommunication equipment, which is very popular in the Service Provider segment.

Karneliuk.com 7 Jul 23, 2022
A toolbar overlay for debugging Flask applications

Flask Debug-toolbar This is a port of the excellent django-debug-toolbar for Flask applications. Installation Installing is simple with pip: $ pip ins

863 Dec 29, 2022
Android automation project with pytest+appium

Android automation project with pytest+appium

1 Oct 28, 2021
Green is a clean, colorful, fast python test runner.

Green -- A clean, colorful, fast python test runner. Features Clean - Low redundancy in output. Result statistics for each test is vertically aligned.

Nathan Stocks 756 Dec 22, 2022
Pytest-rich - Pytest + rich integration (proof of concept)

pytest-rich Leverage rich for richer test session output. This plugin is not pub

Bruno Oliveira 170 Dec 02, 2022
It helps to use fixtures in pytest.mark.parametrize

pytest-lazy-fixture Use your fixtures in @pytest.mark.parametrize. Installation pip install pytest-lazy-fixture Usage import pytest @pytest.fixture(p

Marsel Zaripov 299 Dec 24, 2022
automate the procedure of 403 response code bypass

403bypasser automate the procedure of 403 response code bypass Description i notice a lot of #bugbountytips describe how to bypass 403 response code s

smackerdodi2 40 Dec 16, 2022
User-oriented Web UI browser tests in Python

Selene - User-oriented Web UI browser tests in Python (Selenide port) Main features: User-oriented API for Selenium Webdriver (code like speak common

Iakiv Kramarenko 575 Jan 02, 2023
Rerun pytest when your code changes

A simple watcher for pytest Overview pytest-watcher is a tool to automatically rerun pytest when your code changes. It looks for the following events:

Olzhas Arystanov 74 Dec 29, 2022
This is a Python script for Github Bot which uses Selenium to Automate things.

github-follow-unfollow-bot This is a Python script for Github Bot which uses Selenium to Automate things. Pre-requisites :- Python A Github Account Re

Chaudhary Hamdan 10 Jul 01, 2022
Getting the most out of your hobby servo

ServoProject by Adam Bäckström Getting the most out of your hobby servo Theory The control system of a regular hobby servo looks something like this:

209 Dec 20, 2022