Tattoo - System for automating the Gentoo arch testing process

Related tags

Testingtattoo
Overview

Naming origin

Well, naming things is very hard. Thankfully we have an excellent team at Gentoo capable of naming things - The Infra team. Alec Warner (antarus) was kind enough to decide on the name "tattoo", which came from combining "tatt" (package testing tool used here) and "Gentoo".

Setup

Developer's own machine

  1. Install the following dependencies:
    • app-portage/nattka
    • net-misc/openssh
  2. Select the directory from which we will work. Always run and set files inside this directory. It can be the source files directory.
  3. Create a ssh_config file using the ssh_config.in template. Fill your username, select the wanted path for socket on every host, and define the various hosts and theirs hostname.
  4. Add an environment variable (for example into .bashrc) named ARCHTESTER_BUGZILLA_APIKEY which equals to an API key from bugzilla.

Remote machine - machine manager

  1. Install the following dependencies:
    • dev-lang/python[sqlite(+)] (tested on >=3.9)
    • app-portage/nattka
  2. Select the directory from which we will work. Always run and set files inside this directory. It can be the source files directory. It must be the same directory as set in the ssh_config directory (by default ~/tattoo)

Remote machine - testing container

  1. Install the following dependencies:
    • app-portage/nattka
    • app-portage/tatt
  2. Make sure the working directory of the machine manager is mount bound into the container. The mount destination inside the container would be the working directory for the testing container.
  3. Create a corresponding ~/.tatt file inside container, for example (IMPORTANT: replace ARCH)
    arch=arm64
    emergeopts="--autounmask --autounmask-continue --autounmask-write"
    repodir="/var/db/repos/gentoo/"
    ignoreprefix="elibc_","video_cards_","linguas_","python_targets_","python_single_target_","kdeenablefinal","test","debug","qemu_user_","qemu_softmmu_","libressl","static-libs","systemd","sdjournal","eloginid","doc","ruby_targets_"
    buildlogdir=/root/logs
    rdeps=0
    usecombis=1
    

Running and using

Load all remote machines

  1. Run in selected directory (for example ~/tattoo) the command ./manager.py.
    • Inside this directory a file named tattoo.socket will be created. Through this socket all communication will occur.
    • A SQLite DB named tattoo.db will hold all successes and failures of test runs.
  2. In every container on that machine, run the command ./tester.py -n [NAME] -a [ARCH] -j [JOBS] where NAME is just a nice textual name to know which container did what, ARCH is the arch to test, with amd64 for stable bugs, and ~arm for keyword bugs. JOBS is the maximal concurrent testing jobs.
    • This command must be ran in the mount bound dir from manager, where the tattoo.socket is created (so it can communicate).
  3. Check that the manager logs all containers connecting to it.

Control from developer's own machine

  1. Connect to remote servers listed in ssh_config using ./controller.py -c. Various sockets are created inside /tmp/tattoo/ directory
  2. Send specific bugs using ./controller.py -b {NUM} {NUM} ... or initiate full scan for open bugs per arch using ./controller.py -s
  3. You can use "follower" mode to get live logs from all machines and testers using ./controller.py follower
  4. When bugs are ready, use ./controller.py fetch -n to view all done bugs, but in dry-run mode (no update for bugzilla, and no update last-seen bugs). Btw, the output corresponds to sam's at-commit script.
  5. When ready to apply, run ./controller.py fetch -ar -d [REPO] where REPO is the ::gentoo repo to apply on it the commits. This command also un-CC and closes bugs for what passed. After success, it saves in small file the last seen bugs, so you don't try to reapply them.
  6. From REPO push the commits (if you are unlucky, git pull --rebase before)
  7. Send, fetch, apply how much you want
  8. Disconnect from all using ./controller.py -d
Owner
Arthur Zamarin
Arthur Zamarin
Testing Calculations in Python, using OOP (Object-Oriented Programming)

Testing Calculations in Python, using OOP (Object-Oriented Programming) Create environment with venv python3 -m venv venv Activate environment . venv

William Koller 1 Nov 11, 2021
Load and performance benchmark tool

Yandex Tank Yandextank has been moved to Python 3. Latest stable release for Python 2 here. Yandex.Tank is an extensible open source load testing tool

Yandex 2.2k Jan 03, 2023
0hh1 solver for the web (selenium) and also for mobile (adb)

0hh1 - Solver Aims to solve the '0hh1 puzzle' for all the sizes (4x4, 6x6, 8x8, 10x10 12x12). for both the web version (using selenium) and on android

Adwaith Rajesh 1 Nov 05, 2021
A pytest plugin to skip `@pytest.mark.slow` tests by default.

pytest-skip-slow A pytest plugin to skip @pytest.mark.slow tests by default. Include the slow tests with --slow. Installation $ pip install pytest-ski

Brian Okken 19 Jan 04, 2023
Repository for JIDA SNP Browser Web Application: Local Deployment

JIDA JIDA is a web application that retrieves SNP information for a genomic region of interest in Homo sapiens and calculates specific summary statist

3 Mar 03, 2022
Python Webscraping using Selenium

Web Scraping with Python and Selenium The code shows how to do web scraping using Python and Selenium. We use as data the https://sbot.org.br/localize

Luís Miguel Massih Pereira 1 Dec 01, 2021
A python bot using the Selenium library to auto-buy specified sneakers on the nike.com website.

Sneaker-Bot-UK A python bot using the Selenium library to auto-buy specified sneakers on the nike.com website. This bot is still in development and is

Daniel Hinds 4 Dec 14, 2022
Photostudio是一款能进行自动化检测网页存活并实时给网页拍照的工具,通过调用Fofa/Zoomeye/360qua/shodan等 Api快速准确查询资产并进行网页截图,从而实施进一步的信息筛查。

Photostudio-红队快速爬取网页快照工具 一、简介: 正如其名:这是一款能进行自动化检测,实时给网页拍照的工具 信息收集要求所收集到的信息要真实可靠。 当然,这个原则是信息收集工作的最基本的要求。为达到这样的要求,信息收集者就必须对收集到的信息反复核实,不断检验,力求把误差减少到最低限度。我

s7ck Team 41 Dec 11, 2022
Doing dirty (but extremely useful) things with equals.

Doing dirty (but extremely useful) things with equals. Documentation: dirty-equals.helpmanual.io Source Code: github.com/samuelcolvin/dirty-equals dir

Samuel Colvin 602 Jan 05, 2023
splinter - python test framework for web applications

splinter - python tool for testing web applications splinter is an open source tool for testing web applications using Python. It lets you automate br

Cobra Team 2.6k Dec 27, 2022
Travel through time in your tests.

time-machine Travel through time in your tests. A quick example: import datetime as dt

Adam Johnson 373 Dec 27, 2022
Pymox - open source mock object framework for Python

Pymox is an open source mock object framework for Python. First Steps Installation Tutorial Documentation http://pymox.readthedocs.io/en/latest/index.

Ivan Rocha 7 Feb 02, 2022
Automating the process of sorting files in my downloads folder by file type.

downloads-folder-automation Automating the process of sorting files in a user's downloads folder on Windows by file type. This script iterates through

Eric Mahasi 27 Jan 07, 2023
Scraping Bot for the Covid19 vaccination website of the Canton of Zurich, Switzerland.

Hi 👋 , I'm David A passionate developer from France. 🌱 I’m currently learning Kotlin, ReactJS and Kubernetes 👨‍💻 All of my projects are available

1 Nov 14, 2021
Kent - Fake Sentry server for local development, debugging, and integration testing

Kent is a service for debugging and integration testing Sentry.

Will Kahn-Greene 100 Dec 15, 2022
Fail tests that take too long to run

GitHub | PyPI | Issues pytest-fail-slow is a pytest plugin for making tests fail that take too long to run. It adds a --fail-slow DURATION command-lin

John T. Wodder II 4 Nov 27, 2022
Penetration testing

Penetration testing

3 Jan 11, 2022
Airspeed Velocity: A simple Python benchmarking tool with web-based reporting

airspeed velocity airspeed velocity (asv) is a tool for benchmarking Python packages over their lifetime. It is primarily designed to benchmark a sing

745 Dec 28, 2022
The Penetration Testers Framework (PTF) is a way for modular support for up-to-date tools.

The PenTesters Framework (PTF) is a Python script designed for Debian/Ubuntu/ArchLinux based distributions to create a similar and familiar distribution for Penetration Testing

trustedsec 4.5k Dec 28, 2022