img-proof (IPA) provides a command line utility to test images in the Public Cloud

Overview

Continuous testing & Linting Documentation Status Py Versions License

img-proof

overview

img-proof (IPA) provides a command line utility to test images in the Public Cloud (AWS, Azure, GCE, etc.).

With img-proof you can now test custom images in a cloud framework agnostic way with one tool and one API. In the first release, img-proof supports the openSUSE and SLES distributions. It also supports the three largest cloud frameworks (AWS, Azure and GCE). However, it is intended to be distribution agnostic and framework transparent so both are easily extensible.

For each distribution there are specific synchronization points that must be provided. These currently include soft reboot and system update. The synch points not only test functionality but also act as dividers to separate distinct sections of a test suite. For example you can run a test to ensure the proper repos exist before and after a system update. The system update synch point will guarantee the order of tests. Speaking of tests, if you're already familiar with Pytest conventions there's no need to learn a whole new unit testing framework. img-proof is written in Python and leverages the Pytest framework through Testinfra.

Installation

To install the package use the following commands as root:

$ zypper ar http://download.opensuse.org/repositories/Cloud:/Tools/<distribution>
$ zypper refresh
$ zypper in python3-img-proof

Requirements

  • boto3
  • apache-libcloud
  • azure-common
  • azure-mgmt-compute
  • azure-mgmt-network
  • azure-mgmt-resource
  • Click
  • paramiko
  • pytest
  • PyYaml
  • testinfra
  • oci

Docs

Tests

img-proof uses the Testinfra package for writing unit tests. Testinfra leverages Pytest and provides modules such as Package, Process and Service to test the state of images. See the Testinfra Docs for more information on writing infrastructure tests.

img-proof currently passes the Pytest option -x (stop on first failure) through as --early-exit. If there's an interest or need for any other options/args please submit an issue to Github.

CLI Overview

The CLI provides multiple subcommands to initiate image testing:

  • img-proof test

    Test image in the given framework using the supplied test files.

  • img-proof results

    Invokes the default show subcommand img-proof results show 1.

  • img-proof results clear

    Clear the results from the history file.

  • img-proof results delete

    Delete the specified history item from the history log.

  • img-proof results list`

    Display list of results history.

  • img-proof results show

    Display the results or log file for a history item.

  • img-proof list

    Print a list of test files or test cases.

Issues/Enhancements

Please submit issues and requests to Github.

Contributing

Contributions to ipa are welcome and encouraged. See CONTRIBUTING for info on getting started.

License

Copyright (c) 2018 SUSE LLC.

Distributed under the terms of GPL-3.0+ license, see LICENSE for details.

Comments
  • Add exclude parameter

    Add exclude parameter

    Adds a new --exclude parameter to img-proof test, that allows to pass a comma-separated list of test expressions to skip during a test run.

    This allows for using of common patterns e.g. test_sles without running certain tests. Before one needed to state an explicit list of test runs with the risk of not including new test runs as they are included into img-proof.

    Stale 
    opened by grisu48 12
  • Need `--resource-group` parameter with `--running-instance-id`

    Need `--resource-group` parameter with `--running-instance-id`

    Currently we do the assumption, that the resource group name is equal to the name of the VM. This isn't always the case.

    1. We should add a parameter --resource-group for azure.
    2. We could allow using full instance id's like /subscriptions/c011786b-59d7-4817-880c-7cd8a6ca4b19/resourceGroups/openqa-suse-de-1ec3f5a05b7c0712/providers/Microsoft.Compute/virtualMachines/openqa-suse-de-1ec3f5a05b7c0712 and parse resource-group and instance name from it.
    opened by cfconrad 7
  • EC2 SSH key use user data

    EC2 SSH key use user data

    Use user data to add SSH public key to instance on EC2 launch when ssh key name is not provided.

    • SSH public key is generated based on private key file.
    • ssh_key_name arg is optional but takes precedence if provided.
    opened by smarlowucf 7
  • Handle the API change in cloud-regionsrv-client

    Handle the API change in cloud-regionsrv-client

    • The cloud-regionsrv-client API for the is_registered() function has changed from expecting an object to expecting a string. Change the test implementation to handle both cases.
    opened by rjschwei 5
  • traceback generated during mash run

    traceback generated during mash run

    ERROR 2019-05-20 20:14:03,773 TestingService Pass[1]: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ipa/ipa_cloud.py", line 636, in test_im age client File "/usr/lib/python3.6/site-packages/ipa/ipa_utils.py", line 262, in get_hos t_key_fingerprint client.get_transport().get_remote_server_key().get_fingerprint() File "/usr/lib/python3.6/site-packages/paramiko/transport.py", line 761, in ge t_remote_server_key raise SSHException("No existing session") paramiko.ssh_exception.SSHException: No existing session

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/mash/services/testing/ipa_helper.py", l ine 95, in ipa_test timeout=ipa_timeout File "/usr/lib/python3.6/site-packages/ipa/ipa_controller.py", line 131, in te st_image return cloud.test_image() File "/usr/lib/python3.6/site-packages/ipa/ipa_cloud.py", line 644, in test_im age 'An error occurred retrieving host key: %s' % error ipa.ipa_exceptions.IpaCloudException: An error occurred retrieving host key: No existing session

    mash job id: 4f8c81dd-cfd4-4697-be5f-1777c6c4b220

    opened by rjschwei 4
  • repo test

    repo test

    The suse specific repository test should provide more details. With the switch to the modules and the various flavors of repositories, source, debuginfo etc. it has pretty much become impossible to manually verify registration success. IPA provides the necessary functionality. However, the registration test has a pass/fail status leaving the user/verifyer with the original problem. the test shold produce a diff list in the results if the repo data does not match the expected content. This should hopefully allow the person verifying the test results an opportunity to understand the failure and address it.

    opened by rjschwei 4
  • Apache libcloud

    Apache libcloud

    Consider migrating EC2 and Azure to Apache Libcloud to minimize dependencies and make providers more uniform.

    This would also make adding new providers much easier so long as they are supported by libcloud.

    Todo:

    • [x] GCE
    • [x] EC2
    • [ ] Azure
    enhancement 
    opened by smarlowucf 4
  • Fix wrong exclude type

    Fix wrong exclude type

    Allows the exclude parameter to be any iterable and string. This fixes a regression in which passing any exclude parameter crashed the program.

    Fixes https://github.com/SUSE-Enceladus/img-proof/issues/345

    opened by grisu48 3
  • Add grow root test

    Add grow root test

    Set root disk to non-default size of 50GB except in OCI (TBD). And test confirms that the filesystem properly expands.

    I could not find info in OCI SDK docs about changing root disk parameters when launching instances. Issue opened upstream.

    opened by smarlowucf 3
  • Add Red Hat family distro module and Fedora distro module

    Add Red Hat family distro module and Fedora distro module

    This change adds support for testing Red Hat based distributions, and adds a Fedora module for testing Fedora cloud images.

    This PR is the beginning of investigating using img_proof for automatically testing Fedora Cloud Edition images by the Fedora Cloud SIG.

    (cc: @dustymabe, @major)

    opened by Conan-Kudo 3
  • treat uuid always lowercase

    treat uuid always lowercase

    Since https://github.com/torvalds/linux/commit/712ff25450bd01366301eef81c33e865d901e7b7 the uuid is created with lowercase letters.

    I see failing test with SLE15-SP1: https://openqa.suse.de/tests/2340648#step/EC2_test_sles_ec2_uuid_py__test_sles_ec2_uuid/1

    opened by cfconrad 3
  • configparser>=3.5.0b1 distribution was not found and is required by oci

    configparser>=3.5.0b1 distribution was not found and is required by oci

    Hi,

    when installing img-proof from the http://download.opensuse.org/repositories/Cloud:/Tools/openSUSE_Leap_15.2/ repository, I get the following error message:

    [email protected]:~> img-proof --version
    Traceback (most recent call last):
      File "/usr/bin/img-proof", line 6, in <module>
        from pkg_resources import load_entry_point
      File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3123, in <module>
        @_call_aside
      File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3107, in _call_aside
        f(*args, **kwargs)
      File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3136, in _initialize_master_working_set
        working_set = WorkingSet._build_master()
      File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 578, in _build_master
        ws.require(__requires__)
      File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 895, in require
        needed = self.resolve(parse_requirements(requirements))
      File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 781, in resolve
        raise DistributionNotFound(req, requirers)
    pkg_resources.DistributionNotFound: The 'configparser>=3.5.0b1' distribution was not found and is required by oci
    

    I followed the installation procedure from the README.md

    zypper ar http://download.opensuse.org/repositories/Cloud:/Tools/openSUSE_Leap_15.2/ PC_Tools_15_2
    zypper ref
    zypper in python3-img-proof
    
    opened by grisu48 1
  • Drop extension from test names

    Drop extension from test names

    If a user supplies the .py extension in a test description yaml file or in command line it would be helpful for img-proof to drop this by default prior to doing the test search. Or it would be nice to provide more info with the test not found message to make it clear that the extension is not needed.

    enhancement 
    opened by smarlowucf 0
  • SLES test suite should wait for initialization

    SLES test suite should wait for initialization

    Currently when running the SLES test suite on BYOS images we might reboot to soon. Prior to reboot img-proof should wait for the initialization code to finish:

    • In Azure this is waagent
    • In EC2 and OCI it's cloud-init-final
    • In GCE it's google-startup-scripts
    enhancement 
    opened by smarlowucf 0
  • Allow reading of results from an archive

    Allow reading of results from an archive

    If ipa handles listing results from an archive then the user should just give us the archive name, i.e.

    ipa results list --archive $PATH_TO_MY_ARCHIVE ipa results show 5 --archive $PATH_TO_MY_ARCHIVE

    And ipa should just extract the file on the fly to get the results.

    Question is should this be to a temp dir which is deleted after the command or should it be cached somehow for subsequent calls?

    enhancement 
    opened by smarlowucf 0
  • Add Config setup endpoint

    Add Config setup endpoint

    It would be a nice enhancement to have a config setup option in CLI. This would help the user setup the ipa config and/or all config files with info for testing. Since a user may not have config setup for ec2utils, azurectl or Google.

    ipa setup

    Series of prompts requesting config data.

    enhancement 
    opened by smarlowucf 0
Releases(v7.0.0)
Terminal-based keyboard testing

kbdtest kbdtest is a simple Python program that tests keyboard input using an interactive, terminal-based, visual keyboard display. It was originally

Ruunyox 12 Jul 19, 2022
A CLI for advanced management of your notes with simple commands

PyNoteManager This is a CLI for advanced management of your notes with simple co

3 Dec 30, 2021
Free and Open-Source Command Line tool for Text Replacement

Sniplet Free and Open Source Text Replacement Tool Description: Sniplet is a work in progress CLI tool which can do text replacement globally in Linux

Veeraraghavan Narasimhan 13 Nov 28, 2022
Dynamically Generate GitHub Stats as like Terminal Interface

GitHub Stats Terminal Style Dynamically Generate GitHub Stats as like Terminal Interface Usage Create a New Repository using this Template or click he

YOGESHWARAN R 63 Jan 03, 2023
ddgr is a cmdline utility to search DuckDuckGo (html version) from the terminal

ddgr is a cmdline utility to search DuckDuckGo (html version) from the terminal. While googler is extremely popular among cmdline users, in many forums the need of a similar utility for privacy-aware

Piña Colada 2.5k Dec 25, 2022
Magnificent app which corrects your previous console command.

The Fuck The Fuck is a magnificent app, inspired by a @liamosaur tweet, that corrects errors in previous console commands. Is The Fuck too slow? Try t

Vladimir Iakovlev 75k Jan 02, 2023
MsfMania is a command line tool developed in Python that is designed to bypass antivirus software on Windows and Linux/Mac in the future

MsfMania MsfMania is a command line tool developed in Python that is designed to bypass antivirus software on Windows and Linux/Mac in the future. Sum

446 Dec 21, 2022
AthenaCLI is a CLI tool for AWS Athena service that can do auto-completion and syntax highlighting.

Introduction AthenaCLI is a command line interface (CLI) for the Athena service that can do auto-completion and syntax highlighting, and is a proud me

dbcli 192 Jan 07, 2023
A 3D engine powered by ASCII art

3D engine powered by ASCII art

Lingdong Huang 48 Nov 16, 2022
jenkins-tui is a terminal based user interface for Jenkins.

jenkins-tui 📦 jenkins-tui is a terminal based user interface for Jenkins. 🚧 ⚠️ This app is a prototype and in very early stages of development. Ther

Craig Gumbley 22 Oct 24, 2022
The easiest way to create beautiful CLI for your programs.

The Yandere is a program written in Python3, allowing you to create your own beautiful CLI tool.

Billy 31 Dec 20, 2022
A command-line utility that, given a markdown file, checks whether all its links work.

A command-line utility written in Python that checks validity of links in a markdown file.

Teclado 2 Dec 08, 2021
CLI utility to search and download torrents from major torrent sites

CLI Torrent Downloader About CLI Torrent Downloader provides convenient and quick way to search torrent magnet links (and to run associated torrent cl

x0r0x 86 Dec 19, 2022
Microsoft Azure CLI - Azure Command-Line Interface

A great cloud needs great tools; we're excited to introduce Azure CLI, our next generation multi-platform command line experience for Azure.

Microsoft Azure 3.4k Dec 30, 2022
Commandline Python app to Autodownload mediafire folders and files.

Commandline Python app to Autodownload mediafire folders and files.

Tharuk Renuja 3 May 12, 2022
Voidlx is a terminal cli apps launcher made in python

Voidlx is a terminal cli apps launcher made in python

2 Nov 13, 2021
LSD (Linux Spotify Downloader) is a command line tool for downloading or rather recording content on Spotify.

LSD (Linux Spotify Downloader) is a command line tool for downloading or rather recording content on Spotify.

Jannis Zahn 7 Jun 21, 2022
Arithmos cipher on CLI based

Arithmos Cipher CLI This is the CLI version of Arithmos Cipher. Install pip inst

LyQuid :3 1 Jan 16, 2022
flora-dev-cli (fd-cli) is command line interface software to interact with flora blockchain.

Install git clone https://github.com/Flora-Network/fd-cli.git cd fd-cli python3 -m venv venv source venv/bin/activate pip install -e . --extra-index-u

14 Sep 11, 2022
moviepy-cli: Command line interface for MoviePy.

Moviepy-cli is designed to apply several video editing in a single command with Moviepy as an alternative to Video-cli.

Kentaro Wada 23 Jun 29, 2022