AWS Auto Inventory allows you to quickly and easily generate inventory reports of your AWS resources.

Overview

Photo by Denny Müller on Unsplash

Photo by Denny Müller on Unsplash

AWS Automated Inventory ( aws-auto-inventory )

Automates creation of detailed inventories from AWS resources.

Table of Contents


Screenshots


Expand
how-to-run
How to run
ec2-inventory-result
EC2 Inventory Result

Usage


Expand
aws-auto-inventory --help
usage: aws-auto-inventory [-h] --name NAME

Automates creation of detailed inventories from AWS resources.

optional arguments:
  -h, --help            show this help message and exit
  --name NAME, -n NAME  inventory name

Problem

Projects usually have several resources and fetching all the information about these resources manually is a very time-consuming task. This issue is intensified when the same project have multiple account and/or environments, e.g.: NonProd, QA and/or Prod.

Solution

Provide a simple way to fetch the required information and generate a spreadsheet. The information can be filtered, e.g. filter results by tag:x, vpc, subnets, etc. Additionally, inventories can be generated related to many services, which are collected and organized per sheet in the spreadsheet.

Development

# Linux/MacOS:
# clone the project and enter cloned directory
make init build
./dist/aws-auto-inventory --name <your-inventory-name>

Installing


Expand

You will need to create a config.yaml file in order to tell the tool how to generate your inventory, here are the default search paths for each platform:

  • OS X: ~/.config/aws-auto-inventory/config.yaml or ~/Library/Application Support/aws-auto-inventory/config.yaml
  • Other Unix: $XDG_CONFIG_HOME/aws-auto-inventory/config.yaml or ~/.config/aws-auto-inventory/config.yaml
  • Windows: %APPDATA%\aws-auto-inventory\config.yaml where the APPDATA environment variable falls back to %HOME%\AppData\Roaming\config.yaml if undefined

You can use the config-sample as an example. A snippet can be found below:

inventories:
  - name: your-inventory-name
    aws:
      profile: your-aws-profile
      region:
        - us-east-1
    excel:
      transpose: true
    sheets:
      - name: EC2 # sheet name on Excel
        service: ec2 # the boto3 client of an AWS service
        function: describe_instances # the client method of the service defined above
        result_key: Reservations # [optional]: The first key of the response dict
      - name: EBS
        service: ec2
        function: describe_volumes
        result_key: Volumes

If you are interested in building an inventory for multiple AWS Accounts (within your AWS organization) with the same sheets, you can use the config-sample-for-organization for simplicity. Code snippet:

Sheets: &sheets
  - name: CloudFrontDistros
    service: cloudfront
    function: list_distributions
    result_key: DistributionList
  - name: S3Buckets
    service: s3
    function: list_buckets
    result_key: Buckets

inventories:
  - name: your-org-master
    aws:
      profile: your-org
      region:
        - us-east-1
    excel:
      transpose: true
    sheets: *sheets

  - name: your-org-account1
    aws:
      profile: your-org-account1
      region:
        - us-east-1
    excel:
      transpose: true
    sheets: *sheets

  - name: your-org-account2
    aws:
      profile: your-org-account2
      region:
        - us-east-1
    excel:
      transpose: true
    sheets: *sheets

Then you need to run the auto-inventory script multiple times for your accounts as follows:

./dist/aws-auto-inventory --name your-org-master
./dist/aws-auto-inventory --name your-org-account1
./dist/aws-auto-inventory --name your-org-account1

Now, download the binary according to your operating system and platform and execute it, informing which inventory you want to generate. The tool will create a folder aws-auto-inventory-report, in the current path, with the inventory report inside.

Testing


Expand

AWS-Auto-Inventory uses boto3. You can use any service that contains any list or describe method to fetch information about your resources.

Parameters

You can use boto3 parameters to narrow down your search results.

Filter by tag:Name

sheets:
  - name: VPC
    service: ec2
    function: describe_vpcs
    result_key: Vpcs
    parameters:
      Filters:
        - Name: tag:Name
          Values:
            - my-vpc

Filter by vpc-id

sheets:
  - name: Subnets
    service: ec2
    function: describe_subnets
    result_key: Subnets
    parameters:
      Filters:
        - Name: vpc-id
          Values:
            - vpc-xxx

Find a particular RDS instance

sheets:
  - name: RDS
    service: rds
    function: describe_db_instances
    result_key: DBInstances
    parameters:
      DBInstanceIdentifier: the-name-of-my-rds-instance

Find EC2 instances by a particular tag

sheets:
  - name: EC2
    service: ec2
    function: describe_instances
    result_key: Reservations
    parameters:
      Filters:
        - Name: tag:ApplicationName
          Values:
            - my-application

Find a particular IAM Role

sheets:
  - name: IAM.Role
    service: iam
    function: get_role
    result_key: Role
    parameters:
      RoleName: my-role

Contributors


Expand
Name Email Role
Silva, Valter [email protected] AWS Professional Services - Cloud Architect

Security

See CONTRIBUTING for more information.

References


Expand

License


This project is licensed under the Apache License 2.0.

For more information please read LICENSE.

Copyright


Amazon, Inc. or its affiliates. All Rights Reserved.
Comments
  • running on Ubuntu 18.04, glibc requirement and version issue

    running on Ubuntu 18.04, glibc requirement and version issue

    Hi,

    Upon trying to run the linux binary on Ubuntu 18.04, aws-auto-inventory appears to have a requirement on glibc:

    $ aws-auto-inventory 
    [24646] Error loading Python lib '/tmp/_MEIXX3OpN/libpython3.9.so.1.0': dlopen: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /tmp/_MEIXX3OpN/libpython3.9.so.1.0)
    

    While I did install glibc-source, the package version available in Ubuntu repositories is 2.27:

    Package: glibc-source
    Status: install ok installed
    Priority: optional
    Section: devel
    Installed-Size: 23845
    Maintainer: Ubuntu Developers <[email protected]>
    Architecture: all
    Multi-Arch: foreign
    Source: glibc
    Version: 2.27-3ubuntu1.4
    Replaces: eglibc-source
    Recommends: xz-utils
    Conflicts: eglibc-source
    Description: GNU C Library: sources
     This package contains the sources and patches which are needed to
     build glibc.
    Homepage: https://www.gnu.org/software/libc/libc.html
    Original-Maintainer: GNU Libc Maintainers <[email protected]>
    Original-Vcs-Browser: https://salsa.debian.org/glibc-team/glibc
    Original-Vcs-Git: https://salsa.debian.org/glibc-team/glibc.git
    

    Any chance of packaging required dependencies into the binary file itself, like aws-cli v2?

    opened by darkwizard242 3
  • feature/pre-commit

    feature/pre-commit

    Description of changes: Add pre-commit configuration and apply the recommended changes.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by valter-silva-au 1
  • Feature/workflow hygiene

    Feature/workflow hygiene

    Description of changes:

    Create workflow to ensure code and doc hygiene.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by valter-silva-au 0
  • Feature/aws code habits

    Feature/aws code habits

    Description of changes:

    Adopt AWS Code Habits.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by valter-silva-au 0
  • Update devcontainer configuration

    Update devcontainer configuration

    *Description of changes:

    Update the vscode devcontainer configuration for better performance and stability.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    enhancement 
    opened by valter-silva-au 0
  • Release 0.2.0

    Release 0.2.0

    Description of changes: Release of new features and bug fixes.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    bug enhancement 
    opened by valter-silva-au 0
  • Feature/aws credentials

    Feature/aws credentials

    Description of changes: Allow user to provide AWS environment variables.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by valter-silva-au 0
  • Bump numpy from 1.20.0 to 1.22.0

    Bump numpy from 1.20.0 to 1.22.0

    Bumps numpy from 1.20.0 to 1.22.0.

    Release notes

    Sourced from numpy's releases.

    v1.22.0

    NumPy 1.22.0 Release Notes

    NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

    • Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
    • A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
    • NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
    • New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.
    • A new configurable allocator for use by downstream projects.

    These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

    The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

    Expired deprecations

    Deprecated numeric style dtype strings have been removed

    Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

    (gh-19539)

    Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

    numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

    (gh-19615)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Feature/build on GitHub

    Feature/build on GitHub

    Description of changes: Allow builds to be executed on Github.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by valter-silva-au 0
  • Feature/print file path

    Feature/print file path

    Description of changes:

    Print the file path of the report generated.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by valter-silva-au 0
  • Upgrade urllib3

    Upgrade urllib3

    *Issue #, if available: https://github.com/aws-samples/aws-auto-inventory/security/dependabot/requirements.txt/urllib3/open

    Description of changes: Upgrade urllib3

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by valter-silva-au 0
  • Allow Outputs of Other Sheets/Comands

    Allow Outputs of Other Sheets/Comands

    I would like to better document my inventory, not all boto3 client actions return details of my resources. I would like it to be possible to use outputs from other spreadsheets to generate spreadsheets with more details of AWS resources.

    Possible config.yaml

    ---
    Sheets: &sheets
      - name: EKSClusterNames
        service: eks
        function: list_clusters
    
      - name: EKSClusterDetails
        service: eks
        function: describe_cluster
        output_parameter: EKSClusterNames # loop the client action based in EKS Cluster Name list
    
    inventories:
      - name: aws-test
        aws:
          profile: aws-test
          region:
            - us-east-1
            - sa-east-1
        excel:
          transpose: false
        sheets: *sheets
    
    opened by evairmarinho 1
  • Document `config.yaml` arguments

    Document `config.yaml` arguments

    Congrats on the tool! It seems to be very powerful, however some parameters I need are not documented. Would it be possible to document all possible arguments within the configuration YAML?

    opened by evairmarinho 0
Releases(0.2.0)
  • 0.2.0(Aug 16, 2022)

    Changelog

    Improvements

    • Use of Dev Containers for development

    Features

    • Allow users to use AWS environment variables, making the use of AWS profile optional.

    Bug Fixes

    • General improvements.

    What's Changed

    • Bump numpy from 1.20.0 to 1.22.0 by @dependabot in https://github.com/aws-samples/aws-auto-inventory/pull/9
    • Feature/aws credentials by @valter-silva-au in https://github.com/aws-samples/aws-auto-inventory/pull/12
    • Release 0.2.0 by @valter-silva-au in https://github.com/aws-samples/aws-auto-inventory/pull/13

    New Contributors

    • @dependabot made their first contribution in https://github.com/aws-samples/aws-auto-inventory/pull/9

    Full Changelog: https://github.com/aws-samples/aws-auto-inventory/compare/0.1.3...0.2.0

    Source code(tar.gz)
    Source code(zip)
    aws-auto-inventory-macos(41.74 MB)
    aws-auto-inventory-ubuntu(51.39 MB)
    aws-auto-inventory-windows.exe(37.56 MB)
  • 0.1.2(Jun 5, 2021)

  • 0.1.0(Jun 4, 2021)

    You will need to create a config.yaml file in order to tell the tool how to generate your inventory, here are the default search paths for each platform:

    • OS X: ~/.config/aws-auto-inventory/config.yaml or ~/Library/Application Support/aws-auto-inventory/config.yaml
    • Other Unix: $XDG_CONFIG_HOME/aws-auto-inventory/config.yaml or ~/.config/aws-auto-inventory/config.yaml
    • Windows: %APPDATA%\aws-auto-inventory\config.yaml where the APPDATA environment variable falls back to %HOME%\AppData\Roaming\config.yaml if undefined

    You can use the config-sample as an example. A snippet can be found below:

    inventories:
      - name: your-inventory-name
        aws:
          profile: your-aws-profile
          region:
            - us-east-1
        excel:
          transpose: true
        sheets:
          - name: EC2 # sheet name on Excel
            service: ec2 # the boto3 client of an AWS service
            function: describe_instances # the client method of the service defined above
            result_key: Reservations # [optional]: The first key of the response dict
          - name: EBS
            service: ec2
            function: describe_volumes
            result_key: Volumes
    

    Now, download the binary according to your operating system and platform and execute it, informing which inventory you want to generate. The tool will create a folder aws-auto-inventory-report, in the current path, with the inventory report inside.

    Testing


    AWS-Auto-Inventory uses boto3. You can use any service that contains any list or describe method to fetch information about your resources.

    Parameters

    You can use boto3 parameters to narrow down your search results.

    Filter by tag:Name

    sheets:
      - name: VPC
        service: ec2
        function: describe_vpcs
        result_key: Vpcs
        parameters:
          Filters:
            - Name: tag:Name
              Values:
                - my-vpc
    

    Filter by vpc-id

    sheets:
      - name: Subnets
        service: ec2
        function: describe_subnets
        result_key: Subnets
        parameters:
          Filters:
            - Name: vpc-id
              Values:
                - vpc-xxx
    

    Find a particular RDS instance

    sheets:
      - name: RDS
        service: rds
        function: describe_db_instances
        result_key: DBInstances
        parameters:
          DBInstanceIdentifier: the-name-of-my-rds-instance
    

    Find EC2 instances by a particular tag

    sheets:
      - name: EC2
        service: ec2
        function: describe_instances
        result_key: Reservations
        parameters:
          Filters:
            - Name: tag:ApplicationName
              Values:
                - my-application
    

    Find a particular IAM Role

    sheets:
      - name: IAM.Role
        service: iam
        function: get_role
        result_key: Role
        parameters:
          RoleName: my-role
    
    Source code(tar.gz)
    Source code(zip)
    aws-auto-inventory-darwin-x86_64(38.03 MB)
    aws-auto-inventory-linux-x86_64(46.09 MB)
    aws-auto-inventory-windows-amd64.exe(35.35 MB)
Owner
AWS Samples
AWS Samples
A simple Python wrapper for the archive.is capturing service

archiveis A simple Python wrapper for the archive.is capturing service. Installation pipenv install archiveis Python Usage Import it. import archi

Ben Welsh 157 Dec 28, 2022
HASOKI DDOS TOOL- powerful DDoS toolkit for penetration tests

DDoS Attack Panel includes CloudFlare Bypass (UAM, CAPTCHA, GS ,VS ,BFM, etc..) This is open source code. I am not responsible if you use it for malic

Rebyc 1 Dec 02, 2022
Discord bot that automatically fills out health screenings

Auto Covid Bot Automatically fill out the NYC DOE health screening form by registering with a discord bot School code can be found on https://schoolse

Cleo 2 Jul 29, 2022
Chatbot construido com o framework Rasa para responder dúvidas referentes ao COVID-19.

Racom Chatbot Chatbot construido com o framework Rasa. Como executar Necessário instalar Docker e Docker Compose. Para inicializar a aplicação, basta

Vinícius Souza 4 Jul 28, 2022
Davide Gallitelli 3 Dec 21, 2021
DDoS Script (DDoS Panel) with Multiple Bypass ( Cloudflare UAM,CAPTCHA,BFM,NOSEC / DDoS Guard / Google Shield / V Shield / Amazon / etc.. )

KARMA DDoS DDoS Script (DDoS Panel) with Multiple Bypass ( Cloudflare UAM,CAPTCHA,BFM,NOSEC / DDoS Guard / Google Shield / V Shield / Amazon / etc.. )

Hyuk 256 Jan 02, 2023
Python script to extract all Humble Bundle keys and redeem them on Steam automagically.

humble-steam-key-redeemer Python script to extract all Humble keys and redeem them on Steam automagically. This is primarily designed to be a set-it-a

74 Jan 08, 2023
A simple bot which using an API , detects reported discord scams and kicks the user if possible while deleting the message

A simple bot which using an API , detects reported discord scams and kicks the user if possible while deleting the message

Vioshim 3 Nov 16, 2022
Maintained Fork of Jishaku For nextcord

Onami a debugging and utility extension for nextcord bots Read the documentation online. Fork Onami is a actively maintained fork of Jishaku for nextc

RPS 11 Dec 14, 2022
A library that revolutionizes the way people interact with NextDNS.

NextDNS-API An awesome way to interface with your NextDNS account - via Python! Explore the docs » Report Bug . Request Feature Table Of Contents Abou

34 Dec 07, 2022
Updater for PGCG (Paradox Game Converters Group) converters written in Python.

Updater Updater for PGCG (Paradox Game Converters Group) converters written in Python. Needs to be put inside an "Updater" directory in the root conve

Paradox Game Converters 2 Jan 10, 2022
My beancount practice as a template

my-beancount-template 个人 Beancount 方案的模板仓库 相关博客 复式记账指北(一):What and Why? 复式记账指北(二):做账方法论 复式记账指北(三):如何打造不半途而废的记账方案 配置 详细配置请参考博客三。必须修改的配置有: Bot功能:data/be

KAAAsS 29 Nov 29, 2022
Using DST's API with Python

A short guide on how to access Denmark's Statistics API with python, together with a helper class that facilitates the collection of data and metadata from any DST's table

Alessandro Martinello 16 Dec 02, 2022
Deploy a STAC API and a dynamic mosaic tiler API using AWS CDK.

Earth Observation API Deploy a STAC API and a dynamic mosaic tiler API using AWS CDK.

Development Seed 39 Oct 30, 2022
Programa de código abierto para probar el API de Bitso, el exchange más importante de América Latina.

Bitso Semiautomático Programa de código abierto para probar el API de Bitso, el exchange más importante de América Latina. Desarrollador Fernando Mire

Fernando Mireles 17 Dec 07, 2022
An integrated information collection tool

infoscaner 环境配置 目前infoscaner仅支持在linux上运行,建议运行在最新版本的kali中 infoscaner是基于python3版本实现的,运行之前首先安装python库 如果同时存在python2和python3,请输入以下命令 pip3 install -r requi

CMACCKK 74 Sep 13, 2021
A discord bot with a leveling system (similar to mee6).

Discord.py A discord bot with a leveling system (like mee6) Pre-requisites Knowing how to get create an app/bot via discord's developer portal. Websit

26 Dec 11, 2022
This is a simple program that uses Python and pyTwitchAPI to retrieve the list of users in a streamer's chat and then checks each one of these users to see if they follow the broadcaster or not

This is a simple program that uses Python and pyTwitchAPI to retrieve the list of users in a streamer's chat and then checks each one of these users to see if they follow the broadcaster or not

RwinShow 57 Dec 18, 2022
Telegram group manager moderen and simple.

Upin Robot A Advanced Powerful, Smart And Intelligent Group Management Bot With New And Powerful Features ... Written with Pyrogram and Telethon... If

Muhammad Nawawi 3 Dec 23, 2021
Framework to collect and process weather data from wttr.in.

Weathercrawler Automatic extraction and processing framework for weather data from wttr.in Installation tested with: Python 3.7.3 Python 3.9.4 git clo

Maurice Günder 0 Jul 26, 2021