Tools for Optuna, MLflow and the integration of both.

Related tags

Machine Learningpythonmachine-learningloggingtransformershyperparameter-optimizationhyperparametermlflowoptuna
Overview

HPOflow - Sphinx DOC

DOC MIT License Contributor Covenant Python Version pypi
pytest status Static Code Checks status Build & Deploy Doc GitHub issues

Tools for Optuna, MLflow and the integration of both.

Detailed documentation with examples can be found here: Sphinx DOC

Table of Contents

Maintainers

One Conversation
This project is maintained by the One Conversation team of Deutsche Telekom AG.

The main components are:

Installation

HPOflow is available at the Python Package Index (PyPI). It can be installed with pip:

$ pip install hpoflow

Some additional dependencies might be necessary.

To use hpoflow.optuna_mlflow.OptunaMLflow:

$ pip install mlflow GitPython

To use hpoflow.optuna_transformers.OptunaMLflowCallback:

$ pip install mlflow GitPython transformers

To install all optional dependencies use:

$ pip install hpoflow[optional]

Support and Feedback

The following channels are available for discussions, feedback, and support requests:

Reporting Security Vulnerabilities

This project is built with security and data privacy in mind to ensure your data is safe. We are grateful for security researchers and users reporting a vulnerability to us, first. To ensure that your request is handled in a timely manner and non-disclosure of vulnerabilities can be assured, please follow the below guideline.

Please do not report security vulnerabilities directly on GitHub. GitHub Issues can be publicly seen and therefore would result in a direct disclosure.

Please address questions about data privacy, security concepts, and other media requests to the [email protected] mailbox.

Contribution

Our commitment to open source means that we are enabling - in fact encouraging - all interested parties to contribute and become part of our developer community.

Contribution and feedback is encouraged and always welcome. For more information about how to contribute, as well as additional contribution information, see our Contribution Guidelines.

Code of Conduct

This project has adopted the Contributor Covenant as our code of conduct. Please see the details in our Contributor Covenant Code of Conduct. All contributors must abide by the code of conduct.

Licensing

Copyright (c) 2021 Philip May, Deutsche Telekom AG
Copyright (c) 2021 Philip May
Copyright (c) 2021 Timothy Wolff-Piggott

Licensed under the MIT License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License by reviewing the file LICENSE in the repository.

Comments
  • review README.md and CONTRIBUTING.md

    review README.md and CONTRIBUTING.md

    Review README.md and CONTRIBUTING.md

    • is there something missing? maybe compare with optuna and transformers
    • spelling
    • idiomatic english
    • consistency
    • correctness
    • links ok?
    • ...

    PS: The real documentation is still missing and a know issue.

    opened by PhilipMay 12
  • add typing in optuna_transformers

    add typing in optuna_transformers

    @twolffpiggott can you please tell me the type of this?

    https://github.com/telekom/HPOflow/blob/e2b0943218af419a79ce95e60b67c9a4c2477349/hpoflow/optuna_transformers.py#L47

    opened by PhilipMay 6
  • add `transformers.py`

    add `transformers.py`

    @twolffpiggott should we add this here or to an other project we open source?

    https://github.com/PhilipMay/mltb/blob/master/mltb/integration/transformers.py

    enhancement 
    opened by PhilipMay 6
  • Create Sphinx documentation page

    Create Sphinx documentation page

    • [x] setup
    • [x] make GH action
    • [x] setup page
    • [x] change styling to telekom style
    • switch to MD
    • [x] add more content
    • [x] link from README to page
    • [x] link from pypi to GH page
    • [x] add impressum
    • [x] remove strange mouse over image effect
    • add version info
    documentation 
    opened by PhilipMay 4
  • Problems with direct `_imports.check()` call

    Problems with direct `_imports.check()` call

    When the __init__.py imports OMLflowCallback the optuna_transformers.py script is executed. That executes the _imports.check() call which then throws an exception if transformers or mlflow is not installed. But that should be avoided.

    See here: https://github.com/telekom/HPOflow/blob/d1cce5cbc2a84634d1484a053286000dda05b681/hpoflow/optuna_transformers.py#L11-L17

    The solution would be to put the _imports.check() call into the constructor. But that is not possible because OMLflowCallback inherits from transformers.

    The only solution I have is to put OMLflowCallback into an factory function that creates an OMLflowCallback and does the _imports.check() in there.

    @twolffpiggott what do you think?

    bug 
    opened by PhilipMay 3
  • Flake8 ignore list for Black compatibility

    Flake8 ignore list for Black compatibility

    Flake8 raises a warning for "E203" when it encounters a Black decision to insert whitespace before : in slicing syntax.

    Black's behaviour is more correct here, so my suggestion is to add "E203" to the flake8 config ignore list.

    i.e. in setup.cfg:

    [flake8]
...
extend-ignore = E203
    opened by twolffpiggott 3
  • Simple Example?

    Simple Example?

    I don't understand how to use this package. Could you provide a basic example? I don't understand the import_structure and how it relates to importing the modules? Thanks

    opened by jmrichardson 2
  • WIP prefix in contrib file

    WIP prefix in contrib file

    Should this

    Create Work In Progress [WIP] pull requests only if you need clarification or an explicit review before you can continue your work item.

    be more like this

    Add a [WIP] prefix on your pull request name if you need clarification or an explicit review before you can continue your work item.

    documentation 
    opened by PhilipMay 2
Releases(0.1.4)
Owner
Telekom Open Source Software
published by Deutsche Telekom AG and partner companies
Telekom Open Source Software
GitHub Repository https://telekom.github.io/HPOflow/
Flask app to predict daily radiation from the time series of Solcast from Islamabad, Pakistan

Solar-radiation-ISB-MLOps - Flask app to predict daily radiation from the time series of Solcast from Islamabad, Pakistan.

Abid Ali Awan 1 Dec 31, 2021
Python module for performing linear regression for data with measurement errors and intrinsic scatter

Linear regression for data with measurement errors and intrinsic scatter (BCES) Python module for performing robust linear regression on (X,Y) data po

Rodrigo Nemmen 56 Sep 27, 2022
A data preprocessing package for time series data. Design for machine learning and deep learning.

A data preprocessing package for time series data. Design for machine learning and deep learning.

Allen Chiang 152 Jan 07, 2023
vortex particles for simulating smoke in 2d

vortex-particles-method-2d vortex particles for simulating smoke in 2d -vortexparticles_s

12 Aug 23, 2022
LibRerank is a toolkit for re-ranking algorithms. There are a number of re-ranking algorithms, such as PRM, DLCM, GSF, miDNN, SetRank, EGRerank, Seq2Slate.

LibRerank LibRerank is a toolkit for re-ranking algorithms. There are a number of re-ranking algorithms, such as PRM, DLCM, GSF, miDNN, SetRank, EGRer

126 Dec 28, 2022
李航《统计学习方法》复现

本项目复现李航《统计学习方法》每一章节的算法 特点： 笔记摘要：在每个文件开头都会有一些核心的摘要 pythonic：这里会用尽可能规范的方式来实现，包括编程风格几乎严格按照PEP8 循序渐进：前期的算法会更list的方式来做计算，可读性比较强，后期几乎完全为numpy.array的计算，并且辅助详

58 Oct 22, 2021
GroundSeg Clustering Optimized Kdtree

ground seg and clustering based on kitti velodyne data, and a additional optimized kdtree for knn and radius nn search

2 Dec 02, 2021
Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques

Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learn

Vowpal Wabbit 8.1k Dec 30, 2022
flexible time-series processing & feature extraction

A corona statistics and information telegram bot.

PreDiCT.IDLab 206 Dec 28, 2022
Banpei is a Python package of the anomaly detection.

Banpei Banpei is a Python package of the anomaly detection. Anomaly detection is a technique used to identify unusual patterns that do not conform to

Hirofumi Tsuruta 282 Jan 03, 2023
使用数学和计算机知识投机倒把

偷鸡不成项目集锦 坦率地讲，涉及金融市场的好策略如果公开，必然导致使用的人多，最后策略变差。所以这个仓库只收集我目前失败了的案例。 加密货币组合套利 中国体育彩票预测 我赚不上钱的项目，也许可以帮助更有能力的人去赚钱。

Roy 28 Dec 29, 2022
Machine-Learning with python (jupyter)

Machine-Learning with python (jupyter) 머신러닝 야학 작심 10일과 쥬피터 노트북 기반 데이터 사이언스 시작 들어가기전 https://nbviewer.org/ 페이지를 통해서 쥬피터 노트북 내용을 볼 수 있다. 위 페이지에서 현재 레포 기

HyeonWoo Jeong 1 Jan 23, 2022
Class-imbalanced / Long-tailed ensemble learning in Python. Modular, flexible, and extensible

IMBENS: Class-imbalanced Ensemble Learning in Python Language: English | Chinese/中文 Links: Documentation | Gallery | PyPI | Changelog | Source | Downl

Zhining Liu 176 Jan 04, 2023
PROTEIN EXPRESSION ANALYSIS FOR DOWN SYNDROME

PROTEIN-EXPRESSION-ANALYSIS-FOR-DOWN-SYNDROME Down syndrome (DS) is a chromosomal disorder where organisms have an extra chromosome 21, sometimes know

1 Jan 20, 2022
healthy and lesion models for learning based on the joint estimation of stochasticity and volatility

health-lesion-stovol healthy and lesion models for learning based on the joint estimation of stochasticity and volatility Reference please cite this p

5 Nov 01, 2022
Fourier-Bayesian estimation of stochastic volatility models

fourier-bayesian-sv-estimation Fourier-Bayesian estimation of stochastic volatility models Code used to run the numerical examples of "Bayesian Approa

15 Jun 20, 2022
Python Automated Machine Learning library for tabular data.

Simple but powerful Automated Machine Learning library for tabular data. It uses efficient in-memory SAP HANA algorithms to automate routine Data Scie

Daniel Khromov 47 Dec 17, 2022
Time Series Prediction with tf.contrib.timeseries

TensorFlow-Time-Series-Examples Additional examples for TensorFlow Time Series(TFTS). Read a Time Series with TFTS From a Numpy Array: See "test_input

Zhiyuan He 476 Nov 17, 2022
MegFlow - Efficient ML solutions for long-tailed demands.

Efficient ML solutions for long-tailed demands.

旷视天元 MegEngine 371 Dec 21, 2022
Python 3.6+ toolbox for submitting jobs to Slurm

Submit it! What is submitit? Submitit is a lightweight tool for submitting Python functions for computation within a Slurm cluster. It basically wraps

Facebook Incubator 768 Jan 03, 2023