poetry2nix turns Poetry projects into Nix derivations without the need to actually write Nix expressions

Overview

poetry2nix

poetry2nix turns Poetry projects into Nix derivations without the need to actually write Nix expressions. It does so by parsing pyproject.toml and poetry.lock and converting them to Nix derivations on the fly.

For more information, see the announcement post on the Tweag blog. Development of poetry2nix has been supported by Tweag.

API

The poetry2nix public API consists of the following attributes:

  • mkPoetryApplication: Creates a Python application.
  • mkPoetryEnv: Creates a Python environment with an interpreter and all packages from poetry.lock.
  • mkPoetryPackages: Creates an attribute set providing access to the generated packages and other artifacts.
  • mkPoetryScriptsPackage: Creates a package containing the scripts from tool.poetry.scripts of the pyproject.toml.
  • mkPoetryEditablePackage: Creates a package containing editable sources. Changes in the specified paths will be reflected in an interactive nix-shell session without the need to restart it.
  • defaultPoetryOverrides: A set of bundled overrides fixing problems with Python packages.
  • overrides.withDefaults: A convenience function for specifying overrides on top of the defaults.
  • overrides.withoutDefaults: A convenience function for specifying overrides without defaults.
  • cleanPythonSources: A function to create a source filter for python projects.

mkPoetryApplication

Creates a Python application using the Python interpreter specified based on the designated poetry project and lock files. mkPoetryApplication takes an attribute set with the following attributes (attributes without default are mandatory):

  • projectDir: path to the root of the project.
  • src: project source (default: cleanPythonSources { src = projectDir; }).
  • pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
  • poetrylock: poetry.lock file path (default: projectDir + "/poetry.lock").
  • overrides: Python overrides to apply (default: [defaultPoetryOverrides]).
  • meta: application meta data (default: {}).
  • python: The Python interpreter to use (default: pkgs.python3).

Other attributes are passed through to buildPythonApplication.

Example

poetry2nix.mkPoetryApplication {
    projectDir = ./.;
}

See ./pkgs/poetry/default.nix for a working example.

Dependency environment

The resulting derivation also has the passthru attribute dependencyEnv, which is an environment with a python interpreter, all non-development dependencies and your application. This can be used if your application doesn't provide any binaries on its own and instead relies on dependencies binaries to call its modules (as is often the case with celery or gunicorn). For example, if your application defines a CLI for the module admin and a gunicorn app for the module web, a working default.nix would contain

let
    app = poetry2nix.mkPoetryApplication {
        projectDir = ./.;
    };
in app.dependencyEnv

After building this expression, your CLI and app can be called with these commands

$ result/bin/python -m admin
$ result/bin/gunicorn web:app

Note: If you need to perform overrides on the application, use app.dependencyEnv.override { app = app.override { ... }; }. See ./tests/dependency-environment/default.nix for a full example.

mkPoetryEnv

Creates an environment that provides a Python interpreter along with all dependencies declared by the designated poetry project and lock files. Also allows package sources of an application to be installed in editable mode for fast development. mkPoetryEnv takes an attribute set with the following attributes (attributes without default are mandatory):

  • projectDir: path to the root of the project.
  • pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
  • poetrylock: poetry.lock file path (default: projectDir + "/poetry.lock").
  • overrides: Python overrides to apply (default: [defaultPoetryOverrides]).
  • python: The Python interpreter to use (default: pkgs.python3).
  • editablePackageSources: A mapping from package name to source directory, these will be installed in editable mode (default: {}).

Example

poetry2nix.mkPoetryEnv {
    projectDir = ./.;
}

See ./tests/env/default.nix for a working example.

Example with editable packages

poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    editablePackageSources = {
        my-app = ./src;
    };
}

See ./tests/editable/default.nix for a working example of an editable package.

Example shell.nix

The env attribute of the attribute set created by mkPoetryEnv contains a shell environment.

{ pkgs ? import 
    {} }:
let
  myAppEnv = pkgs.poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    editablePackageSources = {
      my-app = ./src;
    };
  };
in myAppEnv.env

Example shell.nix with external dependencies

For a shell environment including external dependencies, override the env to add dependency packages (for example, pkgs.hello) as build inputs.

{ pkgs ? import 
    {} }:
let
  myAppEnv = pkgs.poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    editablePackageSources = {
      my-app = ./src;
    };
  };
in myAppEnv.env.overrideAttrs (oldAttrs: {
  buildInputs = [ pkgs.hello ]:
})

mkPoetryPackages

Creates an attribute set of the shape { python, poetryPackages, pyProject, poetryLock }. Where python is the interpreter specified, poetryPackages is a list of all generated python packages, pyProject is the parsed pyproject.toml and poetryLock is the parsed poetry.lock file. mkPoetryPackages takes an attribute set with the following attributes (attributes without default are mandatory):

  • projectDir: path to the root of the project.
  • pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
  • poetrylock: poetry.lock file path (default: projectDir + "/poetry.lock").
  • overrides: Python overrides to apply (default: [defaultPoetryOverrides]).
  • python: The Python interpreter to use (default: pkgs.python3).
  • editablePackageSources: A mapping from package name to source directory, these will be installed in editable mode (default: {}).

Example

poetry2nix.mkPoetryPackages {
    projectDir = ./.;
    python = python35;
}

mkPoetryScriptsPackage

Creates a package containing the scripts from tool.poetry.scripts of the pyproject.toml:

  • projectDir: path to the root of the project.
  • pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
  • python: The Python interpreter to use (default: pkgs.python3).

Example

poetry2nix.mkPoetryScriptsPackage {
    projectDir = ./.;
    python = python35;
}

mkPoetryEditablePackage

Creates a package containing editable sources. Changes in the specified paths will be reflected in an interactive nix-shell session without the need to restart it:

  • projectDir: path to the root of the project.
  • pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
  • python: The Python interpreter to use (default: pkgs.python3).
  • editablePackageSources: A mapping from package name to source directory, these will be installed in editable mode (default: {}).

Example

poetry2nix.mkPoetryEditablePackage {
    projectDir = ./.;
    python = python35;
    editablePackageSources = {
        my-app = ./src;
    };
}

defaultPoetryOverrides

poetry2nix bundles a set of default overrides that fix problems with various Python packages. These overrides are implemented in overrides.nix.

overrides.withDefaults

Returns a list containing the specified overlay and defaultPoetryOverrides.

Takes an attribute set with the following attributes (attributes without default are mandatory):

  • src: project source directory

Example

poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    overrides = poetry2nix.overrides.withDefaults (self: super: { foo = null; });
}

See ./tests/override-support/default.nix for a working example.

overrides.withoutDefaults

Returns a list containing just the specified overlay, ignoring defaultPoetryOverrides.

Example

poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    overrides = poetry2nix.overrides.withoutDefaults (self: super: { foo = null; });
}

cleanPythonSources

Provides a source filtering mechanism that:

  • Filters gitignore's
  • Filters pycache/pyc files
  • Uses cleanSourceFilter to filter out .git/.hg, .o/.so, editor backup files & nix result symlinks

Example

poetry2nix.cleanPythonSources {
    src = ./.;
}

Creating a custom Poetry2nix instance

Sometimes when it can be convenient to create a custom instance of poetry2nix with a different set of default overrides.

Example

let
  # self & super refers to poetry2nix
  p2nix = poetry2nix.overrideScope' (self: super: {

    # pyself & pysuper refers to python packages
    defaultPoetryOverrides = super.defaultPoetryOverrides.extend (pyself: pysuper: {

      my-custom-pkg = super.my-custom-pkg.overridePythonAttrs (oldAttrs: { });

    });

  });

in
p2nix.mkPoetryApplication {
  projectDir = ./.;
}

or as a nixpkgs overlay:

let
  pkgs = import 
    {
    overlays = [
      # self & super refers to nixpkgs
      (self: super: {

        # p2self & p2super refers to poetry2nix
        poetry2nix = super.poetry2nix.overrideScope' (p2nixself: p2nixsuper: {

          # pyself & pysuper refers to python packages
          defaultPoetryOverrides = p2nixsuper.defaultPoetryOverrides.extend (pyself: pysuper: {

            my-custom-pkg = super.my-custom-pkg.overridePythonAttrs (oldAttrs: { });

          });

        });
      })

    ];
  };

in pkgs.poetry2nix.mkPoetryApplication {
  projectDir = ./.;
}

Using the flake

For the experimental flakes functionality we provide poetry2nix as a flake providing an overlay to use with nixpkgs. Additionally, the flake provides a flake template to quickly start using poetry2nix in a project:

nix flake init --template github:nix-community/poetry2nix

Contributing

Contributions to this project are welcome in the form of GitHub PRs. Please consider the following before creating PRs:

  • This project uses nixpkgs-fmt for formatting the Nix code. You can use nix-shell --run "nixpkgs-fmt ." to format everything.
  • If you are planning to make any considerable changes, you should first present your plans in a GitHub issue so it can be discussed.
  • If you add new features please consider adding tests. You can run them locally as follows:
nix-build --keep-going --show-trace tests/default.nix

License

poetry2nix is released under the terms of the MIT license.

Owner
Nix community projects
A project incubator that works in parallel of the @NixOS org
Nix community projects
ToDoListAndroid - To-do list application created using Kivymd

ToDoListAndroid To-do list application created using Kivymd. Version 1.0.0 (1/Jan/2022). Planned to do next: -Add setting (theme selector, etc) -Add f

AghnatHs 1 Jan 01, 2022
"Hacking" the (Telekom) Zyxel GPON SFP module (PMG3000-D20B)

"Hacking" the (Telekom) Zyxel GPON SFP module (PMG3000-D20B) The SFP can be sour

Matthias Riegler 52 Jan 03, 2023
p5 is a Python package based on the core ideas of Processing.

p5 p5 is a Python library that provides high level drawing functionality to help you quickly create simulations and interactive art using Python. It c

p5py 645 Jan 04, 2023
simple password manager.

simple password manager.

1 Nov 18, 2021
XlvnsScriptTool - Tool for decompilation and compilation of scripts .SDT from the visual novel's engine xlvns

XlvnsScriptTool English Dual languaged (rus+eng) tool for decompiling and compiling (actually, this tool is more than just (dis)assenbler, but less th

Tester 3 Sep 15, 2022
A Python package for searching journal publications and researchers

scholarpy A python package for searching journal publications and researchers Free software: MIT license Documentation: https://giswqs.github.io/schol

Qiusheng Wu 8 Mar 12, 2022
It was created to conveniently respond to events such as donation, follow, and hosting using the Alert Box provided by twip to streamers

This library is not an official library of twip. It was created to conveniently respond to events such as donation, follow, and hosting using the Alert Box provided by twip to streamers.

junah201 8 Nov 19, 2022
Automatically give thanks to Pypi packages you use in your project!

Automatically give thanks to Pypi packages you use in your project!

Ward 25 Dec 20, 2021
Python script that automates the tasks involved in starting a new coding project

Auto Project Builder Automates the repetitive tasks while starting a new project Installation Use the REQUIREMENTS.txt file to install the dependencie

Prathap S S 1 Feb 03, 2022
an elegant datasets factory

rawbuilder an elegant datasets factory Free software: MIT license Documentation: https://rawbuilder.readthedocs.io. Features Schema oriented datasets

Mina Farag 7 Nov 12, 2022
*考研学习利器,玩电脑控制不住自己时,可以使用该程序定日期锁屏,同时有精美壁纸锁屏显示,也不会枯燥。

LockscreenbyTime_win10 A python program in win10. You can set the time to lock the computer(by setting year, month, day), Fullscreen pictures will sho

PixianDouban 4 Jul 10, 2022
Get a list of the top-10 rejected libraries in your WhiteSource inventory

WhiteSource Top 10 Rejected Libraries Generate a spreadsheet listing the 10 most common libraries in your WhiteSource inventory that were rejected by

WhiteSource-PS-tools 10 Mar 23, 2022
Oregon State University grade distributions from Fall 2018 through Summer 2021

Oregon State University Grades Oregon State University grade distributions from Fall 2018 through Summer 2021 obtained through a Freedom Of Informatio

Melanie Gutzmann 5 May 02, 2022
Generate a wordlist to fuzz amounts or any other numerical values.

Generate a wordlist to fuzz amounts or any other numerical values. Based on Common Security Issues in Financially-Oriented Web Applications.

Ivan Šincek 3 Oct 14, 2022
A Python3 script to decode an encoded VBScript file, often seen with a .vbe file extension

vbe-decoder.py Decode one or multiple encoded VBScript files, often seen with a .vbe file extension. Usage usage: vbe-decoder.py [-h] [-o output] file

John Hammond 147 Nov 15, 2022
Pre-commit hook for upgrading type hints

This is a pre-commit hook configured to automatically upgrade your type hints to the new native types implemented in PEP 585.

snok 54 Nov 14, 2022
Free version of Okuru selfbot, okuru.xyz

Indigo Selfbot Free OpenSource selfbot, Premium version can be found at https://okuru.xyz (5$.) Usage python[3] main.py Installation To install you ca

Dimitri Demarkus 31 Aug 07, 2022
Integer sets where all subsets have unique sums

Evil Sums Generation of sets of numbers where all constituents are recoverable from a partial sum.

Charlotte 5 Sep 24, 2022
Analisador de strings feito em Python // String parser made in Python

Este é um analisador feito em Python, neste programa, estou estudando funções e a sua junção com "if's" e dados colocados pelo usuário. Neste código,

Dev Nasser 1 Nov 03, 2021
March-madness - March Madness results 1985-2021

march-madness Results for all 2,268 NCAA Division I Men's Basketball Tournament games since the modern format was introduced in 1985. Includes years,

Darik Harter 2 Feb 26, 2022