YAML metadata extension for Python-Markdown

Overview

YAML metadata extension for Python-Markdown

Build Status Coverage Status Code style: black Python versions PyPi

This extension adds YAML meta data handling to markdown with all YAML features.

As in the original, metadata is parsed but not used in processing.

Metadata parsed as is by PyYaml and without additional transformations, so this plugin is not compatible with original Meta-Data extension.

Basic Usage

Lorem Ipsum is simply dummy text.

' md.Meta == {'title': 'What is Lorem Ipsum?', 'categories': ['Lorem Ipsum', 'Stupid content']}">
import markdown


text = """---
title: What is Lorem Ipsum?
categories:
  - Lorem Ipsum
  - Stupid content
...

Lorem Ipsum is simply dummy text.
"""

md = markdown.Markdown(extensions=['full_yaml_metadata']})
md.convert(text) == '

Lorem Ipsum is simply dummy text.

'
md.Meta == {'title': 'What is Lorem Ipsum?', 'categories': ['Lorem Ipsum', 'Stupid content']}

Specify a custom YAML loader

By default the full YAML loader is used for parsing, which is insecure when used with untrusted user data. In such cases, you may want to specify a different loader such as yaml.SafeLoader using the extension_configs keyword argument:

import markdown
import yaml

md = markdown.Markdown(extensions=['full_yaml_metadata']}, extension_configs={
        "full_yaml_metadata": {
            "yaml_loader": yaml.SafeLoader,
        },
    },
)

Development and contribution

  • install project dependencies
python setup.py develop
  • install linting, formatting and testing tools
pip install -r requirements.txt
  • run tests
pytest
  • run linters
flake8
mypy ./
black --check ./
  • feel free to contribute!
Comments
  • Move setup_requires to pyproject.yaml

    Move setup_requires to pyproject.yaml

    When build system (setuptools) requirements are specified in setup.py, they end up being installed by distutils, even when pip installing. Because distutils is bit-rotting, it doesn't work with system installed openssl.

    Locally, for me, that means distutils doesn't know about some SSL CA certs, and as such, a pip install markdown-full-yaml-metadata will fail when trying to install setuptools_markdown due to being unable to validate the SSL cert (I am behind a coorporate proxy which MITMs all traffic and resigns with a local cert).

    Moving the setup_requires deps to pyproject.toml fixes this - I suggest something like this:

    [build-system]
    # Minimum requirements for the build system to execute.
    requires = ["setuptools>=36.6", "setuptools_markdown", "wheel",]
    build-backend = "setuptools.build_meta"
    
    opened by jonathanunderwood 16
  • Fixes9+10

    Fixes9+10

    change to README.md on how extension is invoked in python interactive shell closes #9 change to full_yaml_metadata.py to makeExtension parameters closes #10

    opened by philbarker 9
  • RFE: don't pin dependencies exactly

    RFE: don't pin dependencies exactly

    This extension pins every dependency exactly. That's a fine strategy for an end-user application, but for a library it's not really the right thing to do, as you inevitably end up with conflicting dependencies in a project that consumes the library. Please would you consider having more liberal dependencies (eg. markdown>3.0, rather than markdown==3.0.1) for example.

    Thanks for a great extension, by the way!

    opened by jonathanunderwood 4
  • Allow specifing custom loader using configuration option

    Allow specifing custom loader using configuration option

    Currently there is no way to use the yaml.BaseLoader for example. The full loader is unsafe for arbitrary user data and also converts strings like 2020-01-01 10:00:00 to datetime.datetime objects, which might be undesired.

    opened by Holzhaus 1
  • url for pypi at top of page is wrong

    url for pypi at top of page is wrong

    Website link in project description for pypi is wrong https://pypi.python.org/pypi/makrdown… ('makr', should be 'mark') -- I guess this worked before fixing #4

    opened by philbarker 1
  • KeyError: 'configs' on running

    KeyError: 'configs' on running

    After getting extension to load properly, got error:

    >>> import markdown
    >>> md = markdown.Markdown(extensions=['full_yaml_metadata'])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 100, in __init__
        configs=kwargs.get('extension_configs', {}))
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 126, in registerExtensions
        ext = self.build_extension(ext, configs.get(ext, {}))
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 181, in build_extension
        return module.makeExtension(**configs)
      File "/home/phil/Share/Projects/full-yaml-test/python-markdown-full-yaml-metadata/full_yaml_metadata.py", line 45, in makeExtension
        return FullYamlMetadataExtension(configs=configs)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 42, in __init__
        self.setConfigs(kwargs)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 73, in setConfigs
        self.setConfig(key, value)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 61, in setConfig
        if isinstance(self.config[key][0], bool):
    KeyError: 'configs'
    

    I fixed by changing

    def makeExtension(configs: dict={}):
        return FullYamlMetadataExtension(configs=configs)
    

    to

    def makeExtension(*args, **kwargs):
        return FullYamlMetadataExtension(*args, **kwargs)
    

    Hope this helps.

    opened by philbarker 0
  • md = markdown.Markdown(['full_yaml_metadata']) didn't work for me

    md = markdown.Markdown(['full_yaml_metadata']) didn't work for me

    Basic usage on pypi invokes extension with md = markdown.Markdown(['full_yaml_metadata'])

    I got the error:

    >>> md = markdown.Markdown(['full_yaml_metadata'])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: __init__() takes 1 positional argument but 2 were given
    

    But this did work (or at least gave another error) md = markdown.Markdown(extensions=['full_yaml_metadata'])

    opened by philbarker 0
  • Support space after metadata delimitation

    Support space after metadata delimitation

    Hello ! I propose to add the support of spaces after metadata delimiting. Currently if a markdown file contains metadata and a space after the metadata delimiter, the markdown parser crashes. Example :

    ---
    title: What is Lorem Ipsum?
    ---        
    Lorem Ipsum is simply dummy text.
    

    There are spaces after the end of metadata declaration (the third line is equal to '--- '). If we try to parse the example above with the 'full_yaml_metadata' plugin we get the following error :

    E           yaml.composer.ComposerError: expected a single document in the stream
    E             in "<unicode string>", line 1, column 1:
    E               title: What is Lorem Ipsum?
    E               ^
    E           but found another document
    E             in "<unicode string>", line 2, column 1:
    E               ---        
    E               ^
    

    So I propose this little correction, hoping that you will accept it. If you have any remark on the code or on the test don't hesitate to sharing them with me, I would be glad to correct it.

    opened by Ricardaux 2
  • Using pre-commit hooks

    Using pre-commit hooks

    I noticed that some parts of the code don't pass your linter checks. This is common for projects with only one main developer where you sometimes push directly instead of opening a PR. I suggest to use pre-commit, which checks that all linters pass before committing. It also takes care dev dependency installation in a venv (so you can remove dev dependencies from requirements.txt.

    opened by Holzhaus 0
  • Add option to attempt splitting metadata using `\n\n` if metadata delimiters are missing

    Add option to attempt splitting metadata using `\n\n` if metadata delimiters are missing

    This might work for very simple metadata, like:

    title: Foo bar
    date: 2020-01-01 10:00:00
    
    This is text
    

    If the metadata delimiters are not found and the option is enabled, the plugin could do something like this:

    meta, sep, content = text.partition("\n\n")
    try:
        metadata = yaml.load(meta, Loader=...)
    except yaml.error.YAMLError:
        content = text
        metadata = {}
    else:
        # Prevent false-positives if the text does not begin with a metadata block
        if isinstance(metadata, str):
            metadata = {}
            content = text
    
    self.md.Meta = metadata
    return content
    opened by Holzhaus 3
Releases(2.1.0)
Owner
Nikita Sivakov
New World Disorder
Nikita Sivakov
The OpenAPI Specification Repository

The OpenAPI Specification The OpenAPI Specification is a community-driven open specification within the OpenAPI Initiative, a Linux Foundation Collabo

OpenAPI Initiative 25.5k Dec 29, 2022
CoderByte | Practice, Tutorials & Interview Preparation Solutions|

CoderByte | Practice, Tutorials & Interview Preparation Solutions This repository consists of solutions to CoderByte practice, tutorials, and intervie

Eda AYDIN 6 Aug 09, 2022
An open source utility for creating publication quality LaTex figures generated from OpenFOAM data files.

foamTEX An open source utility for creating publication quality LaTex figures generated from OpenFOAM data files. Explore the docs » Report Bug · Requ

1 Dec 19, 2021
Pydocstringformatter - A tool to automatically format Python docstrings that tries to follow recommendations from PEP 8 and PEP 257.

Pydocstringformatter A tool to automatically format Python docstrings that tries to follow recommendations from PEP 8 and PEP 257. See What it does fo

Daniël van Noord 31 Dec 29, 2022
A Python library for setting up projects using tabular data.

A Python library for setting up projects using tabular data. It can create project folders, standardize delimiters, and convert files to CSV from either individual files or a directory.

0 Dec 13, 2022
💻An open-source eBook with 101 Linux commands that everyone should know

This is an open-source eBook with 101 Linux commands that everyone should know. No matter if you are a DevOps/SysOps engineer, developer, or just a Linux enthusiast, you will most likely have to use

Ashfaque Ahmed 0 Oct 29, 2022
Sphinx theme for readthedocs.org

Read the Docs Sphinx Theme This Sphinx theme was designed to provide a great reader experience for documentation users on both desktop and mobile devi

Read the Docs 4.3k Dec 31, 2022
Mayan EDMS is a document management system.

Mayan EDMS is a document management system. Its main purpose is to store, introspect, and categorize files, with a strong emphasis on preserving the contextual and business information of documents.

3 Oct 02, 2021
A python package to import files from an adjacent folder

EasyImports About EasyImports is a python package that allows users to easily access and import files from sister folders: f.ex: - Project - Folde

1 Jun 22, 2022
Generate modern Python clients from OpenAPI

openapi-python-client Generate modern Python clients from OpenAPI 3.x documents. This generator does not support OpenAPI 2.x FKA Swagger. If you need

555 Jan 02, 2023
k3heap is a binary min heap implemented with reference

k3heap k3heap is a binary min heap implemented with reference k3heap is a component of pykit3 project: a python3 toolkit set. In this module RefHeap i

pykit3 1 Nov 13, 2021
OpenAPI (f.k.a Swagger) Specification code generator. Supports C#, PowerShell, Go, Java, Node.js, TypeScript, Python

AutoRest The AutoRest tool generates client libraries for accessing RESTful web services. Input to AutoRest is a spec that describes the REST API usin

Microsoft Azure 4.1k Jan 06, 2023
epub2sphinx is a tool to convert epub files to ReST for Sphinx

epub2sphinx epub2sphinx is a tool to convert epub files to ReST for Sphinx. It uses Pandoc for converting HTML data inside epub files into ReST. It cr

Nihaal 8 Dec 15, 2022
Test utility for validating OpenAPI documentation

DRF OpenAPI Tester This is a test utility to validate DRF Test Responses against OpenAPI 2 and 3 schema. It has built-in support for: OpenAPI 2/3 yaml

snok 106 Jan 05, 2023
Type hints support for the Sphinx autodoc extension

sphinx-autodoc-typehints This extension allows you to use Python 3 annotations for documenting acceptable argument types and return value types of fun

Alex Grönholm 462 Dec 29, 2022
Project created to help beginner programmers to study, despite the lack of internet!

Project created to help beginner programmers to study, despite the lack of internet!

Dev4Dev 2 Oct 25, 2021
charcade is a string manipulation library that can animate, color, and bruteforce strings

charcade charcade is a string manipulation library that can animate, color, and bruteforce strings. Features Animating text for CLI applications with

Aaron 8 May 23, 2022
Python solutions to solve practical business problems.

Python Business Analytics Also instead of "watching" you can join the link-letter, it's already being sent out to about 90 people and you are free to

Derek Snow 357 Dec 26, 2022
Sphinx-performance - CLI tool to measure the build time of different, free configurable Sphinx-Projects

CLI tool to measure the build time of different, free configurable Sphinx-Projec

useblocks 11 Nov 25, 2022
This contains timezone mapping information for when preprocessed from the geonames data

when-data This contains timezone mapping information for when preprocessed from the geonames data. It exists in a separate repository so that one does

Armin Ronacher 2 Dec 07, 2021