YAML metadata extension for Python-Markdown

Overview

YAML metadata extension for Python-Markdown

Build Status Coverage Status Code style: black Python versions PyPi

This extension adds YAML meta data handling to markdown with all YAML features.

As in the original, metadata is parsed but not used in processing.

Metadata parsed as is by PyYaml and without additional transformations, so this plugin is not compatible with original Meta-Data extension.

Basic Usage

Lorem Ipsum is simply dummy text.

' md.Meta == {'title': 'What is Lorem Ipsum?', 'categories': ['Lorem Ipsum', 'Stupid content']}">
import markdown


text = """---
title: What is Lorem Ipsum?
categories:
  - Lorem Ipsum
  - Stupid content
...

Lorem Ipsum is simply dummy text.
"""

md = markdown.Markdown(extensions=['full_yaml_metadata']})
md.convert(text) == '

Lorem Ipsum is simply dummy text.

'
md.Meta == {'title': 'What is Lorem Ipsum?', 'categories': ['Lorem Ipsum', 'Stupid content']}

Specify a custom YAML loader

By default the full YAML loader is used for parsing, which is insecure when used with untrusted user data. In such cases, you may want to specify a different loader such as yaml.SafeLoader using the extension_configs keyword argument:

import markdown
import yaml

md = markdown.Markdown(extensions=['full_yaml_metadata']}, extension_configs={
        "full_yaml_metadata": {
            "yaml_loader": yaml.SafeLoader,
        },
    },
)

Development and contribution

  • install project dependencies
python setup.py develop
  • install linting, formatting and testing tools
pip install -r requirements.txt
  • run tests
pytest
  • run linters
flake8
mypy ./
black --check ./
  • feel free to contribute!
Comments
  • Move setup_requires to pyproject.yaml

    Move setup_requires to pyproject.yaml

    When build system (setuptools) requirements are specified in setup.py, they end up being installed by distutils, even when pip installing. Because distutils is bit-rotting, it doesn't work with system installed openssl.

    Locally, for me, that means distutils doesn't know about some SSL CA certs, and as such, a pip install markdown-full-yaml-metadata will fail when trying to install setuptools_markdown due to being unable to validate the SSL cert (I am behind a coorporate proxy which MITMs all traffic and resigns with a local cert).

    Moving the setup_requires deps to pyproject.toml fixes this - I suggest something like this:

    [build-system]
    # Minimum requirements for the build system to execute.
    requires = ["setuptools>=36.6", "setuptools_markdown", "wheel",]
    build-backend = "setuptools.build_meta"
    
    opened by jonathanunderwood 16
  • Fixes9+10

    Fixes9+10

    change to README.md on how extension is invoked in python interactive shell closes #9 change to full_yaml_metadata.py to makeExtension parameters closes #10

    opened by philbarker 9
  • RFE: don't pin dependencies exactly

    RFE: don't pin dependencies exactly

    This extension pins every dependency exactly. That's a fine strategy for an end-user application, but for a library it's not really the right thing to do, as you inevitably end up with conflicting dependencies in a project that consumes the library. Please would you consider having more liberal dependencies (eg. markdown>3.0, rather than markdown==3.0.1) for example.

    Thanks for a great extension, by the way!

    opened by jonathanunderwood 4
  • Allow specifing custom loader using configuration option

    Allow specifing custom loader using configuration option

    Currently there is no way to use the yaml.BaseLoader for example. The full loader is unsafe for arbitrary user data and also converts strings like 2020-01-01 10:00:00 to datetime.datetime objects, which might be undesired.

    opened by Holzhaus 1
  • url for pypi at top of page is wrong

    url for pypi at top of page is wrong

    Website link in project description for pypi is wrong https://pypi.python.org/pypi/makrdown… ('makr', should be 'mark') -- I guess this worked before fixing #4

    opened by philbarker 1
  • KeyError: 'configs' on running

    KeyError: 'configs' on running

    After getting extension to load properly, got error:

    >>> import markdown
    >>> md = markdown.Markdown(extensions=['full_yaml_metadata'])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 100, in __init__
        configs=kwargs.get('extension_configs', {}))
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 126, in registerExtensions
        ext = self.build_extension(ext, configs.get(ext, {}))
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 181, in build_extension
        return module.makeExtension(**configs)
      File "/home/phil/Share/Projects/full-yaml-test/python-markdown-full-yaml-metadata/full_yaml_metadata.py", line 45, in makeExtension
        return FullYamlMetadataExtension(configs=configs)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 42, in __init__
        self.setConfigs(kwargs)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 73, in setConfigs
        self.setConfig(key, value)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 61, in setConfig
        if isinstance(self.config[key][0], bool):
    KeyError: 'configs'
    

    I fixed by changing

    def makeExtension(configs: dict={}):
        return FullYamlMetadataExtension(configs=configs)
    

    to

    def makeExtension(*args, **kwargs):
        return FullYamlMetadataExtension(*args, **kwargs)
    

    Hope this helps.

    opened by philbarker 0
  • md = markdown.Markdown(['full_yaml_metadata']) didn't work for me

    md = markdown.Markdown(['full_yaml_metadata']) didn't work for me

    Basic usage on pypi invokes extension with md = markdown.Markdown(['full_yaml_metadata'])

    I got the error:

    >>> md = markdown.Markdown(['full_yaml_metadata'])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: __init__() takes 1 positional argument but 2 were given
    

    But this did work (or at least gave another error) md = markdown.Markdown(extensions=['full_yaml_metadata'])

    opened by philbarker 0
  • Support space after metadata delimitation

    Support space after metadata delimitation

    Hello ! I propose to add the support of spaces after metadata delimiting. Currently if a markdown file contains metadata and a space after the metadata delimiter, the markdown parser crashes. Example :

    ---
    title: What is Lorem Ipsum?
    ---        
    Lorem Ipsum is simply dummy text.
    

    There are spaces after the end of metadata declaration (the third line is equal to '--- '). If we try to parse the example above with the 'full_yaml_metadata' plugin we get the following error :

    E           yaml.composer.ComposerError: expected a single document in the stream
    E             in "<unicode string>", line 1, column 1:
    E               title: What is Lorem Ipsum?
    E               ^
    E           but found another document
    E             in "<unicode string>", line 2, column 1:
    E               ---        
    E               ^
    

    So I propose this little correction, hoping that you will accept it. If you have any remark on the code or on the test don't hesitate to sharing them with me, I would be glad to correct it.

    opened by Ricardaux 2
  • Using pre-commit hooks

    Using pre-commit hooks

    I noticed that some parts of the code don't pass your linter checks. This is common for projects with only one main developer where you sometimes push directly instead of opening a PR. I suggest to use pre-commit, which checks that all linters pass before committing. It also takes care dev dependency installation in a venv (so you can remove dev dependencies from requirements.txt.

    opened by Holzhaus 0
  • Add option to attempt splitting metadata using `\n\n` if metadata delimiters are missing

    Add option to attempt splitting metadata using `\n\n` if metadata delimiters are missing

    This might work for very simple metadata, like:

    title: Foo bar
    date: 2020-01-01 10:00:00
    
    This is text
    

    If the metadata delimiters are not found and the option is enabled, the plugin could do something like this:

    meta, sep, content = text.partition("\n\n")
    try:
        metadata = yaml.load(meta, Loader=...)
    except yaml.error.YAMLError:
        content = text
        metadata = {}
    else:
        # Prevent false-positives if the text does not begin with a metadata block
        if isinstance(metadata, str):
            metadata = {}
            content = text
    
    self.md.Meta = metadata
    return content
    opened by Holzhaus 3
Releases(2.1.0)
Owner
Nikita Sivakov
New World Disorder
Nikita Sivakov
EasyModerationKit is an open-source framework designed to moderate and filter inappropriate content.

EasyModerationKit is a public transparency statement. It declares any repositories and legalities used in the EasyModeration system. It allows for implementing EasyModeration into an advanced charact

Aarav 1 Jan 16, 2022
level2-data-annotation_cv-level2-cv-15 created by GitHub Classroom

[AI Tech 3기 Level2 P Stage] 글자 검출 대회 팀원 소개 김규리_T3016 박정현_T3094 석진혁_T3109 손정균_T3111 이현진_T3174 임종현_T3182 Overview OCR (Optimal Character Recognition) 기술

6 Jun 10, 2022
Obmovies - A short guide on setting up the system and environment dependencies required for ob's Movies database

Obmovies - A short guide on setting up the system and environment dependencies required for ob's Movies database

1 Jan 04, 2022
Seamlessly integrate pydantic models in your Sphinx documentation.

Seamlessly integrate pydantic models in your Sphinx documentation.

Franz Wöllert 71 Dec 26, 2022
Leetcode Practice

LeetCode Practice Description This is my LeetCode Practice. Visit LeetCode Website for detailed question description. The code in this repository has

Leo Hsieh 75 Dec 27, 2022
Manage your WordPress installation directly from SublimeText SideBar and Command Palette.

WordpressPluginManager Manage your WordPress installation directly from SublimeText SideBar and Command Palette. Installation Dependencies You will ne

Art-i desenvolvimento 1 Dec 14, 2021
This programm checks your knowlege about the capital of Japan

Introduction This programm checks your knowlege about the capital of Japan. Now, what does it actually do? After you run the programm you get asked wh

1 Dec 16, 2021
100 Days of Code Learning program to keep a habit of coding daily and learn things at your own pace with help from our remote community.

100 Days of Code Learning program to keep a habit of coding daily and learn things at your own pace with help from our remote community.

Git Commit Show by Invide 41 Dec 30, 2022
Data science on SDGs - Udemy Online Course Material: Data Science on Sustainable Development Goals

Data Science on Sustainable Development Goals (SDGs) Udemy Online Course Material: Data Science on Sustainable Development Goals https://bit.ly/data_s

Frank Kienle 1 Jan 04, 2022
Quickly download, clean up, and install public datasets into a database management system

Finding data is one thing. Getting it ready for analysis is another. Acquiring, cleaning, standardizing and importing publicly available data is time

Weecology 274 Jan 04, 2023
A python package to avoid writing and maintaining duplicated python docstrings.

docstring-inheritance is a python package to avoid writing and maintaining duplicated python docstrings.

Antoine Dechaume 15 Dec 07, 2022
Run `black` on python code blocks in documentation files

blacken-docs Run black on python code blocks in documentation files. install pip install blacken-docs usage blacken-docs provides a single executable

Anthony Sottile 460 Dec 23, 2022
Bring RGB to life in Neovim

Bring RGB to life in Neovim Change your RGB devices' color depending on Neovim's mode. Fast and asynchronous plugin to live your vim-life to the fulle

Antoine 40 Oct 27, 2022
A collection and example code of every topic you need to know about in the basics of Python.

The Python Beginners Guide: Master The Python Basics Tonight This guide is a collection of every topic you need to know about in the basics of Python.

Ahmed Baari 1 Dec 19, 2021
A markdown wiki and dashboarding system for Datasette

datasette-notebook A markdown wiki and dashboarding system for Datasette This is an experimental alpha and everything about it is likely to change. In

Simon Willison 19 Apr 20, 2022
layout-parser 3.4k Dec 30, 2022
Netbox Dns is a netbox plugin for managing zone, nameserver and record inventory.

Netbox DNS Netbox Dns is a netbox plugin for managing zone, nameserver and record inventory. Features Manage zones (domains) you have. Manage nameserv

Aurora Research Lab 155 Jan 06, 2023
A python package to import files from an adjacent folder

EasyImports About EasyImports is a python package that allows users to easily access and import files from sister folders: f.ex: - Project - Folde

1 Jun 22, 2022
epub2sphinx is a tool to convert epub files to ReST for Sphinx

epub2sphinx epub2sphinx is a tool to convert epub files to ReST for Sphinx. It uses Pandoc for converting HTML data inside epub files into ReST. It cr

Nihaal 8 Dec 15, 2022
More detailed upload statistics for Nicotine+

More Upload Statistics A small plugin for Nicotine+ 3.1+ to create more detailed upload statistics. ⚠ No data previous to enabling this plugin will be

Nick 1 Dec 17, 2021