YAML metadata extension for Python-Markdown

Overview

YAML metadata extension for Python-Markdown

Build Status Coverage Status Code style: black Python versions PyPi

This extension adds YAML meta data handling to markdown with all YAML features.

As in the original, metadata is parsed but not used in processing.

Metadata parsed as is by PyYaml and without additional transformations, so this plugin is not compatible with original Meta-Data extension.

Basic Usage

Lorem Ipsum is simply dummy text.

' md.Meta == {'title': 'What is Lorem Ipsum?', 'categories': ['Lorem Ipsum', 'Stupid content']}">
import markdown


text = """---
title: What is Lorem Ipsum?
categories:
  - Lorem Ipsum
  - Stupid content
...

Lorem Ipsum is simply dummy text.
"""

md = markdown.Markdown(extensions=['full_yaml_metadata']})
md.convert(text) == '

Lorem Ipsum is simply dummy text.

'
md.Meta == {'title': 'What is Lorem Ipsum?', 'categories': ['Lorem Ipsum', 'Stupid content']}

Specify a custom YAML loader

By default the full YAML loader is used for parsing, which is insecure when used with untrusted user data. In such cases, you may want to specify a different loader such as yaml.SafeLoader using the extension_configs keyword argument:

import markdown
import yaml

md = markdown.Markdown(extensions=['full_yaml_metadata']}, extension_configs={
        "full_yaml_metadata": {
            "yaml_loader": yaml.SafeLoader,
        },
    },
)

Development and contribution

  • install project dependencies
python setup.py develop
  • install linting, formatting and testing tools
pip install -r requirements.txt
  • run tests
pytest
  • run linters
flake8
mypy ./
black --check ./
  • feel free to contribute!
Comments
  • Move setup_requires to pyproject.yaml

    Move setup_requires to pyproject.yaml

    When build system (setuptools) requirements are specified in setup.py, they end up being installed by distutils, even when pip installing. Because distutils is bit-rotting, it doesn't work with system installed openssl.

    Locally, for me, that means distutils doesn't know about some SSL CA certs, and as such, a pip install markdown-full-yaml-metadata will fail when trying to install setuptools_markdown due to being unable to validate the SSL cert (I am behind a coorporate proxy which MITMs all traffic and resigns with a local cert).

    Moving the setup_requires deps to pyproject.toml fixes this - I suggest something like this:

    [build-system]
    # Minimum requirements for the build system to execute.
    requires = ["setuptools>=36.6", "setuptools_markdown", "wheel",]
    build-backend = "setuptools.build_meta"
    
    opened by jonathanunderwood 16
  • Fixes9+10

    Fixes9+10

    change to README.md on how extension is invoked in python interactive shell closes #9 change to full_yaml_metadata.py to makeExtension parameters closes #10

    opened by philbarker 9
  • RFE: don't pin dependencies exactly

    RFE: don't pin dependencies exactly

    This extension pins every dependency exactly. That's a fine strategy for an end-user application, but for a library it's not really the right thing to do, as you inevitably end up with conflicting dependencies in a project that consumes the library. Please would you consider having more liberal dependencies (eg. markdown>3.0, rather than markdown==3.0.1) for example.

    Thanks for a great extension, by the way!

    opened by jonathanunderwood 4
  • Allow specifing custom loader using configuration option

    Allow specifing custom loader using configuration option

    Currently there is no way to use the yaml.BaseLoader for example. The full loader is unsafe for arbitrary user data and also converts strings like 2020-01-01 10:00:00 to datetime.datetime objects, which might be undesired.

    opened by Holzhaus 1
  • url for pypi at top of page is wrong

    url for pypi at top of page is wrong

    Website link in project description for pypi is wrong https://pypi.python.org/pypi/makrdown… ('makr', should be 'mark') -- I guess this worked before fixing #4

    opened by philbarker 1
  • KeyError: 'configs' on running

    KeyError: 'configs' on running

    After getting extension to load properly, got error:

    >>> import markdown
    >>> md = markdown.Markdown(extensions=['full_yaml_metadata'])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 100, in __init__
        configs=kwargs.get('extension_configs', {}))
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 126, in registerExtensions
        ext = self.build_extension(ext, configs.get(ext, {}))
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 181, in build_extension
        return module.makeExtension(**configs)
      File "/home/phil/Share/Projects/full-yaml-test/python-markdown-full-yaml-metadata/full_yaml_metadata.py", line 45, in makeExtension
        return FullYamlMetadataExtension(configs=configs)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 42, in __init__
        self.setConfigs(kwargs)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 73, in setConfigs
        self.setConfig(key, value)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 61, in setConfig
        if isinstance(self.config[key][0], bool):
    KeyError: 'configs'
    

    I fixed by changing

    def makeExtension(configs: dict={}):
        return FullYamlMetadataExtension(configs=configs)
    

    to

    def makeExtension(*args, **kwargs):
        return FullYamlMetadataExtension(*args, **kwargs)
    

    Hope this helps.

    opened by philbarker 0
  • md = markdown.Markdown(['full_yaml_metadata']) didn't work for me

    md = markdown.Markdown(['full_yaml_metadata']) didn't work for me

    Basic usage on pypi invokes extension with md = markdown.Markdown(['full_yaml_metadata'])

    I got the error:

    >>> md = markdown.Markdown(['full_yaml_metadata'])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: __init__() takes 1 positional argument but 2 were given
    

    But this did work (or at least gave another error) md = markdown.Markdown(extensions=['full_yaml_metadata'])

    opened by philbarker 0
  • Support space after metadata delimitation

    Support space after metadata delimitation

    Hello ! I propose to add the support of spaces after metadata delimiting. Currently if a markdown file contains metadata and a space after the metadata delimiter, the markdown parser crashes. Example :

    ---
    title: What is Lorem Ipsum?
    ---        
    Lorem Ipsum is simply dummy text.
    

    There are spaces after the end of metadata declaration (the third line is equal to '--- '). If we try to parse the example above with the 'full_yaml_metadata' plugin we get the following error :

    E           yaml.composer.ComposerError: expected a single document in the stream
    E             in "<unicode string>", line 1, column 1:
    E               title: What is Lorem Ipsum?
    E               ^
    E           but found another document
    E             in "<unicode string>", line 2, column 1:
    E               ---        
    E               ^
    

    So I propose this little correction, hoping that you will accept it. If you have any remark on the code or on the test don't hesitate to sharing them with me, I would be glad to correct it.

    opened by Ricardaux 2
  • Using pre-commit hooks

    Using pre-commit hooks

    I noticed that some parts of the code don't pass your linter checks. This is common for projects with only one main developer where you sometimes push directly instead of opening a PR. I suggest to use pre-commit, which checks that all linters pass before committing. It also takes care dev dependency installation in a venv (so you can remove dev dependencies from requirements.txt.

    opened by Holzhaus 0
  • Add option to attempt splitting metadata using `\n\n` if metadata delimiters are missing

    Add option to attempt splitting metadata using `\n\n` if metadata delimiters are missing

    This might work for very simple metadata, like:

    title: Foo bar
    date: 2020-01-01 10:00:00
    
    This is text
    

    If the metadata delimiters are not found and the option is enabled, the plugin could do something like this:

    meta, sep, content = text.partition("\n\n")
    try:
        metadata = yaml.load(meta, Loader=...)
    except yaml.error.YAMLError:
        content = text
        metadata = {}
    else:
        # Prevent false-positives if the text does not begin with a metadata block
        if isinstance(metadata, str):
            metadata = {}
            content = text
    
    self.md.Meta = metadata
    return content
    opened by Holzhaus 3
Releases(2.1.0)
Owner
Nikita Sivakov
New World Disorder
Nikita Sivakov
Read write method - Read files in various types of formats

一个关于所有格式文件读取的方法 1。 问题描述: 各种各样的文件格式,读写操作非常的麻烦,能够有一种方法,可以整合所有格式的文件,方便用户进行读取和写入。 2

2 Jan 26, 2022
advance python series: Data Classes, OOPs, python

Working With Pydantic - Built-in Data Process ========================== Normal way to process data (reading json file): the normal princiople, it's f

Phung Hưng Binh 1 Nov 08, 2021
Projeto em Python colaborativo para o Bootcamp de Dados do Itaú em parceria com a Lets Code

🧾 lets-code-todo-list por Henrique V. Domingues e Josué Montalvão Projeto em Python colaborativo para o Bootcamp de Dados do Itaú em parceria com a L

Henrique V. Domingues 1 Jan 11, 2022
A collection of online resources to help you on your Tech journey.

Everything Tech Resources & Projects About The Project Coming from an engineering background and looking to up skill yourself on a new field can be di

Mohamed A 396 Dec 31, 2022
A curated list of awesome mathematics resources

A curated list of awesome mathematics resources

Cyrille Rossant 6.7k Jan 05, 2023
Python Advanced --- numpy, decorators, networking

Python Advanced --- numpy, decorators, networking (and more?) Hello everyone 👋 This is the project repo for the "Python Advanced - ..." introductory

Andreas Poehlmann 2 Nov 05, 2021
Documentation of the QR code found on new Austrian ID cards.

Austrian ID Card QR Code This document aims to be a complete documentation of the format used in the QR area on the back of new Austrian ID cards (Per

Gabriel Huber 9 Dec 12, 2022
Make posters from Markdown files.

MkPosters Create posters using Markdown. Supports icons, admonitions, and LaTeX mathematics. At the moment it is restricted to the specific layout of

Patrick Kidger 243 Dec 20, 2022
A `:github:` role for Sphinx

sphinx-github-role A github role for Sphinx. Usage Basic usage MyST: :caption: index.md See {github}`astrojuanlu/sphinx-github-role#1`. reStructuredT

Juan Luis Cano Rodríguez 4 Nov 22, 2022
DocumentPy is a Python application that runs in a command-line interface environment, made for creating HTML documents.

DocumentPy DocumentPy is a Python application that runs in a command-line interface environment, made for creating HTML documents. Usage DocumentPy, a

Lotus 0 Jul 15, 2021
Hasköy is an open-source variable sans-serif typeface family

Hasköy Hasköy is an open-source variable sans-serif typeface family. Designed with powerful opentype features and each weight includes latin-extended

67 Jan 04, 2023
OpenTelemetry Python API and SDK

Getting Started • API Documentation • Getting In Touch (GitHub Discussions) Contributing • Examples OpenTelemetry Python This page describes the Pytho

OpenTelemetry - CNCF 1.1k Jan 08, 2023
xeuledoc - Fetch information about a public Google document.

xeuledoc - Fetch information about a public Google document.

Malfrats Industries 651 Dec 27, 2022
Python document object mapper (load python object from JSON and vice-versa)

lupin is a Python JSON object mapper lupin is meant to help in serializing python objects to JSON and unserializing JSON data to python objects. Insta

Aurélien Amilin 24 Nov 09, 2022
Sphinx Bootstrap Theme

Sphinx Bootstrap Theme This Sphinx theme integrates the Bootstrap CSS / JavaScript framework with various layout options, hierarchical menu navigation

Ryan Roemer 584 Nov 16, 2022
epub2sphinx is a tool to convert epub files to ReST for Sphinx

epub2sphinx epub2sphinx is a tool to convert epub files to ReST for Sphinx. It uses Pandoc for converting HTML data inside epub files into ReST. It cr

Nihaal 8 Dec 15, 2022
Mozilla Campus Club CCEW is a student committee working to spread awareness on Open Source software.

Mozilla Campus Club CCEW is a student committee working to spread awareness on Open Source software. We organize webinars and workshops on different technical topics and making Open Source contributi

Mozilla-Campus-Club-Cummins 8 Jun 15, 2022
Explorative Data Analysis Guidelines

Explorative Data Analysis Get data into a usable format! Find out if the following predictive modeling phase will be successful! Combine everything in

Florian Rohrer 18 Dec 26, 2022
A Power BI/Google Studio Dashboard to analyze previous OTC CatchUps

OTC CatchUp Dashboard A Power BI/Google Studio dashboard analyzing OTC CatchUps. File Contents * ├───data ├───old summaries ─── *.md ├

11 Oct 30, 2022
Speed up Sphinx builds by selectively removing toctrees from some pages

Remove toctrees from Sphinx pages Improve your Sphinx build time by selectively removing TocTree objects from pages. This is useful if your documentat

Executable Books 8 Jan 04, 2023