YAML metadata extension for Python-Markdown

Overview

YAML metadata extension for Python-Markdown

Build Status Coverage Status Code style: black Python versions PyPi

This extension adds YAML meta data handling to markdown with all YAML features.

As in the original, metadata is parsed but not used in processing.

Metadata parsed as is by PyYaml and without additional transformations, so this plugin is not compatible with original Meta-Data extension.

Basic Usage

Lorem Ipsum is simply dummy text.

' md.Meta == {'title': 'What is Lorem Ipsum?', 'categories': ['Lorem Ipsum', 'Stupid content']}">
import markdown


text = """---
title: What is Lorem Ipsum?
categories:
  - Lorem Ipsum
  - Stupid content
...

Lorem Ipsum is simply dummy text.
"""

md = markdown.Markdown(extensions=['full_yaml_metadata']})
md.convert(text) == '

Lorem Ipsum is simply dummy text.

'
md.Meta == {'title': 'What is Lorem Ipsum?', 'categories': ['Lorem Ipsum', 'Stupid content']}

Specify a custom YAML loader

By default the full YAML loader is used for parsing, which is insecure when used with untrusted user data. In such cases, you may want to specify a different loader such as yaml.SafeLoader using the extension_configs keyword argument:

import markdown
import yaml

md = markdown.Markdown(extensions=['full_yaml_metadata']}, extension_configs={
        "full_yaml_metadata": {
            "yaml_loader": yaml.SafeLoader,
        },
    },
)

Development and contribution

  • install project dependencies
python setup.py develop
  • install linting, formatting and testing tools
pip install -r requirements.txt
  • run tests
pytest
  • run linters
flake8
mypy ./
black --check ./
  • feel free to contribute!
Comments
  • Move setup_requires to pyproject.yaml

    Move setup_requires to pyproject.yaml

    When build system (setuptools) requirements are specified in setup.py, they end up being installed by distutils, even when pip installing. Because distutils is bit-rotting, it doesn't work with system installed openssl.

    Locally, for me, that means distutils doesn't know about some SSL CA certs, and as such, a pip install markdown-full-yaml-metadata will fail when trying to install setuptools_markdown due to being unable to validate the SSL cert (I am behind a coorporate proxy which MITMs all traffic and resigns with a local cert).

    Moving the setup_requires deps to pyproject.toml fixes this - I suggest something like this:

    [build-system]
    # Minimum requirements for the build system to execute.
    requires = ["setuptools>=36.6", "setuptools_markdown", "wheel",]
    build-backend = "setuptools.build_meta"
    
    opened by jonathanunderwood 16
  • Fixes9+10

    Fixes9+10

    change to README.md on how extension is invoked in python interactive shell closes #9 change to full_yaml_metadata.py to makeExtension parameters closes #10

    opened by philbarker 9
  • RFE: don't pin dependencies exactly

    RFE: don't pin dependencies exactly

    This extension pins every dependency exactly. That's a fine strategy for an end-user application, but for a library it's not really the right thing to do, as you inevitably end up with conflicting dependencies in a project that consumes the library. Please would you consider having more liberal dependencies (eg. markdown>3.0, rather than markdown==3.0.1) for example.

    Thanks for a great extension, by the way!

    opened by jonathanunderwood 4
  • Allow specifing custom loader using configuration option

    Allow specifing custom loader using configuration option

    Currently there is no way to use the yaml.BaseLoader for example. The full loader is unsafe for arbitrary user data and also converts strings like 2020-01-01 10:00:00 to datetime.datetime objects, which might be undesired.

    opened by Holzhaus 1
  • url for pypi at top of page is wrong

    url for pypi at top of page is wrong

    Website link in project description for pypi is wrong https://pypi.python.org/pypi/makrdown… ('makr', should be 'mark') -- I guess this worked before fixing #4

    opened by philbarker 1
  • KeyError: 'configs' on running

    KeyError: 'configs' on running

    After getting extension to load properly, got error:

    >>> import markdown
    >>> md = markdown.Markdown(extensions=['full_yaml_metadata'])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 100, in __init__
        configs=kwargs.get('extension_configs', {}))
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 126, in registerExtensions
        ext = self.build_extension(ext, configs.get(ext, {}))
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/core.py", line 181, in build_extension
        return module.makeExtension(**configs)
      File "/home/phil/Share/Projects/full-yaml-test/python-markdown-full-yaml-metadata/full_yaml_metadata.py", line 45, in makeExtension
        return FullYamlMetadataExtension(configs=configs)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 42, in __init__
        self.setConfigs(kwargs)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 73, in setConfigs
        self.setConfig(key, value)
      File "/home/phil/Share/Projects/full-yaml-test/venv/lib/python3.6/site-packages/markdown/extensions/__init__.py", line 61, in setConfig
        if isinstance(self.config[key][0], bool):
    KeyError: 'configs'
    

    I fixed by changing

    def makeExtension(configs: dict={}):
        return FullYamlMetadataExtension(configs=configs)
    

    to

    def makeExtension(*args, **kwargs):
        return FullYamlMetadataExtension(*args, **kwargs)
    

    Hope this helps.

    opened by philbarker 0
  • md = markdown.Markdown(['full_yaml_metadata']) didn't work for me

    md = markdown.Markdown(['full_yaml_metadata']) didn't work for me

    Basic usage on pypi invokes extension with md = markdown.Markdown(['full_yaml_metadata'])

    I got the error:

    >>> md = markdown.Markdown(['full_yaml_metadata'])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: __init__() takes 1 positional argument but 2 were given
    

    But this did work (or at least gave another error) md = markdown.Markdown(extensions=['full_yaml_metadata'])

    opened by philbarker 0
  • Support space after metadata delimitation

    Support space after metadata delimitation

    Hello ! I propose to add the support of spaces after metadata delimiting. Currently if a markdown file contains metadata and a space after the metadata delimiter, the markdown parser crashes. Example :

    ---
    title: What is Lorem Ipsum?
    ---        
    Lorem Ipsum is simply dummy text.
    

    There are spaces after the end of metadata declaration (the third line is equal to '--- '). If we try to parse the example above with the 'full_yaml_metadata' plugin we get the following error :

    E           yaml.composer.ComposerError: expected a single document in the stream
    E             in "<unicode string>", line 1, column 1:
    E               title: What is Lorem Ipsum?
    E               ^
    E           but found another document
    E             in "<unicode string>", line 2, column 1:
    E               ---        
    E               ^
    

    So I propose this little correction, hoping that you will accept it. If you have any remark on the code or on the test don't hesitate to sharing them with me, I would be glad to correct it.

    opened by Ricardaux 2
  • Using pre-commit hooks

    Using pre-commit hooks

    I noticed that some parts of the code don't pass your linter checks. This is common for projects with only one main developer where you sometimes push directly instead of opening a PR. I suggest to use pre-commit, which checks that all linters pass before committing. It also takes care dev dependency installation in a venv (so you can remove dev dependencies from requirements.txt.

    opened by Holzhaus 0
  • Add option to attempt splitting metadata using `\n\n` if metadata delimiters are missing

    Add option to attempt splitting metadata using `\n\n` if metadata delimiters are missing

    This might work for very simple metadata, like:

    title: Foo bar
    date: 2020-01-01 10:00:00
    
    This is text
    

    If the metadata delimiters are not found and the option is enabled, the plugin could do something like this:

    meta, sep, content = text.partition("\n\n")
    try:
        metadata = yaml.load(meta, Loader=...)
    except yaml.error.YAMLError:
        content = text
        metadata = {}
    else:
        # Prevent false-positives if the text does not begin with a metadata block
        if isinstance(metadata, str):
            metadata = {}
            content = text
    
    self.md.Meta = metadata
    return content
    opened by Holzhaus 3
Releases(2.1.0)
Owner
Nikita Sivakov
New World Disorder
Nikita Sivakov
OpenTelemetry Python API and SDK

Getting Started • API Documentation • Getting In Touch (GitHub Discussions) Contributing • Examples OpenTelemetry Python This page describes the Pytho

OpenTelemetry - CNCF 1.1k Jan 08, 2023
Highlight Translator can help you translate the words quickly and accurately.

Highlight Translator can help you translate the words quickly and accurately. By only highlighting, copying, or screenshoting the content you want to translate anywhere on your computer (ex. PDF, PPT

Coolshan 48 Dec 21, 2022
The source code that powers readthedocs.org

Welcome to Read the Docs Purpose Read the Docs hosts documentation for the open source community. It supports Sphinx docs written with reStructuredTex

Read the Docs 7.4k Dec 25, 2022
Show Rubygems description and annotate your code right from Sublime Text.

Gem Description for Sublime Text Show Rubygems description and annotate your code. Just mouse over your Gemfile's gem definitions to show the popup. s

Nando Vieira 2 Dec 19, 2022
EasyModerationKit is an open-source framework designed to moderate and filter inappropriate content.

EasyModerationKit is a public transparency statement. It declares any repositories and legalities used in the EasyModeration system. It allows for implementing EasyModeration into an advanced charact

Aarav 1 Jan 16, 2022
Rust Markdown Parsing Benchmarks

Rust Markdown Parsing Benchmarks This repo tries to assess Rust markdown parsing

Ed Page 1 Aug 24, 2022
swagger-codegen contains a template-driven engine to generate documentation, API clients and server stubs in different languages by parsing your OpenAPI / Swagger definition.

Master (2.4.25-SNAPSHOT): 3.0.31-SNAPSHOT: Maven Central ⭐ ⭐ ⭐ If you would like to contribute, please refer to guidelines and a list of open tasks. ⭐

Swagger 15.2k Dec 31, 2022
Variable Transformer Calculator

✠ VASCO - VAriable tranSformer CalculatOr Software que calcula informações de transformadores feita para a matéria de "Conversão Eletromecânica de Ene

Arthur Cordeiro Andrade 2 Feb 12, 2022
Near Zero-Overhead Python Code Coverage

Slipcover: Near Zero-Overhead Python Code Coverage by Juan Altmayer Pizzorno and Emery Berger at UMass Amherst's PLASMA lab. About Slipcover Slipcover

PLASMA @ UMass 325 Dec 28, 2022
A website for courses of Major Computer Science, NKU

A website for courses of Major Computer Science, NKU

Sakura 0 Oct 06, 2022
Quickly download, clean up, and install public datasets into a database management system

Finding data is one thing. Getting it ready for analysis is another. Acquiring, cleaning, standardizing and importing publicly available data is time

Weecology 274 Jan 04, 2023
Sphinx theme for readthedocs.org

Read the Docs Sphinx Theme This Sphinx theme was designed to provide a great reader experience for documentation users on both desktop and mobile devi

Read the Docs 4.3k Dec 31, 2022
Preview title and other information about links sent to chats.

Link Preview A small plugin for Nicotine+ to display preview information like title and description about links sent in chats. Plugin created with Nic

Nick 0 Sep 05, 2021
Xanadu Quantum Codebook is an experimental, exercise-based introduction to quantum computing using PennyLane.

Xanadu Quantum Codebook The Xanadu Quantum Codebook is an experimental, exercise-based introduction to quantum computing using PennyLane. This reposit

Xanadu 43 Dec 09, 2022
layout-parser 3.4k Dec 30, 2022
OpenAPI (f.k.a Swagger) Specification code generator. Supports C#, PowerShell, Go, Java, Node.js, TypeScript, Python

AutoRest The AutoRest tool generates client libraries for accessing RESTful web services. Input to AutoRest is a spec that describes the REST API usin

Microsoft Azure 4.1k Jan 06, 2023
Valentine-with-Python - A Python program generates an animation of a heart with cool texts of your loved one

Valentine with Python Valentines with Python is a mini fun project I have coded.

Niraj Tiwari 4 Dec 31, 2022
Resource hub for Obsidian resources.

Obsidian Community Vault Welcome! This is an experimental vault that is maintained by the Obsidian community. For best results we recommend downloadin

Obsidian Community 320 Jan 02, 2023
Python Deep Dive Course - Accompanying Materials

Python Deep Dive Various Jupyter notebooks and Python sources associated with my Udemy Python 3 Deep Dive course series: Part 1: Mainly functional pro

Fred Baptiste 1.1k Dec 30, 2022
An interview engine for businesses, interview those who are actually qualified and are worth your time!

easyInterview V0.8B An interview engine for businesses, interview those who are actually qualified and are worth your time! Quick Overview You/the com

Vatsal Shukla 1 Nov 19, 2021