Speed up Sphinx builds by selectively removing toctrees from some pages

Overview

Remove toctrees from Sphinx pages

Improve your Sphinx build time by selectively removing TocTree objects from pages. This is useful if your documentation uses auto-generated API documentation, which generates a lot of stub pages

This extension can be used to remove the sidebar links for just the pages you specify, speed up the build considerably.

Who is this for?

This package is for maintainers that use Sphinx and have really large API documentation (or for some other reason, have a ton of nested pages). If you use a Sphinx theme that contains the entire Table of Contents on every page (e.g., any theme that has "collapsable" sidebar sections), this will slow things down considerably. Use this theme to speed up your builds.

Install

Install the extension via pip:

$ pip install git+https://github.com/executablebooks/sphinx-remove-toctree

and activate it by adding it to your Sphinx conf.py file:

extensions = ["sphinx_remove_toctree"]

Use

In conf.py, provide a list of glob-like paths relative to your documentation root. Each entry should match to pages that should be removed from the sidebar.

For example, the following configuration will remove all pages from the folder api/generated, and the specific page subfolder/page_two.rst:

remove_toctrees_from = ["api/generated/*", "subfolder/page_two.rst"]

This is particularly useful in combination with the autosummary directive, which tends to generate a ton of stub-pages that slows things down.

If you have the following autosummary directive in a page at myfolder/page1.rst:

.. autosummary: datetime.datetime
   :toctree: api_gen

This will generate stub-pages in a myfolder/api_gen/ folder. To remove each of these pages from your sidebar, you would configure this extension like so:

remove_toctrees_from = ["myfolder/api_gen/*"]

Try it with this documentation

This extension doesn't have a hosted documentation page, but there is one in the docs/ folder of this repository. You can use that folder to preview this extension in action.

How this works

Sphinx keeps track of toctree objects to represent the structure of your documentation. These exist in the Sphinx environment object, at env.tocs. There are two places in the build where this is relevant here:

  • Early in the build, Sphinx uses these tocs to ensure that files in your documentation are linked somewhere, and will raise warnings if it finds a file that is not in one of the tocs.
  • Later in the build, Sphinx uses these tocs to build the HTML toctree with links to pages in your documentation. If there are many elements in tocs, it will take a long time to resolve all of these links!

This extension runs after the first step, but before the second step. It removes all the toctree objects that you specify, so that no warnings are raised about missing files, but they are removed from the sidebar and don't slow down your build.

You might also like...
Sphinx Bootstrap Theme

Sphinx Bootstrap Theme This Sphinx theme integrates the Bootstrap CSS / JavaScript framework with various layout options, hierarchical menu navigation

A powerful Sphinx changelog-generating extension.

What is Releases? Releases is a Python (2.7, 3.4+) compatible Sphinx (1.8+) extension designed to help you keep a source control friendly, merge frien

📖  Generate markdown API documentation from Google-style Python docstring. The lazy alternative to Sphinx.
📖 Generate markdown API documentation from Google-style Python docstring. The lazy alternative to Sphinx.

lazydocs Generate markdown API documentation for Google-style Python docstring. Getting Started • Features • Documentation • Support • Contribution •

Seamlessly integrate pydantic models in your Sphinx documentation.
Seamlessly integrate pydantic models in your Sphinx documentation.

Seamlessly integrate pydantic models in your Sphinx documentation.

python package sphinx template

python-package-sphinx-template python-package-sphinx-template

epub2sphinx is a tool to convert epub files to ReST for Sphinx
epub2sphinx is a tool to convert epub files to ReST for Sphinx

epub2sphinx epub2sphinx is a tool to convert epub files to ReST for Sphinx. It uses Pandoc for converting HTML data inside epub files into ReST. It cr

A `:github:` role for Sphinx

sphinx-github-role A github role for Sphinx. Usage Basic usage MyST: :caption: index.md See {github}`astrojuanlu/sphinx-github-role#1`. reStructuredT

This is a template (starter kit) for writing Maturity Work with Sphinx / LaTeX at Collège du Sud

sphinx-tm-template Ce dépôt est un template de base utilisable pour écrire ton travail de maturité dans le séminaire d'informatique du Collège du Sud.

An MkDocs plugin to export content pages as PDF files

MkDocs PDF Export Plugin An MkDocs plugin to export content pages as PDF files The pdf-export plugin will export all markdown pages in your MkDocs rep

Comments
  • Feedback for the package

    Feedback for the package

    This is a quick issue for feedback on the UX of this package.

    It would be great if folks that have lots of API docs could try the following:

    1. Read the instructions / usage at https://github.com/executablebooks/sphinx-remove-toctrees#readme
    2. Install the extension and activate it
    3. Configure it to suppress the API docs you want out of your sidebar
    4. See how it goes and report back!

    Questions to answer:

    • Did the UX and config make sense?
    • Is this extension explained well enough?
    • Is it named and described properly?

    cc @jorisvandenbossche @rgommers @rossbar @stefanv @jrbourbeau in case any want to give this a shot w/ their docs.

    also cc @pradyunsg and @humitos because I think this extension might be useful for the Furo / RTD themes, as it is handy for any theme that uses collapse=False.

    opened by choldgraf 17
  • 0.0.3: pytest is failing and shows warnings warnings

    0.0.3: pytest is failing and shows warnings warnings

    Describe the bug

    I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

    • python3 -sBm build -w --no-isolation
    • because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
    • install .whl file in </install/prefix>
    • run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

    Here is pytest output:

    + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-remove-toctrees-0.0.3-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-remove-toctrees-0.0.3-2.fc35.x86_64/usr/lib/python3.8/site-packages
    + /usr/bin/pytest -ra
    =========================================================================== test session starts ============================================================================
    platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
    rootdir: /home/tkloczko/rpmbuild/BUILD/sphinx-remove-toctrees-0.0.3
    plugins: timeout-2.1.0, vcr-1.0.2
    collected 1 item
    
    sphinx_remove_toctrees/tests/test_build.py .                                                                                                                         [100%]
    
    ============================================================================= warnings summary =============================================================================
    sphinx_remove_toctrees/tests/test_build.py::test_build_html
      /home/tkloczko/rpmbuild/BUILD/sphinx-remove-toctrees-0.0.3/sphinx_remove_toctrees/tests/test_build.py:21: GuessedAtParserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("lxml"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.
    
      The code that caused this warning is on line 21 of the file /home/tkloczko/rpmbuild/BUILD/sphinx-remove-toctrees-0.0.3/sphinx_remove_toctrees/tests/test_build.py. To get rid of this warning, pass the additional argument 'features="lxml"' to the BeautifulSoup constructor.
    
        index = BeautifulSoup(index.read_text())
    
    -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
    ======================================================================= 1 passed, 1 warning in 0.77s =======================================================================
    

    Reproduce the bug

    N/A

    List your environment

    N/A

    bug 
    opened by kloczek 3
Releases(v0.0.3)
Owner
Executable Books
An open collaboration to create executable books with Jupyter
Executable Books
A hack to run custom shell commands when building documentation on Read the Docs.

readthedocs-custom-steps A hack to run custom steps when building documentation on Read the Docs. Important: This module should not be installed outsi

Niklas Rosenstein 5 Feb 22, 2022
Always know what to expect from your data.

Great Expectations Always know what to expect from your data. Introduction Great Expectations helps data teams eliminate pipeline debt, through data t

Great Expectations 7.8k Jan 05, 2023
🐱‍🏍 A curated list of awesome things related to Hugo themes.

awesome-hugo-themes Automated deployment @ 2021-10-12 06:24:07 Asia/Shanghai &sorted=updated Theme Author License GitHub Stars Updated Blonde wamo MIT

13 Dec 12, 2022
Python-slp - Side Ledger Protocol With Python

Side Ledger Protocol Run python-slp node First install Mongo DB and run the mong

Solar 3 Mar 02, 2022
Searches a document for hash tags. Support multiple natural languages. Works in various contexts.

ht-getter Searches a document for hash tags. Supports multiple natural languages. Works in various contexts. This package uses a non-regex approach an

Rairye 1 Mar 01, 2022
Soccerdata - Efficiently scrape soccer data from various sources

SoccerData is a collection of wrappers over soccer data from Club Elo, ESPN, FBr

Pieter Robberechts 195 Jan 04, 2023
An interview engine for businesses, interview those who are actually qualified and are worth your time!

easyInterview V0.8B An interview engine for businesses, interview those who are actually qualified and are worth your time! Quick Overview You/the com

Vatsal Shukla 1 Nov 19, 2021
100 numpy exercises (with solutions)

100 numpy exercises This is a collection of numpy exercises from numpy mailing list, stack overflow, and numpy documentation. I've also created some p

Nicolas P. Rougier 9.5k Dec 30, 2022
This is the data scrapped of all the pitches made up potential startup's to established bussiness tycoons of India with all the details of Investments made, equity share, Name of investor etc.

SharkTankInvestor This is the data scrapped of all the pitches made up potential startup's to established bussiness tycoons of India with all the deta

Subradip Poddar 2 Aug 02, 2022
Data-science-on-gcp - Source code accompanying book: Data Science on the Google Cloud Platform, Valliappa Lakshmanan, O'Reilly 2017

data-science-on-gcp Source code accompanying book: Data Science on the Google Cloud Platform, 2nd Edition Valliappa Lakshmanan O'Reilly, Jan 2022 Bran

Google Cloud Platform 1.2k Dec 28, 2022
Comprehensive Python Cheatsheet

Comprehensive Python Cheatsheet Download text file, Buy PDF, Fork me on GitHub or Check out FAQ. Contents 1. Collections: List, Dictionary, Set, Tuple

Jefferson 1 Jan 23, 2022
300+ Python Interview Questions

300+ Python Interview Questions

Pradeep Kumar 1.1k Jan 02, 2023
This is a repository for "100 days of code challenge" projects. You can reach all projects from beginner to professional which are written in Python.

100 Days of Code It's a challenge that aims to gain code practice and enhance programming knowledge. Day #1 Create a Band Name Generator It's actually

SelenNB 2 May 12, 2022
A Collection of Cheatsheets, Books, Questions, and Portfolio For DS/ML Interview Prep

Here are the sections: Data Science Cheatsheets Data Science EBooks Data Science Question Bank Data Science Case Studies Data Science Portfolio Data J

James Le 2.5k Jan 02, 2023
A Json Schema Generator

JSON Schema Generator Author : Eru Michael About A Json Schema Generator. This is a generic program that: Reads a JSON file similar to what's present

1 Nov 10, 2021
A pluggable API specification generator. Currently supports the OpenAPI Specification (f.k.a. the Swagger specification)..

apispec A pluggable API specification generator. Currently supports the OpenAPI Specification (f.k.a. the Swagger specification). Features Supports th

marshmallow-code 1k Jan 01, 2023
Run `black` on python code blocks in documentation files

blacken-docs Run black on python code blocks in documentation files. install pip install blacken-docs usage blacken-docs provides a single executable

Anthony Sottile 460 Dec 23, 2022
An awesome Data Science repository to learn and apply for real world problems.

AWESOME DATA SCIENCE An open source Data Science repository to learn and apply towards solving real world problems. This is a shortcut path to start s

Academic.io 20.3k Jan 09, 2023
Literate-style documentation generator.

888888b. 888 Y88b 888 888 888 d88P 888 888 .d8888b .d8888b .d88b. 8888888P" 888 888 d88P" d88P" d88""88b 888 888 888

Pycco 808 Dec 27, 2022
Automatic links from code examples to reference documentation

sphinx-codeautolink Automatic links from Python code examples to reference documentation at the flick of a switch! sphinx-codeautolink analyses the co

Felix Hildén 41 Dec 17, 2022