Seamlessly Connecting Notion Database with Python Pandas DataFrame

Overview

notion-df: Seamlessly Connecting Notion Database with Pandas DataFrame

Please Note: This project is currently in pre-alpha stage. The code are not appropriately documented and tested. Please report any issues you find.

Installation

pip install notion-df

Usage

  • Before starting, please follow the instructions to create a new integration and add it to your Notion page or database.

    • We'll refer Internal Integration Token as the api_key below.
  • Pandas-flavored APIs: Just need to add two additional lines of code:

    import notion_df
    notion_df.pandas() #That's it!
    
    import pandas as pd
    df = pd.read_notion(page_url, api_key=api_key)
    df.to_notion(page_url)
  • Download your Notion table as a pandas DataFrame

    import notion_df
    df = notion_df.load(notion_database_url, api_key=api_key)
    # Equivalent to: df = pd.read_notion(notion_database_url, api_key=api_key)
    df.head()
  • Append a local df to a Notion database:

    import notion_df
    notion_df.upload(df, notion_database_url, title="page-title", api_key=api_key)
    # Equivalent to: df.to_notion(notion_database_url, title="page-title", api_key=api_key)
  • Upload a local df to a newly created database in a Notion page:

    import notion_df
    notion_df.upload(df, notion_page_url, title="page-title", api_key=api_key)
    # Equivalent to: df.to_notion(notion_page_url, title="page-title", api_key=api_key)
  • Tired of typing api_key=api_key each time?

    import notion_df
    notion_df.config(api_key=api_key) # Or set an environment variable `NOTION_API_KEY`
    df = notion_df.load(notion_database_url)
    notion_df.upload(df, notion_page_url, title="page-title")
    # Similarly in pandas APIs: df.to_notion(notion_page_url, title="page-title")

TODOs

  • Add tests for
    • load
    • upload
    • values.py
    • configs.py
    • base.py
  • Better class organizations/namings for *Configs and *Values
Comments
  • Upload df with children

    Upload df with children

    This PR allows uploading page content (aka children) alongside the dataframe.

    How to use it?

    import pandas as pd 
    import notion_df
    from notion_df.blocks import *
    
    child1 = ParagraphBlock(paragraph={"rich_text": RichTextObject.encode_string("A")},)
    child2 = ParagraphBlock(paragraph={"rich_text": RichTextObject.encode_string("B")},)
    
    notion_df.upload(
        df = pd.DataFrame([{"Name": "A"}, {"Name": "B"}]), 
        notion_url = "<>", 
        children=[child1, [child2]], #<- this is the update
        client=notion
    )
    

    Note: Given the experimental nature of this function, we only put it in the upload function but not the pd.to_notion API. We will update when it is appropriately tested.

    opened by lolipopshock 0
  • The tables df columns and schema might not be always consistent

    The tables df columns and schema might not be always consistent

    For example, if you have a relation column inside a table, which points to an relation that the Notion integration doesn't have access to, then

    1. the downloaded data will contain that column
    2. the downloaded schema will not contain that column
    opened by lolipopshock 0
  • Get Page ID?

    Get Page ID?

    Is it possible to get the page id of a database item into the data frame? This way I can map relations without setting resolve_relation_values=True and I won't have a problem with relation values that are named the same.

    opened by thielem 0
  • Error when resolve_relation_values=True

    Error when resolve_relation_values=True

    Hey! In contrast to #26 , I can see rollups an relations when resolve_relation_values =False. When I set resolve_relation_values=True, I get the following error:

    Traceback (most recent call last):
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\indexes\base.py", line 3621, in get_loc
        return self._engine.get_loc(casted_key)
      File "pandas\_libs\index.pyx", line 136, in pandas._libs.index.IndexEngine.get_loc
      File "pandas\_libs\index.pyx", line 163, in pandas._libs.index.IndexEngine.get_loc
      File "pandas\_libs\hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item
      File "pandas\_libs\hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item
    KeyError: 'Beschreibung'
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "a:\xxx\notion_fin_test.py", line 10, in <module>
        df = notion_df.download(page_url, resolve_relation_values=True)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\notion_df\agent.py", line 54, in wrapper
        out = func(client=client, *args, **kwargs)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\notion_df\agent.py", line 216, in download
        relation_df.notion_ids, relation_df[rel_title_col]
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\frame.py", line 3505, in __getitem__
        indexer = self.columns.get_loc(key)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\indexes\base.py", line 3623, in get_loc
        raise KeyError(key) from err
    KeyError: 'Beschreibung'
    
    opened by thielem 1
  • get user information from database

    get user information from database

    When I try to read a databse from notion, the column that shows the name of the mate assigned to this task is shown as a code but not the name. But the property last edited shows the name. Someone knows what is happening ?

    Thanks

    opened by AlmendrosCarmona 0
  • Uploading pages url issue

    Uploading pages url issue

    I like this package and I am starting to use it to automate some of my note taking. I noticed if I want to add a page into database, only full database url works but not the short version database ID. For downloading, both short ID and full url works. This is a minor issue. I mentioned it just in case you want to fix it.

    Thank you! Eric

    opened by yygou 1
Releases(v0.0.5)
  • v0.0.5(Feb 22, 2022)

    What's Changed

    • Better rich text by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/21

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.4...v0.0.5

    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Feb 12, 2022)

    What's Changed

    • Add rollup by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/15
    • Enable relation resolution by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/16
    • Better string length checking #9

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.3...v0.0.4

    Source code(tar.gz)
    Source code(zip)
  • v0.0.3(Feb 1, 2022)

  • v0.0.2(Jan 10, 2022)

    • Better validation for Select Option names #1
    • Allow ignoring errors for uploading #2
    • Handle pagination when downloading #3

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.1...v0.0.2

    Source code(tar.gz)
    Source code(zip)
  • v0.0.1(Jan 6, 2022)

    This release implements the basic function of the notion_df library.

    Usage

    • Before starting, please follow the instructions to create a new integration and add it to your Notion page or database.

      • We'll refer Internal Integration Token as the api_key below.
    • Pandas-flavored APIs: Just need to add two additional lines of code:

      import notion_df
      notion_df.pandas() #That's it!
      
      import pandas as pd
      df = pd.read_notion(page_url, api_key=api_key)
      df.to_notion(page_url)
      
    • Download your Notion table as a pandas DataFrame

      import notion_df
      df = notion_df.load(notion_database_url, api_key=api_key)
      # Equivalent to: df = pd.read_notion(notion_database_url, api_key=api_key)
      df.head()
      
    • Append a local df to a Notion database:

      import notion_df
      notion_df.upload(df, notion_database_url, title="page-title", api_key=api_key)
      # Equivalent to: df.to_notion(notion_database_url, title="page-title", api_key=api_key)
      
    • Upload a local df to a newly created database in a Notion page:

      import notion_df
      notion_df.upload(df, notion_page_url, title="page-title", api_key=api_key)
      # Equivalent to: df.to_notion(notion_page_url, title="page-title", api_key=api_key)
      
    • Tired of typing api_key=api_key each time?

      import notion_df
      notion_df.config(api_key=api_key) # Or set an environment variable `NOTION_API_KEY`
      df = notion_df.load(notion_database_url)
      notion_df.upload(df, notion_page_url, title="page-title")
      # Similarly in pandas APIs: df.to_notion(notion_page_url, title="page-title")
      

    Full Changelog: https://github.com/lolipopshock/notion-df/commits/v0.0.1

    Source code(tar.gz)
    Source code(zip)
Companion "receiver" to matrix-appservice-webhooks for [matrix].

Matrix Webhook Receiver Companion "receiver" to matrix-appservice-webhooks for [matrix]. The purpose of this app is to listen for generic webhook mess

Kim Brose 13 Sep 29, 2022
Bin Checker with Aiogram, Telegram

Bin Checker with Aiogram, Telegram

JEFF 1 Aug 16, 2022
Often discord bots just die, and we hardly find one that is durable

Muitas vezes bots do discord simplesmente morrem, e dificilmente achamos um que seja durável. Então porque não ter um próprio para emergências? Como c

Guilherme Almeida 3 Dec 06, 2022
A Telegram Most Powerful Media Info Bot.

Media Info Bot Support 🚑 Demo For The Bot -Test Our Bot By Clicking The Button Below Deploy To Heroku 🗳 Press the Deploy Button to Get Your Own Bot.

Anonymous 5 May 16, 2022
Asca - Antiscam Discord Bot With Python

asca Antiscam Discord Bot Asca moderates scammers and deletes scam messages Opti

11 Nov 01, 2022
Biblioteca Python que extrai dados de mercado do Bacen (Séries Temporais)

Pybacen This library was developed for economic analysis in the Brazilian scenario (Investments, micro and macroeconomic indicators) Installation Inst

42 Jan 05, 2023
Console BeautifulDiscord theme manager

BeautifulDiscord theme manager Console script for downloading & managing Discord .css themes via BeautifulDiscord. Setup Simply run # Linux/MacOS pip3

1 Dec 15, 2022
🐍 The official Python client library for Google's discovery based APIs.

Google API Client This is the Python client library for Google's discovery based APIs. To get started, please see the docs folder. These client librar

Google APIs 6.2k Dec 31, 2022
A bot for Large Fry Larrys

GroupMe Bot Driver This driver is written entirely in Python, and with easy configuration in mind. Using this driver, you'll be able to monitor multip

1 Oct 25, 2021
The most versatile torrent leecher and youtube-dl bot for telegram

TorToolkit Telegram So basically Tortoolkit is aimed to be the most versatile torrent leecher and youtube-dl bot for telegram. This bot is highly cust

αвιנтн 1 Nov 11, 2021
An Amazon Price Tracker app helps you to buy which product you want within sale price by sending an E-Mail.

Amazon Price Tracker An Amazon Price Tracker app helps you to buy which product you want within sale price by sending an E-Mail. Installing Download t

Aytaç Kaşoğlu 2 Feb 10, 2022
A Git Alert Bot - Github Integration for Pyrogram & Telethon

Yet Another GitAlertBot Inspired From @Pokurt's GitGram Run Bot: Local Host Git Clone Repo : For Telethon Version : git clone https://github.com/DevsE

DevsExpo 23 Oct 21, 2022
The Fasted Proxyless Multi-Threaded Discord Call Crasher

Discord-Call-Crasher The Fasted Proxyless Multi-Threaded Discord Call Crasher (Created By Jonah) Requirements / Setting up There will be a few things

8ua 10 Jun 17, 2022
Boto3 code assistance for any API in any IDE, always up to date

botostubs Gives you code assistance for any boto3 API in any IDE. Get started by running pip install botostubs Demo Features PyPI package automaticall

Jeshan Giovanni BABOOA 94 Nov 14, 2022
Discord Remote Administration Tool

Discord Remote Administration Tool

Rdimo 82 Aug 15, 2022
SongLink Discord Bot - Discord bot to share music links easily

SongLink_Discord_Bot Discord bot to share music links easily. Take a link from y

Edgar Lefevre 4 Feb 18, 2022
Discord Token Finder - Find half of your target's token with just their ID.

Discord Token Finder - Find half of your target's token with just their ID.

Ttawi 2 Apr 07, 2022
A collection of automation aids to connect various database systems into Lookout for Metrics

A collection of automation aids to connect various database systems into Lookout for Metrics

AWS Samples 3 Apr 28, 2022
The worst but simplest webhook bot for GitHub and Matrix.

gh-bot gh-bot is maybe the worst (but simplest) Matrix webhook bot for Github. Example of commits: Example of workflow finished: Setting up Server You

Jae Lo Presti 4 Aug 18, 2022
Holly ♥️ is usefull group management bot in telegram 🎋

Holly ♥️ is usefull group management bot in telegram 🎋

Kasun bandara 1 Dec 03, 2021