Seamlessly Connecting Notion Database with Python Pandas DataFrame

Overview

notion-df: Seamlessly Connecting Notion Database with Pandas DataFrame

Please Note: This project is currently in pre-alpha stage. The code are not appropriately documented and tested. Please report any issues you find.

Installation

pip install notion-df

Usage

  • Before starting, please follow the instructions to create a new integration and add it to your Notion page or database.

    • We'll refer Internal Integration Token as the api_key below.
  • Pandas-flavored APIs: Just need to add two additional lines of code:

    import notion_df
    notion_df.pandas() #That's it!
    
    import pandas as pd
    df = pd.read_notion(page_url, api_key=api_key)
    df.to_notion(page_url)
  • Download your Notion table as a pandas DataFrame

    import notion_df
    df = notion_df.load(notion_database_url, api_key=api_key)
    # Equivalent to: df = pd.read_notion(notion_database_url, api_key=api_key)
    df.head()
  • Append a local df to a Notion database:

    import notion_df
    notion_df.upload(df, notion_database_url, title="page-title", api_key=api_key)
    # Equivalent to: df.to_notion(notion_database_url, title="page-title", api_key=api_key)
  • Upload a local df to a newly created database in a Notion page:

    import notion_df
    notion_df.upload(df, notion_page_url, title="page-title", api_key=api_key)
    # Equivalent to: df.to_notion(notion_page_url, title="page-title", api_key=api_key)
  • Tired of typing api_key=api_key each time?

    import notion_df
    notion_df.config(api_key=api_key) # Or set an environment variable `NOTION_API_KEY`
    df = notion_df.load(notion_database_url)
    notion_df.upload(df, notion_page_url, title="page-title")
    # Similarly in pandas APIs: df.to_notion(notion_page_url, title="page-title")

TODOs

  • Add tests for
    • load
    • upload
    • values.py
    • configs.py
    • base.py
  • Better class organizations/namings for *Configs and *Values
Comments
  • Upload df with children

    Upload df with children

    This PR allows uploading page content (aka children) alongside the dataframe.

    How to use it?

    import pandas as pd 
    import notion_df
    from notion_df.blocks import *
    
    child1 = ParagraphBlock(paragraph={"rich_text": RichTextObject.encode_string("A")},)
    child2 = ParagraphBlock(paragraph={"rich_text": RichTextObject.encode_string("B")},)
    
    notion_df.upload(
        df = pd.DataFrame([{"Name": "A"}, {"Name": "B"}]), 
        notion_url = "<>", 
        children=[child1, [child2]], #<- this is the update
        client=notion
    )
    

    Note: Given the experimental nature of this function, we only put it in the upload function but not the pd.to_notion API. We will update when it is appropriately tested.

    opened by lolipopshock 0
  • The tables df columns and schema might not be always consistent

    The tables df columns and schema might not be always consistent

    For example, if you have a relation column inside a table, which points to an relation that the Notion integration doesn't have access to, then

    1. the downloaded data will contain that column
    2. the downloaded schema will not contain that column
    opened by lolipopshock 0
  • Get Page ID?

    Get Page ID?

    Is it possible to get the page id of a database item into the data frame? This way I can map relations without setting resolve_relation_values=True and I won't have a problem with relation values that are named the same.

    opened by thielem 0
  • Error when resolve_relation_values=True

    Error when resolve_relation_values=True

    Hey! In contrast to #26 , I can see rollups an relations when resolve_relation_values =False. When I set resolve_relation_values=True, I get the following error:

    Traceback (most recent call last):
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\indexes\base.py", line 3621, in get_loc
        return self._engine.get_loc(casted_key)
      File "pandas\_libs\index.pyx", line 136, in pandas._libs.index.IndexEngine.get_loc
      File "pandas\_libs\index.pyx", line 163, in pandas._libs.index.IndexEngine.get_loc
      File "pandas\_libs\hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item
      File "pandas\_libs\hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item
    KeyError: 'Beschreibung'
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "a:\xxx\notion_fin_test.py", line 10, in <module>
        df = notion_df.download(page_url, resolve_relation_values=True)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\notion_df\agent.py", line 54, in wrapper
        out = func(client=client, *args, **kwargs)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\notion_df\agent.py", line 216, in download
        relation_df.notion_ids, relation_df[rel_title_col]
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\frame.py", line 3505, in __getitem__
        indexer = self.columns.get_loc(key)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\indexes\base.py", line 3623, in get_loc
        raise KeyError(key) from err
    KeyError: 'Beschreibung'
    
    opened by thielem 1
  • get user information from database

    get user information from database

    When I try to read a databse from notion, the column that shows the name of the mate assigned to this task is shown as a code but not the name. But the property last edited shows the name. Someone knows what is happening ?

    Thanks

    opened by AlmendrosCarmona 0
  • Uploading pages url issue

    Uploading pages url issue

    I like this package and I am starting to use it to automate some of my note taking. I noticed if I want to add a page into database, only full database url works but not the short version database ID. For downloading, both short ID and full url works. This is a minor issue. I mentioned it just in case you want to fix it.

    Thank you! Eric

    opened by yygou 1
Releases(v0.0.5)
  • v0.0.5(Feb 22, 2022)

    What's Changed

    • Better rich text by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/21

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.4...v0.0.5

    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Feb 12, 2022)

    What's Changed

    • Add rollup by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/15
    • Enable relation resolution by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/16
    • Better string length checking #9

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.3...v0.0.4

    Source code(tar.gz)
    Source code(zip)
  • v0.0.3(Feb 1, 2022)

  • v0.0.2(Jan 10, 2022)

    • Better validation for Select Option names #1
    • Allow ignoring errors for uploading #2
    • Handle pagination when downloading #3

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.1...v0.0.2

    Source code(tar.gz)
    Source code(zip)
  • v0.0.1(Jan 6, 2022)

    This release implements the basic function of the notion_df library.

    Usage

    • Before starting, please follow the instructions to create a new integration and add it to your Notion page or database.

      • We'll refer Internal Integration Token as the api_key below.
    • Pandas-flavored APIs: Just need to add two additional lines of code:

      import notion_df
      notion_df.pandas() #That's it!
      
      import pandas as pd
      df = pd.read_notion(page_url, api_key=api_key)
      df.to_notion(page_url)
      
    • Download your Notion table as a pandas DataFrame

      import notion_df
      df = notion_df.load(notion_database_url, api_key=api_key)
      # Equivalent to: df = pd.read_notion(notion_database_url, api_key=api_key)
      df.head()
      
    • Append a local df to a Notion database:

      import notion_df
      notion_df.upload(df, notion_database_url, title="page-title", api_key=api_key)
      # Equivalent to: df.to_notion(notion_database_url, title="page-title", api_key=api_key)
      
    • Upload a local df to a newly created database in a Notion page:

      import notion_df
      notion_df.upload(df, notion_page_url, title="page-title", api_key=api_key)
      # Equivalent to: df.to_notion(notion_page_url, title="page-title", api_key=api_key)
      
    • Tired of typing api_key=api_key each time?

      import notion_df
      notion_df.config(api_key=api_key) # Or set an environment variable `NOTION_API_KEY`
      df = notion_df.load(notion_database_url)
      notion_df.upload(df, notion_page_url, title="page-title")
      # Similarly in pandas APIs: df.to_notion(notion_page_url, title="page-title")
      

    Full Changelog: https://github.com/lolipopshock/notion-df/commits/v0.0.1

    Source code(tar.gz)
    Source code(zip)
Maubot azuracast - A maubot to fetch data from your radio station

Maubot Azuracast A maubot to fetch data from your radio station Setup Configure

3 Mar 14, 2022
Lumi-Bot - Discord bot that fetches cryptocurrency prices utilizing CoinGeko API

Lumi-Bot Discord bot that fetches and monitors cryptocurrency prices utilizing C

Diego Castro 2 Oct 08, 2022
VC-Music , Playing music without bot.

VC-Userbot A Telegram Userbot to play or streaming Audio and Video songs / files in Telegram Voice Chats. It's made with PyTgCalls and Pyrogram Requir

RioProjectX 8 Aug 04, 2022
A project that automatically sends you a Medium article on a topic of your choosing to your email address daily.

Daily Article from Medium ✏️ About A project that automatically sends you a Medium article on a topic of your choosing to your email address daily. No

Orhan Emre Dikicigil 2 Apr 27, 2022
An async-ready Python wrapper around FerrisChat's API.

FerrisWheel An async-ready Python wrapper around FerrisChat's API. Installation Instructions Linux: $ python3.9 -m pip install -U ferriswheel Python 3

FerrisChat 8 Feb 08, 2022
"zpool iostats" for humans; find the slow parts of your ZFS pool

Getting the gist of zfs statistics vpool-demo.mp4 The ZFS command "zpool iostat" provides a histogram listing of how often it takes to do things in pa

Chad 57 Oct 24, 2022
tgEasy's Official Assistant Bot and Example Bot

tgEasy Assistant The assistant bot that helps people with tgEasy directly on Telegram. This repository contains the source code of @tgEasyRobot and th

Divide Projects™ 4 Dec 26, 2022
Automation for grabbing keys from a Linux host. Useful during red team exercises to quickly help assess what access to a Linux host can lead to.

keygrabber Automation for grabbing keys from a Linux host. This can be helpful during red team exercises when you gain access to a Linux host and want

Cedric Owens 14 Sep 27, 2022
An Open Source ALL-In-One Telegram RoBot, that can do lot of things.

An Open Source ALL-In-One Telegram RoBot, that can do lot of things.

JOBIN 0 Dec 01, 2021
A simple bot discord in PY with moderation controls

Voila un bot discord en py avec les commandes simples de modération tout simplement faut changer les lignes 70 vous mettez votre token de votre bot 53

Ethan 1 Nov 20, 2021
A Telegram bot to transcribe audio, video and image into text.

Transcriber Bot A Telegram bot to transcribe audio, video and image into text. Deploy to Heroku Local Deploying Install the FFmpeg. Make sure you have

10 Dec 19, 2022
Powerful spammer bots for telegram made with python and telethon.

Powerful spammer bots for telegram made with python and telethon. We can deploy upto 70 bots at a time.

32 Dec 15, 2022
Instagram - Instagram Account Reporting Tool

Instagram Instagram Account Reporting Tool Installation On Termux $ apt update $

Aryan 6 Nov 18, 2022
[Fullversion]Web3 Pancakeswap Sniper bot written in python3.

🚀 Pancakeswap BSC Sniper Bot 🚀 Web3 Pancakeswap Sniper && Take Profit/StopLose bot written in python3, Please note the license conditions! The secon

21 Dec 11, 2022
iCloudPy is a simple iCloud webservices wrapper library written in Python

iCloudPy 🤟 Please star this repository if you end up using the library. It will help me continue supporting this product. 🙏 iCloudPy is a simple iCl

Mandar Patil 49 Dec 26, 2022
Easy and simple, Telegram Bot to Show alert when some edits a message in Group

Edit-Message-Alert Just a simple bot to show alert when someone edits a message sent by them, Just 17 Lines of Code These codes are for those who incu

Nuhman Pk 6 Dec 15, 2021
Python wrapper for Interactive Brokers Client Portal Web API

EasyIB: Unofficial Wrapper for Interactive Brokers API EasyIB is an unofficial python wrapper for Interactive Brokers Client Portal Web API. Features

39 Dec 13, 2022
Slack Developer Kit for Python

Python Slack SDK The Slack platform offers several APIs to build apps. Each Slack API delivers part of the capabilities from the platform, so that you

SlackAPI 3.5k Jan 02, 2023
Automatically send commands to send Twitch followers to any Twitch account.

Automatically send commands to send Twitch followers to any Twitch account. You just need to be in a Twitch follow bot Discord server!

Thomas Keig 6 Nov 27, 2022
Gathers data and displays metrics related to climate change and resource depletion on a PowerBI report.

Apocalypse Status Dashboard Purpose Climate change and resource depletion are grave long-term dangers. The code in this repository will pull data from

Summer Is Here 1 Nov 12, 2021