Seamlessly Connecting Notion Database with Python Pandas DataFrame

Overview

notion-df: Seamlessly Connecting Notion Database with Pandas DataFrame

Please Note: This project is currently in pre-alpha stage. The code are not appropriately documented and tested. Please report any issues you find.

Installation

pip install notion-df

Usage

  • Before starting, please follow the instructions to create a new integration and add it to your Notion page or database.

    • We'll refer Internal Integration Token as the api_key below.
  • Pandas-flavored APIs: Just need to add two additional lines of code:

    import notion_df
    notion_df.pandas() #That's it!
    
    import pandas as pd
    df = pd.read_notion(page_url, api_key=api_key)
    df.to_notion(page_url)
  • Download your Notion table as a pandas DataFrame

    import notion_df
    df = notion_df.load(notion_database_url, api_key=api_key)
    # Equivalent to: df = pd.read_notion(notion_database_url, api_key=api_key)
    df.head()
  • Append a local df to a Notion database:

    import notion_df
    notion_df.upload(df, notion_database_url, title="page-title", api_key=api_key)
    # Equivalent to: df.to_notion(notion_database_url, title="page-title", api_key=api_key)
  • Upload a local df to a newly created database in a Notion page:

    import notion_df
    notion_df.upload(df, notion_page_url, title="page-title", api_key=api_key)
    # Equivalent to: df.to_notion(notion_page_url, title="page-title", api_key=api_key)
  • Tired of typing api_key=api_key each time?

    import notion_df
    notion_df.config(api_key=api_key) # Or set an environment variable `NOTION_API_KEY`
    df = notion_df.load(notion_database_url)
    notion_df.upload(df, notion_page_url, title="page-title")
    # Similarly in pandas APIs: df.to_notion(notion_page_url, title="page-title")

TODOs

  • Add tests for
    • load
    • upload
    • values.py
    • configs.py
    • base.py
  • Better class organizations/namings for *Configs and *Values
Comments
  • Upload df with children

    Upload df with children

    This PR allows uploading page content (aka children) alongside the dataframe.

    How to use it?

    import pandas as pd 
    import notion_df
    from notion_df.blocks import *
    
    child1 = ParagraphBlock(paragraph={"rich_text": RichTextObject.encode_string("A")},)
    child2 = ParagraphBlock(paragraph={"rich_text": RichTextObject.encode_string("B")},)
    
    notion_df.upload(
        df = pd.DataFrame([{"Name": "A"}, {"Name": "B"}]), 
        notion_url = "<>", 
        children=[child1, [child2]], #<- this is the update
        client=notion
    )
    

    Note: Given the experimental nature of this function, we only put it in the upload function but not the pd.to_notion API. We will update when it is appropriately tested.

    opened by lolipopshock 0
  • The tables df columns and schema might not be always consistent

    The tables df columns and schema might not be always consistent

    For example, if you have a relation column inside a table, which points to an relation that the Notion integration doesn't have access to, then

    1. the downloaded data will contain that column
    2. the downloaded schema will not contain that column
    opened by lolipopshock 0
  • Get Page ID?

    Get Page ID?

    Is it possible to get the page id of a database item into the data frame? This way I can map relations without setting resolve_relation_values=True and I won't have a problem with relation values that are named the same.

    opened by thielem 0
  • Error when resolve_relation_values=True

    Error when resolve_relation_values=True

    Hey! In contrast to #26 , I can see rollups an relations when resolve_relation_values =False. When I set resolve_relation_values=True, I get the following error:

    Traceback (most recent call last):
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\indexes\base.py", line 3621, in get_loc
        return self._engine.get_loc(casted_key)
      File "pandas\_libs\index.pyx", line 136, in pandas._libs.index.IndexEngine.get_loc
      File "pandas\_libs\index.pyx", line 163, in pandas._libs.index.IndexEngine.get_loc
      File "pandas\_libs\hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item
      File "pandas\_libs\hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item
    KeyError: 'Beschreibung'
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "a:\xxx\notion_fin_test.py", line 10, in <module>
        df = notion_df.download(page_url, resolve_relation_values=True)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\notion_df\agent.py", line 54, in wrapper
        out = func(client=client, *args, **kwargs)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\notion_df\agent.py", line 216, in download
        relation_df.notion_ids, relation_df[rel_title_col]
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\frame.py", line 3505, in __getitem__
        indexer = self.columns.get_loc(key)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\indexes\base.py", line 3623, in get_loc
        raise KeyError(key) from err
    KeyError: 'Beschreibung'
    
    opened by thielem 1
  • get user information from database

    get user information from database

    When I try to read a databse from notion, the column that shows the name of the mate assigned to this task is shown as a code but not the name. But the property last edited shows the name. Someone knows what is happening ?

    Thanks

    opened by AlmendrosCarmona 0
  • Uploading pages url issue

    Uploading pages url issue

    I like this package and I am starting to use it to automate some of my note taking. I noticed if I want to add a page into database, only full database url works but not the short version database ID. For downloading, both short ID and full url works. This is a minor issue. I mentioned it just in case you want to fix it.

    Thank you! Eric

    opened by yygou 1
Releases(v0.0.5)
  • v0.0.5(Feb 22, 2022)

    What's Changed

    • Better rich text by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/21

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.4...v0.0.5

    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Feb 12, 2022)

    What's Changed

    • Add rollup by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/15
    • Enable relation resolution by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/16
    • Better string length checking #9

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.3...v0.0.4

    Source code(tar.gz)
    Source code(zip)
  • v0.0.3(Feb 1, 2022)

  • v0.0.2(Jan 10, 2022)

    • Better validation for Select Option names #1
    • Allow ignoring errors for uploading #2
    • Handle pagination when downloading #3

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.1...v0.0.2

    Source code(tar.gz)
    Source code(zip)
  • v0.0.1(Jan 6, 2022)

    This release implements the basic function of the notion_df library.

    Usage

    • Before starting, please follow the instructions to create a new integration and add it to your Notion page or database.

      • We'll refer Internal Integration Token as the api_key below.
    • Pandas-flavored APIs: Just need to add two additional lines of code:

      import notion_df
      notion_df.pandas() #That's it!
      
      import pandas as pd
      df = pd.read_notion(page_url, api_key=api_key)
      df.to_notion(page_url)
      
    • Download your Notion table as a pandas DataFrame

      import notion_df
      df = notion_df.load(notion_database_url, api_key=api_key)
      # Equivalent to: df = pd.read_notion(notion_database_url, api_key=api_key)
      df.head()
      
    • Append a local df to a Notion database:

      import notion_df
      notion_df.upload(df, notion_database_url, title="page-title", api_key=api_key)
      # Equivalent to: df.to_notion(notion_database_url, title="page-title", api_key=api_key)
      
    • Upload a local df to a newly created database in a Notion page:

      import notion_df
      notion_df.upload(df, notion_page_url, title="page-title", api_key=api_key)
      # Equivalent to: df.to_notion(notion_page_url, title="page-title", api_key=api_key)
      
    • Tired of typing api_key=api_key each time?

      import notion_df
      notion_df.config(api_key=api_key) # Or set an environment variable `NOTION_API_KEY`
      df = notion_df.load(notion_database_url)
      notion_df.upload(df, notion_page_url, title="page-title")
      # Similarly in pandas APIs: df.to_notion(notion_page_url, title="page-title")
      

    Full Changelog: https://github.com/lolipopshock/notion-df/commits/v0.0.1

    Source code(tar.gz)
    Source code(zip)
Python library for Seeedstudio Grove devices

grove.py Python library for Seeedstudio Grove Devices on embeded Linux platform, especially good on below platforms: Coral Dev Board (Wiki) NVIDIA Jet

Seeed Studio 123 Dec 17, 2022
Send alert to telegram use telegram cli

Run standalone: Rename conf.yml.example to conf.yml Change block cli(Add your api_id and hash) Install requirements.txt Run python AlertManagerTG.py I

Eugene Arkharov 1 Nov 12, 2021
This is RequestTrackerBot and it used for tracking request made by user in a group

This is a Request Tracker Bot repo, It is for those who upload content like movies, anime, etc. It can be used for tracking request of content that your members asked for.

Abhijeet 27 Dec 29, 2022
A Powerful, Smart And Advance Group Manager ... Written with AioGram , Pyrogram and Telethon...

❤️ Shadow ❤️ A Powerful, Smart And Advance Group Manager ... Written with AioGram , Pyrogram and Telethon... ⭐️ Thanks to everyone who starred Shadow,

TeamShadow 17 Oct 21, 2022
🛰️ Scripts démontrant l'utilisation de l'imagerie RADARSAT-1 à partir d'un seau AWS | 🛰️ Scripts demonstrating the use of RADARSAT-1 imagery from an AWS bucket

🛰️ Scripts démontrant l'utilisation de l'imagerie RADARSAT-1 à partir d'un seau AWS | 🛰️ Scripts demonstrating the use of RADARSAT-1 imagery from an AWS bucket

Agence spatiale canadienne - Canadian Space Agency 4 May 18, 2022
A telegram media to gofile bot

GoFile-Bot A telegram media to gofile bot Made with Python3 (C) @FayasNoushad Copyright permission under MIT License License - https://github.com/Fay

Fayas Noushad 37 Nov 14, 2022
Github repository started notify 💕

Github repository started notify 💕

4 Aug 06, 2022
A casino discord bot written in Python

Casino Bot Casino bot is a gambling discord bot I made for my friends. It is able to play blackjack, slots, flip a coin, and roll dice. It stores ever

Connor Swislow 27 Dec 30, 2022
Telegram tools

Telegram-Tools Telegram tools. Explanation English | 中文 Features Export group memebrs Add users to the group Send message to users Setup API Open http

4 Apr 02, 2022
Terraform module to ship CloudTrail logs stored in a S3 bucket into a Kinesis stream for further processing and real-time analysis.

AWS infrastructure to ship CloudTrail logs from S3 to Kinesis This repository contains a Terraform module to ship CloudTrail logs stored in a S3 bucke

Nexthink 8 Sep 20, 2022
Grade Notifyer Bot

A bot that automatically crawl the submission platform of montefiore to notify the student when a project has been graded.

Julien Gustin 2 Jun 02, 2022
Weee - Advanced project's versions bumper

Weee - Advanced project's versions bumper

Yan Kurbatov 2 Jun 06, 2022
BroBot's files, code and tester.

README - BroBOT Made by Rohan Chaturvedi [email protected] DISCLAIMER: Th

1 Jan 09, 2022
Retrieve information from DBLP and update BibTex files automatically

Rebib TLDR: This script retrieves information from DBLP to update your BibTex files. python rebib.py --bibfile xxx.bib It first parses the bib entries

Shangtong Zhang 49 Jan 01, 2023
Python SDK for accessing the Hanko Authentication API

Hanko Authentication SDK for Python This package is maintained by Hanko. Contents Introduction Documentation Installation Usage Prerequisites Create a

Hanko.io 3 Mar 08, 2022
Get random jokes bapack2 on telegram

Jokes Bapack2 Telegram Bot Get random jokes bapack2 from jokes-bapack2-api on telegram bot Screenshot Requirements python pip pipenv python-telegram-b

Miftah Afina 2 Nov 17, 2021
NFT which pays royalties to its creator each time it is sold.

Chialisp NFT with Perpetual Creator Royalties This is a chialisp NFT in which the creator/minter defines a puzzle hash which will capture a fixed perc

Geoff Walmsley 20 Jun 28, 2022
AWS Lambda Fast API starter application

AWS Lambda Fast API Fast API starter application compatible with API Gateway and Lambda Function. How to deploy it? Terraform AWS Lambda API is a reus

OBytes 6 Apr 20, 2022
A custom discord bot maker in python

custom-discord-bot-maker Sorry for using Translator. Each description may be inaccurate. how to use 1. Make new application at https://discord.com/dev

2 Nov 29, 2021
Sends messages to a Discord webhook whenever you make a new commit to your local git repository.

Git-Notif Sends messages to a Discord webhook whenever you make a new commit to your local git repository. Usage Just drop notifier.py into your git h

1 May 29, 2022