Seamlessly Connecting Notion Database with Python Pandas DataFrame

Overview

notion-df: Seamlessly Connecting Notion Database with Pandas DataFrame

Please Note: This project is currently in pre-alpha stage. The code are not appropriately documented and tested. Please report any issues you find.

Installation

pip install notion-df

Usage

  • Before starting, please follow the instructions to create a new integration and add it to your Notion page or database.

    • We'll refer Internal Integration Token as the api_key below.
  • Pandas-flavored APIs: Just need to add two additional lines of code:

    import notion_df
    notion_df.pandas() #That's it!
    
    import pandas as pd
    df = pd.read_notion(page_url, api_key=api_key)
    df.to_notion(page_url)
  • Download your Notion table as a pandas DataFrame

    import notion_df
    df = notion_df.load(notion_database_url, api_key=api_key)
    # Equivalent to: df = pd.read_notion(notion_database_url, api_key=api_key)
    df.head()
  • Append a local df to a Notion database:

    import notion_df
    notion_df.upload(df, notion_database_url, title="page-title", api_key=api_key)
    # Equivalent to: df.to_notion(notion_database_url, title="page-title", api_key=api_key)
  • Upload a local df to a newly created database in a Notion page:

    import notion_df
    notion_df.upload(df, notion_page_url, title="page-title", api_key=api_key)
    # Equivalent to: df.to_notion(notion_page_url, title="page-title", api_key=api_key)
  • Tired of typing api_key=api_key each time?

    import notion_df
    notion_df.config(api_key=api_key) # Or set an environment variable `NOTION_API_KEY`
    df = notion_df.load(notion_database_url)
    notion_df.upload(df, notion_page_url, title="page-title")
    # Similarly in pandas APIs: df.to_notion(notion_page_url, title="page-title")

TODOs

  • Add tests for
    • load
    • upload
    • values.py
    • configs.py
    • base.py
  • Better class organizations/namings for *Configs and *Values
Comments
  • Upload df with children

    Upload df with children

    This PR allows uploading page content (aka children) alongside the dataframe.

    How to use it?

    import pandas as pd 
    import notion_df
    from notion_df.blocks import *
    
    child1 = ParagraphBlock(paragraph={"rich_text": RichTextObject.encode_string("A")},)
    child2 = ParagraphBlock(paragraph={"rich_text": RichTextObject.encode_string("B")},)
    
    notion_df.upload(
        df = pd.DataFrame([{"Name": "A"}, {"Name": "B"}]), 
        notion_url = "<>", 
        children=[child1, [child2]], #<- this is the update
        client=notion
    )
    

    Note: Given the experimental nature of this function, we only put it in the upload function but not the pd.to_notion API. We will update when it is appropriately tested.

    opened by lolipopshock 0
  • The tables df columns and schema might not be always consistent

    The tables df columns and schema might not be always consistent

    For example, if you have a relation column inside a table, which points to an relation that the Notion integration doesn't have access to, then

    1. the downloaded data will contain that column
    2. the downloaded schema will not contain that column
    opened by lolipopshock 0
  • Get Page ID?

    Get Page ID?

    Is it possible to get the page id of a database item into the data frame? This way I can map relations without setting resolve_relation_values=True and I won't have a problem with relation values that are named the same.

    opened by thielem 0
  • Error when resolve_relation_values=True

    Error when resolve_relation_values=True

    Hey! In contrast to #26 , I can see rollups an relations when resolve_relation_values =False. When I set resolve_relation_values=True, I get the following error:

    Traceback (most recent call last):
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\indexes\base.py", line 3621, in get_loc
        return self._engine.get_loc(casted_key)
      File "pandas\_libs\index.pyx", line 136, in pandas._libs.index.IndexEngine.get_loc
      File "pandas\_libs\index.pyx", line 163, in pandas._libs.index.IndexEngine.get_loc
      File "pandas\_libs\hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item
      File "pandas\_libs\hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item
    KeyError: 'Beschreibung'
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "a:\xxx\notion_fin_test.py", line 10, in <module>
        df = notion_df.download(page_url, resolve_relation_values=True)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\notion_df\agent.py", line 54, in wrapper
        out = func(client=client, *args, **kwargs)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\notion_df\agent.py", line 216, in download
        relation_df.notion_ids, relation_df[rel_title_col]
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\frame.py", line 3505, in __getitem__
        indexer = self.columns.get_loc(key)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\indexes\base.py", line 3623, in get_loc
        raise KeyError(key) from err
    KeyError: 'Beschreibung'
    
    opened by thielem 1
  • get user information from database

    get user information from database

    When I try to read a databse from notion, the column that shows the name of the mate assigned to this task is shown as a code but not the name. But the property last edited shows the name. Someone knows what is happening ?

    Thanks

    opened by AlmendrosCarmona 0
  • Uploading pages url issue

    Uploading pages url issue

    I like this package and I am starting to use it to automate some of my note taking. I noticed if I want to add a page into database, only full database url works but not the short version database ID. For downloading, both short ID and full url works. This is a minor issue. I mentioned it just in case you want to fix it.

    Thank you! Eric

    opened by yygou 1
Releases(v0.0.5)
  • v0.0.5(Feb 22, 2022)

    What's Changed

    • Better rich text by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/21

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.4...v0.0.5

    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Feb 12, 2022)

    What's Changed

    • Add rollup by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/15
    • Enable relation resolution by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/16
    • Better string length checking #9

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.3...v0.0.4

    Source code(tar.gz)
    Source code(zip)
  • v0.0.3(Feb 1, 2022)

  • v0.0.2(Jan 10, 2022)

    • Better validation for Select Option names #1
    • Allow ignoring errors for uploading #2
    • Handle pagination when downloading #3

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.1...v0.0.2

    Source code(tar.gz)
    Source code(zip)
  • v0.0.1(Jan 6, 2022)

    This release implements the basic function of the notion_df library.

    Usage

    • Before starting, please follow the instructions to create a new integration and add it to your Notion page or database.

      • We'll refer Internal Integration Token as the api_key below.
    • Pandas-flavored APIs: Just need to add two additional lines of code:

      import notion_df
      notion_df.pandas() #That's it!
      
      import pandas as pd
      df = pd.read_notion(page_url, api_key=api_key)
      df.to_notion(page_url)
      
    • Download your Notion table as a pandas DataFrame

      import notion_df
      df = notion_df.load(notion_database_url, api_key=api_key)
      # Equivalent to: df = pd.read_notion(notion_database_url, api_key=api_key)
      df.head()
      
    • Append a local df to a Notion database:

      import notion_df
      notion_df.upload(df, notion_database_url, title="page-title", api_key=api_key)
      # Equivalent to: df.to_notion(notion_database_url, title="page-title", api_key=api_key)
      
    • Upload a local df to a newly created database in a Notion page:

      import notion_df
      notion_df.upload(df, notion_page_url, title="page-title", api_key=api_key)
      # Equivalent to: df.to_notion(notion_page_url, title="page-title", api_key=api_key)
      
    • Tired of typing api_key=api_key each time?

      import notion_df
      notion_df.config(api_key=api_key) # Or set an environment variable `NOTION_API_KEY`
      df = notion_df.load(notion_database_url)
      notion_df.upload(df, notion_page_url, title="page-title")
      # Similarly in pandas APIs: df.to_notion(notion_page_url, title="page-title")
      

    Full Changelog: https://github.com/lolipopshock/notion-df/commits/v0.0.1

    Source code(tar.gz)
    Source code(zip)
Using DST's API with Python

A short guide on how to access Denmark's Statistics API with python, together with a helper class that facilitates the collection of data and metadata from any DST's table

Alessandro Martinello 16 Dec 02, 2022
SmsSender v3.0.0 - the script is designed to send free SMS to any number and with any text.

SmsSender v3.0.0 - скрипт предназначен для бесплатной отправки SMS на любой номер и с любым текстом. Возможны небольшие баги, в скором времени исправл

Андрей Сергеев 20 Dec 03, 2021
A Slash Commands Discord Bot created using Pycord!

Hey, I am Slash Bot. A Bot which works with Slash Commands! Prerequisites Python 3+ Check out. the requirements.txt and install all the pakages. Insta

Saumya Patel 18 Nov 15, 2022
A discord bot written in python

arch-bot A discord bot written in python prefix: . help: .help Installation Requirements A discord bot token Your user id Python installed. For window

3 Jan 10, 2022
A telegram bot that can upload telegram media files to anonfiles.com and give you direct download link

✯ AnonFilesBot ✯ Telegram Files to AnonFiles Upload Bot It will Also Give Direct Download Link Process : Fork This Repositry And Simply Cick On Heroku

Avishkar Patil 38 Dec 30, 2022
Crud-python-sqlite: used to manage telephone contacts through python and sqlite

crud-python-sqlite This program is used to manage telephone contacts through python and sqlite. Dependencicas python3 sqlite3 Installation Clone the r

Luis Negrón 0 Jan 24, 2022
Repositorio que contiene el material mostrado en la primera PyCON de Chile

Buenas prácticas de desarrollo en Python Repositorio que contiene el material mostrado en la primera PyCON de Chile, realizada del 5 al 7 de Noviembre

Erick Castillo 5 Feb 01, 2022
To send an Instagram message using Python

To send an Instagram message using Python, you must have an Instagram account and install the Instabot library in your Python virtual environment.

Coding Taggers 1 Dec 18, 2021
Coin-based opinion monitoring system

介绍 本仓库提供了基于币安 (Binance) 的二级市场舆情系统,可以根据自己的需求修改代码,设定各类告警提示 代码结构 binance.py - 与币安API交互 data_loader.py - 数据相关的读写 monitor.py - 监控的核心方法实现 analyze.py - 基于历史数

luv_dusk 6 Jun 08, 2022
Telegram Voice-Chat Bot

Telegram Voice-Chat Bot [PyTGCalls] ⇝ Requirements ⇜ Account requirements A Telegram account to use as the music bot, You cannot use regular bot accou

1 Dec 03, 2021
This repository will (hopefully) always contain the latest version of the libProfessorP.asm.so shared object.

libPuhfessorP - Deploy Repo This repo should (hopefully) always contain the latest version of the libPuhfessorP.asm.so shared object, to be linked wit

Puhfessor P - CPSC 240 3 Sep 30, 2021
AWS EC2 S3 Automated With python

AWS_EC2_S3_Automated Description This programme is a Python3 script that utilizes Boto3 to automate the process of creating an AWS EC2 instance with a

niall_crowe 2 Nov 16, 2021
A Python app to serve Conveyor room requests and run approvals through Slack

✨ CONVEYOR FOR SLACK ✨ This is a friendly little Python app that will allow you to integrate your instance of Conveyor with your Slack workspace. In o

Vivienne 4 Sep 27, 2021
A modern, easy to use, feature-rich, and async ready API wrapper improved and revived from original discord.py.

A Python API wrapper that is improved and revived from the original discord.py

Orion 19 Nov 06, 2021
Скрипт, позволяющий импортировать плейлисты из Spotify, а также обычные треклисты в VK музыку.

vk-music-import Программа для переноса плейлистов из Spotify и текстовых треклистов в VK Музыку. Преимущества: Позволяет быстро импортировать плейлист

Mew Forest 32 Nov 23, 2022
Python Wrapper for handling payment requests through the Daraja MPESA API

Python Daraja Description Python Wrapper for handling payment requests through the Daraja MPESA API Contribution Refer to the CONTRIBUTING GUIDE. Usag

William Otieno 18 Dec 14, 2022
Play Video & Music on Telegram Group Video Chat

Video Stream is an Advanced Telegram Bot that's allow you to play Video & Music on Telegram Group Video Chat 🧪 Get SESSION_NAME from below: Pyrogram

Sehath Perera 1 Jan 17, 2022
Yes, it's true :revolving_hearts: This repository has 301 stars.

Yes, it's true! Inspired by a similar repository from @RealPeha, but implemented using a webhook on AWS Lambda and API Gateway, so it's serv

511 Dec 30, 2022
This is a simple Python bot to identify sentiments in tweets

Twitter-Sentiment 👋 Hi There! 📱 This is a simple Python bot to identify sentiments in tweets 👨‍💻 This project was made for study, and pratice. You

Guilherme Silva 1 Oct 28, 2021