A very simple asynchronous wrapper that allows you to get access to the Oracle database in asyncio programs.

Overview

cx_Oracle_async

fury licence pyversions Publish Build Docs

A very simple asynchronous wrapper that allows you to get access to the Oracle database in asyncio programs.

Easy to use , buy may not the best practice for efficiency concern.

Requirements

  • cx_Oracle >= 8.1.0 (Take into consideration that author of cx_Oracle said he's trying to implement asyncio support , APIs maybe change in future version. Switch to 8.1.0 if there's something wrong makes it not gonna work.)
  • ThreadPoolExecutorPlus >= 0.2.0

Install

pip install cx_Oracle_async

Feature

  • Nearly all the same as aiomysql in asynchronous operational approach , with limited cx_Oracle feature support.
  • No automaticly date format transition built-in.
  • AQ feature added , check docs here for further information.
  • You can modify some of the connection properties simply like you're using cx_Oracle.
  • You can do basic insert / select / delete etc.
  • If you're connecting to database which is on a different machine from python process , you need to install oracle client module in order to use this library. Check cx-Oracle's installation guide for further information.

Documentation

https://cx_oracle_async.readthedocs.io

Performance

query type asynchronous multithreading synchronous multithreading synchronous single thread
fast single line query 6259.80 q/s 28906.93 q/s 14805.61 q/s
single line insertion 1341.88 q/s 1898 q/s 1685.17 q/s

/* Test platform: */
AMD Ryzen 3700x
Windows 10 LTSC
Oracle 19c
You can find performance test codes here.

Examples

Before running examples , make sure you've already installed a Oracle Client on your machine.

# basic_usages.py
import asyncio
import cx_Oracle_async

async def main():
    oracle_pool = await cx_Oracle_async.create_pool(
        host='localhost', 
        port='1521',
        user='user', 
        password='password',
        service_name='orcl', 
        min = 2,
        max = 4,
    )

    async with oracle_pool.acquire() as connection:
        async with connection.cursor() as cursor:
            await cursor.execute("SELECT * FROM V$SESSION")
            print(await cursor.fetchall())

    await oracle_pool.close()

if __name__ == '__main__':
    asyncio.run(main())
Comments
  • Default cx_Oracle parameters

    Default cx_Oracle parameters

    Hello,

    1. Sometimes I have got cx_Oracle.DatabaseError: ORA-24422 I want to use FORCE parameter of SessionPool.close(force=False) for to make it true. https://cx-oracle.readthedocs.io/en/latest/api_manual/session_pool.html Could you please add this option.

    2. cx_Oracle.SessionPool defaults is encoding=None, nencoding=None Could you please make it as original values.

    bug 
    opened by sinys15 16
  • Support connection with dsn

    Support connection with dsn

    Could you please add create_pool function with dsn parameter. And it looks better to use cx_Oracle.makedsn to build dsn string instead of your variant "f"{host}:{port}/{db}"". Example from documentation: dsn = cx_Oracle.makedsn("dbhost.example.com", 1521, service_name="orclpdb1") connection = cx_Oracle.connect("hr", userpwd, dsn, encoding="UTF-8")

    opened by sinys15 11
  • Support Rollback and Ping

    Support Rollback and Ping

    Could you please support https://cx-oracle.readthedocs.io/en/latest/api_manual/connection.html Connection.ping() and Connection.rollback() methods. This is must have.

    Connection.module Connection.action Connection.client_identifier Connection.clientinfo Would be very nice too.

    Thank you.

    enhancement 
    opened by sinys15 8
  • Async Close procedure for Oracle pool

    Async Close procedure for Oracle pool

    Seems like the Close method of Oracle pool is missed for async. I'm not shure is it just

        async def close_connection_async(self) -> None:
            if self._connection:
                await self._connection.close()
    

    or should be like

        async def close_connection_async(self) -> None:
            if self._connection:
                self._connection.close()
                await self._connection.wait_closed()
    
    opened by sinys15 4
  • Poetry SolverProblemError

    Poetry SolverProblemError

    When I add cx-Oracle-async by Poetry and call "poetry show" command I got the next error: mydir>poetry show

    SolverProblemError

    Because myproject depends on cx-Oracle-async (^0.1.3) which doesn't match any versions, version solving failed.

    at c:\users\myuser\appdata\local\programs\python\python39-32\lib\site-packages\poetry\puzzle\solver.py:241 in _solve 237│ packages = result.packages 238│ except OverrideNeeded as e: 239│ return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest) 240│ except SolveFailure as e: → 241│ raise SolverProblemError(e) 242│ 243│ results = dict( 244│ depth_first_search( 245│ PackageNode(self._package, packages), aggregate_package_nodes

    opened by sinys15 1
  • AsyncCursorWarper add var

    AsyncCursorWarper add var

    Please add class AsyncCursorWarper: ... async def var(self, args): return await self._loop.run_in_executor(self._thread_pool , self._cursor.var, args)

    And why "Warper", may be "Wrapper"?

    enhancement 
    opened by sinys15 0
  • Consider using the new python-oracledb driver

    Consider using the new python-oracledb driver

    Hi, while I was wrangling with cx_Oracle and my M1, I found out that Oracle released a new way to connect to their database systems: https://oracle.github.io/python-oracledb/

    Maybe that will make it easier to write the async functionality due to better integration with Python and the THIN approach.

    opened by ESRE-dev 0
  • callproc keyword_parameters needs functools.partial

    callproc keyword_parameters needs functools.partial

    To prevent this error:

    File /path/to/cx_Oracle_async/cursors.py:45, in AsyncCursorWrapper.callproc(self, *args, **kwargs)
         44 async def callproc(self, *args , **kwargs):
    ---> 45     return await self._loop.run_in_executor(self._thread_pool , self._cursor.callproc, *args , **kwargs)
    
    TypeError: BaseEventLoop.run_in_executor() got an unexpected keyword argument 'keyword_parameters'
    
    opened by Jaza 0
  • Missing cursor attribute 'description'

    Missing cursor attribute 'description'

    Other python database packages (including cx_Oracle) include a method to get the column names which involves using cursor.description. It looks like the object 'description' is not available for cx_Oracle_async for the AsyncCursorWrapper. Is there a different method for returning the column names when using this package?

    opened by bbaker1229 1
  • When the number of async tasks awaiting acquire reaches ThreadPoolExecutorPlus max_workers, this program hangs

    When the number of async tasks awaiting acquire reaches ThreadPoolExecutorPlus max_workers, this program hangs

    Describe the bug When the number of async tasks awaiting acquire reaches ThreadPoolExecutorPlus max_workers, this program hangs.

    To Reproduce Run the following with the sixth line of code commented out (as-is) to hang. Run the following with the sixth line of code uncommented to run successfully. (sixth line has the # toggle commenting... comment on it) (ThreadPoolExecutorPlus DEFAULT_MAXIMUM_WORKER_NUM is 32 * num-cores for Linux)

    import cx_Oracle_async
    import asyncio
    
    dbspecs = dict(dsn="WFX742D", user="ars", password="ars")
    
    max_pool_cnxs = 5
    num_acquire_tasks = 32 * os.cpu_count() + max_pool_cnxs
    
    #num_acquire_tasks -= 1 # toggle commenting of this line to hang/run
    queue_len = 0
    
    async def acquire_conn(pool, i):
       global queue_len
       queue_len += 1
       print_pool_info(f't-{i} acquire...', pool)
       conn = await pool.acquire()
       print_pool_info(f't-{i} postacquire', pool)
       queue_len -= 1
       conn._conn.begin()
       cursor = await conn.cursor()
       await cursor.execute('select current_timestamp from dual', [])
       rows = await cursor.fetchall()
       print(f"t-{i} rows: {len(rows)}")
       await conn.commit()
       await pool.release(conn)
    
    async def test_many_threads():
       pool = await cx_Oracle_async.create_pool(**dbspecs, max=max_pool_cnxs)
       print_pool_info('init', pool)
       tasks = [];
       for i in range(0, num_acquire_tasks):
          tasks.append(asyncio.create_task(acquire_conn(pool, i), name=f'task-{i}'))
       await asyncio.wait(tasks)
       print_pool_info('done', pool)
    
    def print_pool_info(action, pool):
       ipool = pool._pool
       print(f'{action} (queued {queue_len}, busy {ipool.busy}/{ipool.max})')
    
    asyncio.run(test_many_threads())
    

    Expected behavior I expected the queries to be queued and run when connections became available.

    Platform:

    • OS: Red Hat Enterprise Linux Server release 7.9 (Maipo)
    • Oracle version: 11.2.0.4
    • Python version 3.8.11

    Additional context I worked around this by added a semaphore in my code to limit the number of acquires to the max cnxs. What is the benefit of allowing more threads than cnxs?

    bug 
    opened by ericwaldheim 1
  • Cursor context manager is missing __aexit__

    Cursor context manager is missing __aexit__

    Describe the bug Potential memory leak in cursors context manager.

    To Reproduce Call many times cursor.fetchone().

    Gist with sample code: https://gist.github.com/ilosamart/272a7dadb639f19fd62b7947ae12ab5f It contains three versions of the same method: one async (this lib), one async (this lib) which explicitly closes the cursor, one with pure cx_oracle.

    Expected behavior I expected that the cursor context manager closed the cursor.

    Platform:

    • OS: Ubuntu 20.04
    • Oracle version: 12.2
    • Python version 3.8

    Additional context I used the instructions on https://oracle.github.io/odpi/doc/user_guide/debugging.html to debug memory alloc inside OCI.

    bug 
    opened by ilosamart 1
Releases(0.3.3)
Owner
Data Science, Full Stack Developer.
The web framework for inventors

Emmett is a full-stack Python web framework designed with simplicity in mind. The aim of Emmett is to be clearly understandable, easy to be learned an

Emmett 796 Dec 26, 2022
Free and open source full-stack enterprise framework for agile development of secure database-driven web-based applications, written and programmable in Python.

Readme web2py is a free open source full-stack framework for rapid development of fast, scalable, secure and portable database-driven web-based applic

2k Dec 31, 2022
JustPy is an object-oriented, component based, high-level Python Web Framework

JustPy Docs and Tutorials Introduction JustPy is an object-oriented, component based, high-level Python Web Framework that requires no front-en

927 Jan 08, 2023
Web APIs for Django. 🎸

Django REST framework Awesome web-browsable Web APIs. Full documentation for the project is available at https://www.django-rest-framework.org/. Fundi

Encode 24.7k Jan 03, 2023
Mini Web Framework on MicroPython (Esp8266)

dupgee Dupgee is a mini web framework developed for micro-python(Tested on esp8266). Installation pip install dupgee Create Project dupgee create newp

ahmet kotan 38 Jul 25, 2022
The core of a service layer that integrates with the Pyramid Web Framework.

pyramid_services The core of a service layer that integrates with the Pyramid Web Framework. pyramid_services defines a pattern and helper methods for

Michael Merickel 78 Apr 15, 2022
News search API developed for the purposes of the ColdCase Project.

Saxion - Cold Case - News Search API Setup Local – Linux/MacOS Make sure you have python 3.9 and pip 21 installed. This project uses a MySQL database,

Dimitar Rangelov 3 Jul 01, 2021
Library for building WebSocket servers and clients in Python

What is websockets? websockets is a library for building WebSocket servers and clients in Python with a focus on correctness and simplicity. Built on

Aymeric Augustin 4.3k Dec 31, 2022
You can use the mvc pattern in your flask application using this extension.

You can use the mvc pattern in your flask application using this extension. Installation Run the follow command to install mvc_flask: $ pip install mv

Marcus Pereira 37 Dec 17, 2022
Pretty tornado wrapper for making lightweight REST API services

CleanAPI Pretty tornado wrapper for making lightweight REST API services Installation: pip install cleanapi Example: Project folders structure: . ├──

Vladimir Kirievskiy 26 Sep 11, 2022
Pyrin is an application framework built on top of Flask micro-framework to make life easier for developers who want to develop an enterprise application using Flask

Pyrin A rich, fast, performant and easy to use application framework to build apps using Flask on top of it. Pyrin is an application framework built o

Mohamad Nobakht 10 Jan 25, 2022
Dockerized web application on Starlite, SQLAlchemy1.4, PostgreSQL

Production-ready dockerized async REST API on Starlite with SQLAlchemy and PostgreSQL

Artur Shiriev 10 Jan 03, 2023
The little ASGI framework that shines. ?

✨ The little ASGI framework that shines. ✨ Documentation: https://www.starlette.io/ Community: https://discuss.encode.io/c/starlette Starlette Starlet

Encode 7.7k Jan 01, 2023
Free & open source Rest API for YTDislike

RestAPI Free & open source Rest API for YTDislike, read docs.ytdislike.com for implementing. Todo Add websockets Installation Git clone git clone http

1 Nov 25, 2021
Goblet is an easy-to-use framework that enables developers to quickly spin up fully featured REST APIs with python on GCP

GOBLET Goblet is a framework for writing serverless rest apis in python in google cloud. It allows you to quickly create and deploy python apis backed

Austen 78 Dec 27, 2022
A comprehensive reference for all topics related to building and maintaining microservices

This pandect (πανδέκτης is Ancient Greek for encyclopedia) was created to help you find and understand almost anything related to Microservices that i

Ivan Bilan 64 Dec 09, 2022
Official mirror of https://gitlab.com/pgjones/quart

Quart Quart is an async Python web microframework. Using Quart you can, render and serve HTML templates, write (RESTful) JSON APIs, serve WebSockets,

Phil Jones 2 Oct 05, 2022
Bromelia-hss implements an HSS by using the Python micro framework Bromélia.

Bromélia HSS bromelia-hss is the second official implementation of a Diameter-based protocol application by using the Python micro framework Bromélia.

henriquemr 7 Nov 02, 2022
The comprehensive WSGI web application library.

Werkzeug werkzeug German noun: "tool". Etymology: werk ("work"), zeug ("stuff") Werkzeug is a comprehensive WSGI web application library. It began as

The Pallets Projects 6.2k Jan 01, 2023
An easy-to-use high-performance asynchronous web framework.

中文 | English 一个易用的高性能异步 web 框架。 Index.py 文档 Index.py 实现了 ASGI3 接口,并使用 Radix Tree 进行路由查找。是最快的 Python web 框架之一。一切特性都服务于快速开发高性能的 Web 服务。 大量正确的类型注释 灵活且高效的

Index.py 264 Dec 31, 2022