Object-data mapper and advanced query manager for non relational databases

Overview

Object data mapper and advanced query manager for non relational databases.

The data is owned by different, configurable back-end databases and it is accessed using a light-weight Object Data Mapper (ODM). The ODM presents a method of associating user-defined Python classes with database collections, and instances of those classes with items in their corresponding collections. Collections and items are different for different backend databases but are treated in the same way in the python language domain.

Master CI: master-build coverage
Dev CI: dev-build coverage-dev
Documentation: http://pythonhosted.org/python-stdnet/
Dowloads: http://pypi.python.org/pypi/python-stdnet/
Source: https://github.com/lsbardel/python-stdnet
Platforms: Linux, OS X, Windows. Python 2.6, 2.7, 3.2, 3.3, pypy
Mailing List: https://groups.google.com/group/python-stdnet
Keywords: server, database, redis, odm

Contents

Features

  • Models with scalar and multi-value fields.
  • Rich query API including unions, intersections, exclusions, ranges and more.
  • Minimal server round-trips via backend scripting (lua for redis).
  • Full text search.
  • Signals handling to allow decoupled applications to get notified on changes.
  • Synchronous and asynchronous database connection.
  • Multi-variate numeric timeseries application.
  • Asynchronous Publish/Subscribe application.
  • 90% Test coverage.
  • Fully documented.

Requirements

  • Python 2.6, 2.7, 3.2, 3.3 and pypy. Single code-base.
  • redis-py for redis backend.
  • Optional pulsar when using the asynchronous connections or the test suite.
  • You need access to a Redis server version 2.6 or above.

Philosophy

Key-valued pairs databases, also know as key-value stores, have many differences from traditional relational databases, most important being they do not use SQL as their query language, storage does not require a fixed table schemas and usually they do not support complex queries.

Stdnet aims to accommodate a flexible schema and join type operations via a lightweight object data mapper. Importantly, it is designed with large data sets in mind. You pull data you need, nothing more, nothing less. Bandwidth and server round-trips can be reduced to the bare minimum so that your application is fast and memory efficient.

Installing

To install, download, uncompress and type:

python setup.py install

otherwise use easy_install:

easy_install python-stdnet

or pip:

pip install python-stdnet

Version Check

To know which version you have installed:

>>> import stdnet
>>> stdnet.__version__
'0.8.0'
>>> stdnet.VERSION
stdnet_version(major=0, minor=8, micro=0, releaselevel='final', serial=1)

Backends

Backend data-stores are the backbone of the library. Currently the list is limited to

Object Data Mapper

The stdnet.odm module is the ODM, it maps python objects into database data and vice-versa. It is design to be fast and safe to use:

from stdnet import odm

class Base(odm.StdModel):
    '''An abstract model. This won't have any data in the database.'''
    name = odm.SymbolField(unique = True)
    ccy  = odm.SymbolField()

    def __unicode__(self):
        return self.name

    class Meta:
        abstract = True


class Instrument(Base):
    itype = odm.SymbolField()


class Fund(Base):
    description = odm.CharField()


class PositionDescriptor(odm.StdModel):
    dt    = odm.DateField()
    size  = odm.FloatField()
    price = odm.FloatField()
    position = odm.ForeignKey("Position", index=False)


class Position(odm.StdModel):
    instrument = odm.ForeignKey(Instrument, related_name='positions')
    fund       = odm.ForeignKey(Fund)
    history    = odm.ListField(model=PositionDescriptor)

    def __unicode__(self):
        return '%s: %s @ %s' % (self.fund,self.instrument,self.dt)

Register models with backend:

models = orm.Router('redis://localhost?db=1')
models.register(Instrument)
models.register(Fund)
models.register(PositionDescriptor,'redis://localhost?db=2')
models.register(Position,'redis://localhost?db=2')

And play with the API:

>>> f = models.fund.new(name="pluto, description="The pluto fund", ccy="EUR")
>>> f
Fund: pluto

Running Tests

At the moment, only redis back-end is available and therefore to run tests you need to install Redis. If you are using linux, it can be achieved simply by downloading, uncompressing and running make, if you are using windows you can find sources from MSOpenTech.

Requirements for running tests:

  • python-stdnet project directory.
  • pulsar.

To run tests open a shell and launch Redis. On another shell, from within the python-stdnet package directory, type:

python runtests.py

Tests are run against a local redis server on port 6379 and database 7 by default. To change the server and database where to run tests pass the --server option as follow:

python runtests.py --server redis://myserver.com:6450?db=12&password=bla

For more information type:

python runtests.py -h

Kudos

  • Redis simply because this library uses its awesome features.
  • SQLAlchemy and Django for ideas and API design.

Contributing

Development of stdnet happens at Github: http://github.com/lsbardel/python-stdnet

We very much welcome your contribution of course. To do so, simply follow these guidelines:

  1. Fork python-stdnet on github
  2. Create a topic branch git checkout -b my_branch
  3. Push to your branch git push origin my_branch
  4. Create an issue at https://github.com/lsbardel/python-stdnet/issues with a link to your patch

License

This software is licensed under the New BSD License. See the LICENSE file in the top distribution directory for the full license text.

Owner
Luca Sbardella
quantitative wizard at @quantmind
Luca Sbardella
Python / C++ based particle reaction-diffusion simulator

ReaDDy (Reaction Diffusion Dynamics) is an open source particle based reaction-diffusion simulator that can be configured and run via Python. Currentl

ReaDDy 46 Dec 09, 2022
Async timeit - Async version of python's timeit

Async Timeit Replica of default python timeit module with small changes to allow

Raghava G Dhanya 3 Apr 13, 2022
Path of Exile Vendor Recipe Tracker (Chaos/Regal orb)

Path of Exile Vendor Trade Tracker Are you tired of manually keeping track of collected and missing items for farming Chaos or Regal Orbs in PoE? Me t

1 Nov 09, 2021
En este repositorio pondré archivos graciositos de python que hago de vez en cuando

🐍 Apuntes de python 🐍 ¿Quién soy? 👽 Saludos,mi nombre es Carlos Lara. Pero mi nickname en internet es Hercules Kan. Soy un programador autodidacta

Carlos E. Lara 3 Nov 16, 2021
Starscape is a Blender add-on for adding stars to the background of a scene.

Starscape Starscape is a Blender add-on for adding stars to the background of a scene. Features The add-on provides the following features: Procedural

Marco Rossini 5 Jun 24, 2022
An OBS script to fuze files together

OBS TEXT FUZE Fuze text files and inject the output into a text source. The Index file directory should be a list of file directorys for the text file

SuperZooper3 1 Dec 27, 2021
take home quiz

guess the correlation data inspection a pretty normal distribution train/val/test split splitting amount .dataset: 150000 instances ├─8

HR Wu 1 Nov 04, 2021
Cobalt Strike Sleep Python Bridge

This project is 'bridge' between the sleep and python language. It allows the control of a Cobalt Strike teamserver through python without the need for for the standard GUI client. NOTE: This project

Cobalt Strike 140 Jan 04, 2023
firefox session recovery

firefox session recovery

Ahmad Sadraei 5 Nov 29, 2022
A Google sheet which keeps track of the locations that want to visit and a price cutoff

FlightDeals Here's how the program works. First, I have a Google sheet which keeps track of the locations that I want to visit and a price cutoff. It

Lynne Munini 5 Nov 21, 2022
A Red Team tool for exfiltrating sensitive data from Jira tickets.

Jir-thief This Module will connect to Jira's API using an access token, export to a word .doc, and download the Jira issues that the target has access

Antonio Piazza 82 Dec 12, 2022
Identify unused production dependencies and avoid a bloated virtual environment.

creosote Identify unused production dependencies and avoid a bloated virtual environment. Quickstart # Install creosote in separate virtual environmen

Fredrik Averpil 7 Dec 29, 2022
A Tool to validate domestic New Zealand vaccine passes

Vaccine Validator Tool to validate domestic New Zealand vaccine passes Create a new virtual environment: python3 -m venv ./venv Activate virtual envi

8 May 01, 2022
Multtable is a collection of multiplication table generators in various languages.

Multtable Multtable is a collection of multiplication table generators in various languages. This project was created as a joke based on one of my bro

pollen__ 7 Mar 05, 2022
Reso is a low-level circuit design language and simulator, inspired by things like Redstone, Conway's Game of Life, and Wireworld.

Reso Reso is a low-level circuit design language and simulator, inspired by things like Redstone, Conway's Game of Life, and Wireworld. What is Reso?

Lynn 287 Nov 26, 2022
Oppia is an online learning tool that enables anyone to easily create and share interactive activities

Oppia is an online learning tool that enables anyone to easily create and share interactive activities (called 'explorations'). These activities simulate a one-on-one conversation with a tutor, makin

Oppia 4.7k Dec 29, 2022
Expense-manager - Expense manager with python

Expense_manager TO-DO Source extractor: Credit Card, Wallet Destination extracto

1 Feb 13, 2022
Additional useful operations for Python

Pyteal Extensions Additional useful operations for Python Available Operations MulDiv64: calculate m1*m2/d with no overflow on multiplication (TEAL 3+

Ulam Labs 11 Dec 14, 2022
DRF magic links

drf-magic-links Installation pip install drf-magic-links Add URL patterns # urls.py

Dmitry Kalinin 1 Nov 07, 2021
MDAnalysis tool to calculate membrane curvature.

The MDAkit for membrane curvature analysis is part of the Google Summer of Code program and it is linked to a Code of Conduct.

MDAnalysis 19 Oct 20, 2022