Feapder的管道扩展

Overview

FEAPDER 管道扩展

简介

此模块为feapderpipelines扩展,感谢广大开发者对feapder的贡献

随着feapder支持的pipelines越来越多,为减少feapder的体积,特将pipelines提出,使用者可按需安装

管道

PostgreSQL

贡献者:沈瑞祥

联系方式:[email protected]

安装

pip install feapder_pipelines[pgsql]

使用

feapder项目的setting.py中使用如下配置

# PostgreSQL
PGSQL_IP = 
PGSQL_PORT = 
PGSQL_DB = 
PGSQL_USER_NAME = 
PGSQL_USER_PASS = 

ITEM_PIPELINES = [
    "feapder_pipelines.pipelines.pgsql_pipeline.PgsqlPipeline"
]
细节 注:入库时 ON CONFLICT(key) 默认为id或通过如下sql查出来的第一个值
select column_names from(
        select
            t.relname as table_name,
            i.relname as index_name,
            array_to_string(array_agg(a.attname), ', ') as column_names
        from
            pg_class t,
            pg_class i,
            pg_index ix,
            pg_attribute a
        where
            t.oid = ix.indrelid
            and i.oid = ix.indexrelid
            and a.attrelid = t.oid
            and a.attnum = ANY(ix.indkey)
            and t.relkind = 'r'
            and t.relname like '%'
        group by
            t.relname,
            i.relname
        order by
            t.relname,
            i.relname) as res
    where table_name = 'table_name';
You might also like...
Releases(v1.0.3)
Owner
boris
爬虫工程师、数据工程师
boris
Python MQTT v5.0 async client

gmqtt: Python async MQTT client implementation. Installation The latest stable version is available in the Python Package Index (PyPi) and can be inst

Gurtam 306 Jan 03, 2023
Script to produce `.tex` files of example GAP sessions

Introduction The main file GapToTex.py in this directory is used to produce .tex files of example GAP sessions. Instructions Run python GapToTex.py [G

Friedrich Rober 2 Oct 06, 2022
Scripts to integrate DFIR-IRIS, MISP and TimeSketch

Scripts to integrate DFIR-IRIS, MISP and TimeSketch

Koen Van Impe 20 Dec 16, 2022
A simple python project which control paint brush in microsoft paint app

Paint Buddy In Python A simple python project which control paint brush in micro

Ordinary Pythoneer 1 Dec 27, 2021
Runtime fault injection platform by Daniele Rizzieri (2021)

GDBitflip [v1.04] Runtime fault injection platform by Daniele Rizzieri (2021) This platform executes N times a binary and during each execution it inj

Daniele Rizzieri 1 Dec 07, 2021
Generate Azure Blob Storage account authentication headers for Munki

Azure Blob Storage Authentication for Munki The Azure Blob Storage Middleware allows munki clients to connect securely, and directly to a munki repo h

Oliver Kieselbach 10 Apr 12, 2022
a package that provides a marketstrategy for whitelisting on golem

filterms a package that provides a marketstrategy for whitelisting on golem watching requestor logs distribute 10 tasks asynchronously is fun. but you

KJM 3 Aug 03, 2022
Код файнтюнинга оригинального CLIP на русский язык

О чем репозиторий В этом репозитории представлен способ файтюнить оригинальный CLIP на новый язык Почему модель не видит женщину и откуда на картинке

Valentina Biryukova 7 Feb 06, 2022
The Great Autoencoder Bake Off

The Great Autoencoder Bake Off The companion repository to a post on my blog. It contains all you need to reproduce the results. Features Currently fe

Tilman Krokotsch 61 Jan 06, 2023
Necst-lib - Pure Python tools for NECST

necst-lib Pure Python tools for NECST. Features This library provides: something

NANTEN2 Group 5 Dec 15, 2022
Generate your personal 8-bit avatars using Cellular Automata, a mathematical model that simulates life, survival, and extinction

Try the interactive demo here ✨ ✨ Sprites-as-a-Service is an open-source web application that allows you to generate custom 8-bit sprites using Cellul

Lj Miranda 265 Dec 26, 2022
Software that extracts spreadsheets from various .pdf files to .csv

Extração de planilhas de diversos arquivos .pdf para .csv O código inteiro foi desenvolvido em Python. Foi utilizado o pacote "tabula" e a biblioteca

Marcos Silva 2 Jan 09, 2022
A collection of Workflows samples for various use cases

Workflows Samples Workflows allow you to orchestrate and automate Google Cloud and HTTP-based API services with serverless workflows.

Google Cloud Platform 76 Jan 07, 2023
Albert launcher extension for rolling dice.

dice-roll-albert-ext Extension for rolling dice in Albert launcher Installation Locate the modules directory in the Python extension data directory. T

Jonah Lawrence 1 Nov 18, 2021
Ferramenta de monitoramento do risco de colapso no sistema de saúde em municípios brasileiros com a Covid-19.

FarolCovid 🚦 Ferramenta de monitoramento do risco de colapso no sistema de saúde em municípios brasileiros com a Covid-19. Monitoring tool & simulati

Impulso 49 Jul 10, 2022
Research on how Gboard Stickers work.

Google-Sticker-Mashup-Research Research on how Gboard Stickers work. Contribute Contributing is nice, and you will be listed below for contributing. C

Jeremiah 45 Oct 28, 2022
Hands-on machine learning workshop

emb-ntua-workshop This workshop discusses introductory concepts of machine learning and data mining following a hands-on approach using popular tools

ISSEL Soft Eng Team 12 Oct 30, 2022
Trashselected - Plugin for fman.io to move files that has been selected in fman to trash

TrashSelected Plugin for fman.io to move files that has been selected in fman to

1 Feb 04, 2022
Wordle is fun, so let's ruin it with computers.

ruin-wordle Wordle is fun, so let's ruin it with computers. Metrics This repository assesses two metrics about each algorithm: Success: how many of th

Charles Tapley Hoyt 11 Feb 11, 2022
Headless chatbot that detects spam and posts links to it to chatrooms for quick deletion.

SmokeDetector Headless chatbot that detects spam and posts it to chatrooms. Uses ChatExchange, takes questions from the Stack Exchange realtime tab, a

Charcoal 421 Dec 21, 2022