![Slack Logo](/_next/image?url=%2Fslack_logo.png&w=32&q=20)
multicorn
Multicorn is a PostgreSQL extension that allows you to create foreign data wrappers in Python.
Install
trunk install multicorn
Multicorn2
Multicorn Python3 Wrapper for Postgresql Foreign Data Wrapper. Tested on Linux w/ Python 3.6+ & Postgres 12 thru 16. Support for Postgres 14+ is known to have some issues with predicate pushdown and updating.
The Multicorn Foreign Data Wrapper allows you to fetch foreign data in Python in your PostgreSQL server.
Multicorn2 is distributed under the PostgreSQL license. See the LICENSE file for details.
How it works
In use, Multicorn includes a Python package which contains:
- A
__init__.py
which provides a base class from which your new, custom, FDW will derive. - A
utils.py
containing diagnostic support code your FDW can use. - Several useful example FDW implementations.
Multicorn also includes, under the covers, two shared libraries:
multicorn.so
contains a generic Postgres FDW extension which interfaces between Postgres and your custom FDW._utils.so
is a CPython extension which provides support for the previously mentionedutils.py
.
Using in OSCG.IO
1.) Install OSCG.IO from the command line, in your home directory, with the curl command at the top of https://oscg.io/usage.html
2.) Change into the oscg directory and install pgXX
cd oscg
./io install pg13 --start
./io install multicorn2
3.) Use multicorn as you normally would AND you can install popular FDW's that use multicorn such as ElasticSerachFDW & BigQueryFDW
./io install esfdw
./io install bqfdw
Building Multicorn2 against Postgres from Source
It is built the same way all standard postgres extensions are built with following dependcies needing to be installed:
Install Dependencies for Building Postgres & the Multicorn2 extension
On Debian/Ubuntu systems:
sudo apt install -y build-essential libreadline-dev zlib1g-dev flex bison libxml2-dev libxslt-dev libssl-dev libxml2-utils xsltproc
sudo apt install -y python3 python3-dev python3-setuptools python3-pip
On CentOS/Rocky/Redhat systems:
sudo yum install -y bison-devel readline-devel libedit-devel zlib-devel openssl-devel bzip2-devel libmxl2 libxslt-devel wget
sudo yum groupinstall -y 'Development Tools'
sudo yum -y install git python3 python3-devel python3-pip
Upgrade to latest PIP (recommended)
cd ~
wget https://bootstrap.pypa.io/get-pip.py
sudo python3 get-pip.py
rm get-pip.py
Download & Compile Postgres 12+ source code
cd ~
wget https://ftp.postgresql.org/pub/source/v14.3/postgresql-14.3.tar.gz
tar -xvf postgresql-14.3.tar.gz
cd postgresql-14.3
./configure
make
sudo make install
Download & Compile Multicorn2
set PATH=/usr/local/pgsql/bin:$PATH
cd ~/postgresql-14.3/contrib
wget https://github.com/pgsql-io/multicorn2/archive/refs/tags/v2.3.tar.gz
tar -xvf v2.3.tar.gz
cd multicorn2-2.3
make
sudo make install
Create Multicorn2 Extension
In your running instance of Postgres from the PSQL command line
CREATE EXTENSION multicorn;
Building Multicorn2 against pre-built Postgres
When using a pre-built Postgres installed using your OS package manager, you will need the additional package that enables Postgres extensions to be built:
Install Dependencies for Building the Multicorn2 extension
On Debian/Ubuntu systems:
sudo apt install -y build-essential ... postgresql-server-dev-13
sudo apt install -y python3 python3-dev python3-setuptools python3-pip
On CentOS/Rocky/Redhat systems:
sudo yum install -y ... <postgres server dev package>
sudo yum groupinstall -y 'Development Tools'
sudo yum -y install git python3 python3-devel python3-pip
Download & Compile Multicorn2
wget https://github.com/pgsql-io/multicorn2/archive/refs/tags/v2.5.tar.gz
tar -xvf v2.5.tar.gz
cd multicorn2-2.5
make
sudo make install
Note that the last step installs both the extension into Postgres and also the Python part. The latter is done using "pip install", and so can be undone using "pip uninstall".
Create Multicorn2 Extension
In your running instance of Postgres from the PSQL command line
CREATE EXTENSION multicorn;