Fetch PostgreSQL Metadata Via Linkedin datahub

Setup python virtual environment

python3 -m venv venv
source venv/bin/activate
pip3 install -r requirements.txt

download psycopg2 package from

https://github.com/psycopg/psycopg2

python3 setup.py install

if you get “Error: pg_config executable not found.”

then install PostgreSQL (Refer: https://stackoverflow.com/questions/35104097/how-to-install-psycopg2-with-pg-config-error )

brew install postgresql

update URL at postgres_etl.py file.

URL = 'postgresql+psycopg2://user:password@hansken.db.elephantsql.com'

cd ./datahub/metadata-ingestion/sql-etl
python3 postgres_etl.py

if you get the below error:

then follow step 2 at https://github.com/linkedin/datahub/tree/master/metadata-ingestion#prerequisites