Dec 28, 2020

Setup OIDC Authentication for Lyft Amundsen.

1. Okta Integration if your organization has already using Okta then you can register your application with Okta or if you doing POC and test out okta integration then you can create a free Okta developer account: https://developer.okta.com/signup/ You can follow the page (written by Randall Degges), it gives a very detailed explanation…

Amundsen

4 min read

Amundsen

4 min read

Nov 18, 2020

Fetch PostgreSQL Metadata Via Linkedin datahub

Setup python virtual environment python3 -m venv venv source venv/bin/activate pip3 install -r requirements.txt download psycopg2 package from https://github.com/psycopg/psycopg2 python3 setup.py install if you get “Error: pg_config executable not found.” then install PostgreSQL (Refer: https://stackoverflow.com/questions/35104097/how-to-install-psycopg2-with-pg-config-error ) brew install postgresql

Data Catalog

2 min read

Fetch PostgreSQL Metadata Via Linkedin datahub

Data Catalog

2 min read

Fetch PostgreSQL Metadata Via Linkedin datahub

Setup python virtual environment

python3 -m venv venv
source venv/bin/activate
pip3 install -r requirements.txt

download psycopg2 package from

https://github.com/psycopg/psycopg2

python3 setup.py install

if you get “Error: pg_config executable not found.”

then install PostgreSQL (Refer: https://stackoverflow.com/questions/35104097/how-to-install-psycopg2-with-pg-config-error )

brew install postgresql

Apr 28, 2020

CloudKarafka + Spark Setup:

cloudkarafka provides a free shared plan for development and testing. you can create an account on cloudkafka and start your development or testing quickly. Configure Kafka instance ( For Additional Information, Please refer https://www.cloudkarafka.com/docs/index.html )

Kafka

2 min read

Kafka

2 min read

Sep 26, 2019

Confluent Kafka Sink connector for MySQL

##Download Confluent Community version. https://www.confluent.io/confirmation/ ## Install Confluent Kafka Local. (Untar setup file) tar -xvf confluent-community-5.3.1–2.12.tar ##Setup Conflunet Cli. curl -L https://cnfl.io/cli | sh -s — -b ~/confluent-5.3.1/bin ##Include Confluent bin into PATH. export PATH=${PATH}:~/confluent-5.3.1/bin ## Start Conflunet confluent local start ##Download MySQL Jar file place under

Confluent Kafka

2 min read

Confluent Kafka

2 min read

Setup OIDC Authentication for Lyft Amundsen.

Fetch PostgreSQL Metadata Via Linkedin datahub

Fetch PostgreSQL Metadata Via Linkedin datahub

CloudKarafka + Spark Setup:

Confluent Kafka Sink connector for MySQL

Nirav Langaliya