CloudKarafka + Spark Setup:

Nirav Langaliya
2 min readApr 28, 2020


cloudkarafka provides a free shared plan for development and testing. you can create an account on cloudkafka and start your development or testing quickly.

Configure Kafka instance ( For Additional Information, Please refer )

Once it is created, you can have connection credentials.

check the version of Scala


Invoke the Spark Shell and provide the correct package.

spark-shell --packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.4.5,org.apache.spark:spark-avro_2.11:2.4.5

Read Kafka Topic via Spark
Publish Data to Kafka

Once you publish data on the Kafka topic, Kafka consumer at spark will start display data on the console.



Nirav Langaliya

Lead Data Engineer@Nisum Technologies