CloudKarafka + Spark Setup:

cloudkarafka provides a free shared plan for development and testing. you can create an account on cloudkafka and start your development or testing quickly.

https://www.cloudkarafka.com/plans.html

Configure Kafka instance ( For Additional Information, Please refer https://www.cloudkarafka.com/docs/index.html )

Once it is created, you can have connection credentials.

check the version of Scala

util.Properties.versionString

Invoke the Spark Shell and provide the correct package.

spark-shell --packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.4.5,org.apache.spark:spark-avro_2.11:2.4.5

Read Kafka Topic via Spark

Publish Data to Kafka

Once you publish data on the Kafka topic, Kafka consumer at spark will start display data on the console.