Skip to content

Latest commit

 

History

History
44 lines (27 loc) · 1.15 KB

README.md

File metadata and controls

44 lines (27 loc) · 1.15 KB

MySQL Maxwell Kafka Python Consumer CDC Flow

This is a PoC I prepared to demonstrate CDC (change data capture).

Here is my post about this PoC.

The Ingredients

  • Source database
  • Message broker
  • Destination (consumer)

Technologies Used

  • MySQL
  • Maxwell
  • Kafka
  • Python

To Run

Open up a terminal and browse to the cloned folder and execute the following command to see the magic happen:

docker-compose up

Or to have everything run at the background silently, add -d

docker-compose up -d

Open another terminal to observe the Kafka consumer output:

docker ps

Note the container id of the Kafka consumer container (that is spawned using the image python:3)

docker logs <noted-container-id> --follow

Open an SQL client (like Sequel Pro) and use the following information to connect:

Host: 127.0.0.1
Port:13306
User: root
Password: root

Add a new database, a new table, insert, delete and update some rows and observe the logs on the Python consumer console (which is the final Kafka consumer, subscribed to the topic called maxwell).