A minimal configuration for a dockerized kafka project.

Related tags

Miscellaneousdockerkafkaapachezookeeper
Overview

Docker Kafka Quickstart

A minimal configuration for a dockerized kafka project.

Usage:

  • Run this command to build kafka and zookeeper containers, and create the network kafka-network, where your containers can access the endpoint kafka:29092.
# creates the network `kafka-network` and build and
# runs the containers for zookeeper and kafka
docker-compose -f "docker-compose.kafka.yml" up -d --build # remove -d if you want to see logs in the same shell
  • You can then run as many broker/consumer containers on your kafka-network, in this case: generator is a broker, and indicator is a consumer.
# runs the generator and indicator containers
docker-compose up -d --build
  • If you check the logs in your generator container you should be seeing the messages sent to the queuing.sentences.test topic:
### KAFKA_BROKER_URL kafka:29092
09:35:09.135969 {'time': '09:35:08.107419', 'player': 'player2', 'sentence': 'sDZRJk nwENFG I sR VOOZNWll aGDdZV'}
09:35:10.138498 {'time': '09:35:10.138207', 'player': 'player2', 'sentence': 'vgIXxE  sa'}
09:35:11.141023 {'time': '09:35:11.140690', 'player': 'player1', 'sentence': 'E raH jzIeKHqXZF JGGB   tx  M oLusi '}
09:35:12.142630 {'time': '09:35:12.142360', 'player': 'player2', 'sentence': 'WxsPlteC  yxh  ogME dvbtFn l b TxVB  gaMFZe'}
09:35:13.144693 {'time': '09:35:13.144454', 'player': 'player1', 'sentence': ' aIRVm W TeIcKuigy  HPLOvJrQSXSl ZlLHlL FRH'}
09:35:14.146182 {'time': '09:35:14.145966', 'player': 'player1', 'sentence': 'cmMH k phZ LapRdGv C geTDzIT'}
09:35:15.147609 {'time': '09:35:15.147378', 'player': 'player2', 'sentence': 'QihJPzjFM WmkmDzMLl DkyydRVeyEI Rm'}
09:35:16.149716 {'time': '09:35:16.149502', 'player': 'player1', 'sentence': '  C '}
09:35:17.151371 {'time': '09:35:17.151155', 'player': 'player2', 'sentence': ''}
09:35:18.152736 {'time': '09:35:18.152508', 'player': 'player1', 'sentence': 'Oh tgfSl tY iBrxMHf aNX h'}
...
  • And the logs for indicator show the messages recieved on the queuing.sentences.test topic:
### KAFKA_BROKER_URL kafka:29092
Connecting...
Connected to queuing.sentences.test topic!
09:35:13.320414 {'time': '09:35:13.144454', 'player': 'player1', 'sentence': ' aIRVm W TeIcKuigy  HPLOvJrQSXSl ZlLHlL FRH'}
09:35:14.150851 {'time': '09:35:14.145966', 'player': 'player1', 'sentence': 'cmMH k phZ LapRdGv C geTDzIT'}
09:35:15.279800 {'time': '09:35:15.147378', 'player': 'player2', 'sentence': 'QihJPzjFM WmkmDzMLl DkyydRVeyEI Rm'}
09:35:16.153846 {'time': '09:35:16.149502', 'player': 'player1', 'sentence': '  C '}
09:35:17.156841 {'time': '09:35:17.151155', 'player': 'player2', 'sentence': ''}
09:35:18.156372 {'time': '09:35:18.152508', 'player': 'player1', 'sentence': 'Oh tgfSl tY iBrxMHf aNX h'}
09:35:19.160985 {'time': '09:35:19.154674', 'player': 'player2', 'sentence': 'qgSAzkUNm  fBQyruw S U   PvLnharO '}
09:35:20.183376 {'time': '09:35:20.157269', 'player': 'player1', 'sentence': 'OlX SW  a gcpXVGHxW'}

Refs:

For more information about using kafka with Docker: https://hub.docker.com/r/wurstmeister/kafka/

Owner
Nouamane Tazi
Software Engineer | AI Passionate. Seeking end-of-studies 6-month internship.
Nouamane Tazi
GitHub Repository
This is the key combo trainer for League of Legends and Dota 2 players.

This is the key combo trainer for League of Legends and Dota 2 players. Place the mouse cursor on the blue point and press the key combo from the upper-left side of the screen.

Ilya Shpigor 1 Jan 31, 2022
This program can calculate the Aerial Distance between two cities.

Aerial_Distance_Calculator This program can calculate the Aerial Distance between two cities. This repository include both Jupyter notebook and Python

InvisiblePro 1 Apr 08, 2022
3x - This Is 3x Friendlist Cloner Tools

3X FRIENDLIST CLONER TOOLS COMMAND $ apt update $ apt upgrade $ apt install pyth

MAHADI HASAN AFRIDI 2 Jan 17, 2022
In this project we will be using OpenCV to virtually drag a rectangle and drop it at a different location. It will be further used for Virtual Mouse.

Virtual Drag & Drog using OpenCV In this project we will be using OpenCV to virtually drag a rectangle and drop it at a different location. It will be

Hassan Shahzad 5 Sep 27, 2021
ERPNext Easy Letterhead

ERPNext Easy Letterhead Intro Quality letterheads are a problem for non-technical users. So we've built (really hacked together) a slightly easier sol

Bantoo 3 Jan 02, 2023
Sorter makes file organisation and management easier.

Sorter Sorter makes file organisation easier. It simply helps you organise several files that contain similar characteristics into a single folder. Yo

Aswa Paul 34 Aug 14, 2022
WinBoost: Boost your windows system.

Winboost runs a complete checkup of your entire system locating junk files, speed-reducing issues and causes of any system or application glitches or crashes. Through a lot of research and testing, w

Smit Parmar 4 Oct 01, 2021
A scuffed remake of Kahoot... Made by Y9 and Y10 SHSB

A scuffed remake of Kahoot... Made by Y9 and Y10 SHSB

Tobiloba Kujore 3 Oct 28, 2022
Data Poisoning based on Adversarial Attacks using Non-Robust Features

Data Poisoning based on Adversarial Attacks using Non-Robust Features Usage python main.py [-h] [--gpu | -g GPU] [--eps |-e EPSILON] [--pert | -p PER

Jonathan E. 1 Nov 02, 2021
Python binding to rust zw-fast-quantile

zw_fast_quantile_py zw-fast-quantile python binding Installation pip install zw_fast_quantile_py Usage import zw_fast_quantile_py

Paul Meng 1 Dec 30, 2021
A collection of daily usage utility scripts in python. Helps in automation of day to day repetitive tasks.

Kush's Utils Tool is my personal collection of scripts which is used to automated daily tasks. It is a evergrowing collection of scripts and will continue to evolve till the day I program. This is al

Kushagra 10 Jan 16, 2022
Hacktoberfest2021 🥳- Contribute Any Pattern In Any Language😎 Every PR will be accepted Pls contribute

✨ Hacktober Fest 2021 ✨ 🙂 All Contributors are requested to star this repo and follow me for a successful merge of pull request. 🙂 👉 Add any patter

Md. Almas Ali 103 Jan 07, 2023
ChronoRace is a tool to accurately perform timed race conditions to circumvent application business logic.

ChronoRace is a tool to accurately perform timed race conditions to circumvent application business logic. I've found in my research that w

Tanner 64 Aug 04, 2022
Logging-monitoring-instrumentation - A brief repository on logging monitoring and instrumentation in Python

logging-monitoring-instrumentation A brief repository on logging monitoring and

Noah Gift 6 Feb 17, 2022
Streamlit component to display topics from Streamlit's community forum related to any exception.

streamlit-forum Streamlit component to display topics from Streamlit's community forum related to any exception. Installation pip install streamlit-fo

Snehan Kekre 7 Jul 15, 2022
A calculator for common measurements used in sci-fi books.

Sci-fi-speed-calculator A calculator for common measurements used in sci-fi books. Author: Tyler Windmemuth Purpose: This program allows sci-fi author

Tyler Windemuth 0 Apr 22, 2022
Scripts to convert the Ted-MDB corpora into the formats for DISRPT shared task and the converted corpora

Scripts to convert the Ted-MDB corpora into the formats for DISRPT shared task and the converted corpora.

1 Feb 08, 2022
It is a personal assistant chatbot, capable to perform many tasks same as Google Assistant plus more extra features...

PersonalAssistant It is an Personal Assistant, capable to perform many tasks with some unique features, that you haven'e seen yet.... Features / Tasks

Roshan Kumar 95 Dec 21, 2022
Like Docker, but for Squeak. You know, for kids.

Squeaker Like Docker, but for Smalltalk images. You know, for kids. It's a small program that helps in automated derivation of configured Smalltalk im

Tony Garnock-Jones 14 Sep 11, 2022
A carrot-based color palette you didn't know you needed.

A package to produce a carrot-inspired color palette for python/matplotlib. Install: pip install carrotColors Update: pip install --upgrade carrotColo

10 Sep 28, 2021