Kafka简介

kafka官网下载

基本概念

Kafka安装和测试

下载

kafka_2.12-2.8.0下载

配置

zookeeper.properties

# the directory where the snapshot is stored.
dataDir=/home/huangyc/kafka_2.12-2.8.0/zk_logs
# the port at which the clients will connect
clientPort=2183

server.properties

# The address the socket server listens on. It will get the value returned from 
# java.net.InetAddress.getCanonicalHostName() if not configured.
#   FORMAT:
#     listeners = listener_name://host_name:port
#   EXAMPLE:
#     listeners = PLAINTEXT://your.host.name:9092
listeners=PLAINTEXT://:9092

# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
zookeeper.connect=localhost:2183

# A comma separated list of directories under which to store log files
log.dirs=/home/huangyc/kafka_2.12-2.8.0/kafka_logs

启动

启动zookeeper

1	./bin/zookeeper-server-start.sh ./config/zookeeper.properties

启动kafka

1	./bin/kafka-server-start.sh ./config/server.properties

查看Kafka进程

1 2	[root@Hmaster kafka_2.12-2.8.0]# jps \| grep Kafka 23425 Kafka

测试

查看主题列表

1	bin/kafka-topics.sh --zookeeper localhost:2183 --list

消息生产者

1	bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

消息消费者

1
2
3

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
0.9以前版本: 
kafka-console-consumer.bat --zookeeper localhost:2183 --topic test --from-beginning

另一套方案

server.properties

# The address the socket server listens on. It will get the value returned from 
# java.net.InetAddress.getCanonicalHostName() if not configured.
#   FORMAT:
#     listeners = listener_name://host_name:port
#   EXAMPLE:
#     listeners = PLAINTEXT://your.host.name:9092
listeners=PLAINTEXT://:9092

# Hostname and port the broker will advertise to producers and consumers. If not set, 
# it uses the value for "listeners" if configured.  Otherwise, it will use the value
# returned from java.net.InetAddress.getCanonicalHostName().
advertised.listeners=PLAINTEXT://192.168.123.21:9092

# A comma separated list of directories under which to store log files
log.dirs=/home/huangyc/kafka_2.12-2.8.0/kafka_logs

# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
zookeeper.connect=localhost:2183

消息生产者

1	bin/kafka-console-producer.sh --broker-list localhost:9092 --topiest

消息消费者

1	bin/kafka-console-consumer.sh --bootstrap-server 192.168.123.21:9092

其他问题

win10访问

需要配置host

C:\Windows\System32\drivers\etc\hosts

192.168.123.21 Hmaster

# 查看 hostname
[root@Hmaster ~]# hostnamectl
   Static hostname: Hmaster
         Icon name: computer-desktop
           Chassis: desktop
        Machine ID: 87af5eaaaf7c48f6b1cd9f42bfb51c76
           Boot ID: 9fe58388f3274f0e838b7cd1b720eacd
  Operating System: CentOS Linux 7 (Core)
       CPE OS Name: cpe:/o:centos:centos:7
            Kernel: Linux 3.10.0-1160.el7.x86_64
      Architecture: x86-64