使用Docker快速搭建Zookeeper和kafka集群-創(chuàng)新互聯(lián)

集群搭建

鏡像選擇

Zookeeper和Kafka集群分別運行在不同的容器中
zookeeper官方鏡像，版本3.4
kafka采用wurstmeister/kafka鏡像

專注于為中小企業(yè)提供成都做網(wǎng)站、成都網(wǎng)站設計服務,電腦端+手機端+微信端的三站合一,更高效的管理,為中小企業(yè)睢陽免費做網(wǎng)站提供優(yōu)質的服務。我們立足成都，凝聚了一批互聯(lián)網(wǎng)行業(yè)人才，有力地推動了成百上千家企業(yè)的穩(wěn)健成長，幫助中小企業(yè)通過網(wǎng)站建設實現(xiàn)規(guī)模擴充和轉變。

集群規(guī)劃

hostname	Ip addr	port	listener
zoo1	172.19.0.11	2184:2181
zoo2	172.19.0.12	2185:2181
zoo3	172.19.0.13	2186:2181
kafka1	172.19.0.14	9092:9092	kafka1
kafka2	172.19.0.15	9093:9093	kafka2
Kafka3	172.19.0.16	9094:9094	Kafka3
宿主機root OSX	192.168.21.139【DHCP獲取，會變動】

實現(xiàn)目標

kafka集群在docker網(wǎng)絡中可用，和zookeeper處于同一網(wǎng)絡
宿主機可以訪問zookeeper集群和kafka的broker list
docker重啟時集群自動重啟
集群的數(shù)據(jù)文件映射到宿主機器目錄中
使用yml文件和$ docker-compose up -d命令創(chuàng)建或重建集群

$ docker-compose up -d

zk集群的docker-compose.yml

version: '3.4'

services:
  zoo1:
    image: zookeeper
    restart: always
    hostname: zoo1
    container_name: zoo1
    ports:
    - 2184:2181
    volumes:
    - "/Users/shaozhipeng/Development/volume/zkcluster/zoo1/data:/data"
    - "/Users/shaozhipeng/Development/volume/zkcluster/zoo1/datalog:/datalog"
    environment:
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=0.0.0.0:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
    networks:
      br17219:
        ipv4_address: 172.19.0.11

  zoo2:
    image: zookeeper
    restart: always
    hostname: zoo2
    container_name: zoo2
    ports:
    - 2185:2181
    volumes:
    - "/Users/shaozhipeng/Development/volume/zkcluster/zoo2/data:/data"
    - "/Users/shaozhipeng/Development/volume/zkcluster/zoo2/datalog:/datalog"
    environment:
      ZOO_MY_ID: 2
      ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=0.0.0.0:2888:3888 server.3=zoo3:2888:3888
    networks:
      br17219:
        ipv4_address: 172.19.0.12

  zoo3:
    image: zookeeper
    restart: always
    hostname: zoo3
    container_name: zoo3
    ports:
    - 2186:2181
    volumes:
    - "/Users/shaozhipeng/Development/volume/zkcluster/zoo3/data:/data"
    - "/Users/shaozhipeng/Development/volume/zkcluster/zoo3/datalog:/datalog"
    environment:
      ZOO_MY_ID: 3
      ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=0.0.0.0:2888:3888
    networks:
      br17219:
        ipv4_address: 172.19.0.13

networks:
  br17219:
    external:
      name: br17219

kafka集群的docker-compose.yml

kfkluster少拼了個c...

version: '2'

services:
  kafka1:
    image: wurstmeister/kafka
    restart: always
    hostname: kafka1
    container_name: kafka1
    ports:
    - 9092:9092
    environment:
      KAFKA_ADVERTISED_HOST_NAME: kafka1
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: zoo1:2181,zoo2:2181,zoo3:2181
    volumes:
    - /Users/shaozhipeng/Development/volume/kfkluster/kafka1/logs:/kafka
    external_links:
    - zoo1
    - zoo2
    - zoo3
    networks:
      br17219:
        ipv4_address: 172.19.0.14

  kafka2:
    image: wurstmeister/kafka
    restart: always
    hostname: kafka2
    container_name: kafka2
    ports:
    - 9093:9093
    environment:
      KAFKA_ADVERTISED_HOST_NAME: kafka2
      KAFKA_ADVERTISED_PORT: 9093
      KAFKA_ZOOKEEPER_CONNECT: zoo1:2181,zoo2:2181,zoo3:2181
    volumes:
    - /Users/shaozhipeng/Development/volume/kfkluster/kafka2/logs:/kafka
    external_links:
    - zoo1
    - zoo2
    - zoo3
    networks:
      br17219:
        ipv4_address: 172.19.0.15

  kafka3:
    image: wurstmeister/kafka
    restart: always
    hostname: kafka3
    container_name: kafka3
    ports:
    - 9094:9094
    environment:
      KAFKA_ADVERTISED_HOST_NAME: kafka3
      KAFKA_ADVERTISED_PORT: 9094
      KAFKA_ZOOKEEPER_CONNECT: zoo1:2181,zoo2:2181,zoo3:2181
    volumes:
    - /Users/shaozhipeng/Development/volume/kfkluster/kafka3/logs:/kafka
    external_links:
    - zoo1
    - zoo2
    - zoo3
    networks:
      br17219:
        ipv4_address: 172.19.0.16

networks:
  br17219:
    external:
      name: br17219

結果查看和測試

宿主機命令行創(chuàng)建topic

$ pwd
/Users/shaozhipeng/Development/kafka_2.11-2.0.0/bin
$ ./kafka-topics.sh --create --zookeeper localhost:2184,localhost:2185,localhost:2186 --replication-factor 1 --partitions 1 --topic test1

Kafka Tool查看

docker ps查看正在運行的容器

查看宿主機IP地址，并設置Host

這樣宿主機就可以訪問kafka集群了

原文地址

另外有需要云服務器可以了解下創(chuàng)新互聯(lián)scvps.cn，海內外云服務器15元起步，三天無理由+7*72小時售后在線，公司持有idc許可證，提供“云服務器、裸金屬服務器、高防服務器、香港服務器、美國服務器、虛擬主機、免備案服務器”等云主機租用服務以及企業(yè)上云的綜合解決方案，具有“安全穩(wěn)定、簡單易用、服務可用性高、性價比高”等特點與優(yōu)勢，專為企業(yè)上云打造定制，能夠滿足用戶豐富、多元化的應用場景需求。

文章題目：使用Docker快速搭建Zookeeper和kafka集群-創(chuàng)新互聯(lián)
轉載來于：http://muchs.cn/article28/sppcp.html

成都網(wǎng)站建設公司_創(chuàng)新互聯(lián)，為您提供企業(yè)建站、靜態(tài)網(wǎng)站、做網(wǎng)站、企業(yè)網(wǎng)站制作、網(wǎng)站導航、網(wǎng)站營銷

廣告

聲明：本網(wǎng)站發(fā)布的內容（圖片、視頻和文字）以用戶投稿、用戶轉載內容為主，如果涉及侵權請盡快告知，我們將會在第一時間刪除。文章觀點不代表本網(wǎng)站立場，如需處理請聯(lián)系客服。電話：028-86922220；郵箱：631063699@qq.com。內容未經(jīng)允許不得轉載，或轉載時需注明來源： 創(chuàng)新互聯(lián)