kafka部署时出现的天坑
2021/9/14 23:06:07
本文主要是介绍kafka部署时出现的天坑,对大家解决编程问题具有一定的参考价值,需要的程序猿们随着小编来一起学习吧!
前言
最近在部署kafka单节点时遇到的问题,想着自己弄个单节点演示以下, 好家伙,单启动的时候就报了很多错误;为防止在坑,故将错误信息记录下来;以防万一!
kafka版本
2.4.1
部署环境
阿里云服务器,系统为centos 7.9
解决kafka启动报错 :Socket server failed to bind to 123.57.207.179:9092: 无法指定被请求的地址.
控制台整体错误信息如下
[2021-09-14 10:59:11,134] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) org.apache.kafka.common.KafkaException: Socket server failed to bind to 123.57.207.179:9092: 无法指定被请求的地址. at kafka.network.Acceptor.openServerSocket(SocketServer.scala:632) at kafka.network.Acceptor.<init>(SocketServer.scala:508) at kafka.network.SocketServer.kafka$network$SocketServer$$createAcceptor(SocketServer.scala:271) at kafka.network.SocketServer$$anonfun$createDataPlaneAcceptorsAndProcessors$1.apply(SocketServer.scala:240) at kafka.network.SocketServer$$anonfun$createDataPlaneAcceptorsAndProcessors$1.apply(SocketServer.scala:238) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at kafka.network.SocketServer.createDataPlaneAcceptorsAndProcessors(SocketServer.scala:238) at kafka.network.SocketServer.startup(SocketServer.scala:121) at kafka.server.KafkaServer.startup(KafkaServer.scala:263) at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44) at kafka.Kafka$.main(Kafka.scala:84) at kafka.Kafka.main(Kafka.scala) Caused by: java.net.BindException: 无法指定被请求的地址 at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:433) at sun.nio.ch.Net.bind(Net.java:425) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67) at kafka.network.Acceptor.openServerSocket(SocketServer.scala:628) ... 12 more
原因
其实是kafka的监听地址有问题,不知道为啥,server.properties配置文件中的listeners=PLAINTEXT://123.57.66.105:9092
选项
一定要使用内网ip,我这边使用的是公网ip地址(我用的是阿里云服务器),
解决步骤
1、先获取当前系统的内网ip地址
[root@iZ2zee7r7jat94g6t1guhtZ ~]# ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 172.21.227.173 netmask 255.255.240.0 broadcast 172.21.239.255 inet6 fe80::216:3eff:fe06:1775 prefixlen 64 scopeid 0x20<link> ether 00:16:3e:06:17:75 txqueuelen 1000 (Ethernet) RX packets 4053113 bytes 1013685895 (966.7 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 3283585 bytes 722364027 (688.9 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
可以看到。内网ip地址为:172.21.227.173
2、重新设置kafka的server.properties配置文件
接下来,重新设置kafka的server.properties配置文件,将公网ip改为内网ip,使用编辑器打开server.properties
vim /usr/local/kafka/kafka_2.11-2.4.1/server.properties
找到listeners=PLAINTEXT://123.57.66.105:9092
,将其改为内网ip
listeners=PLAINTEXT://172.21.227.173:9092
3、重新启动kafka
启动kafka ,-daemon
表示守护进程,也就是后台运行
/usr/local/kafka/kafka_2.11-2.4.1/bin/kafka-server-start.sh -daemon /usr/local/kafka/kafka_2.11-2.4.1/config/server.properties
大功告成
解决报错 Timed out waiting for connection while in state: CONNECTING
在启动kafka时报错如下信息
[2021-09-14 11:43:55,494] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply$mcV$sp(ZooKeeperClient.scala:259) at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply(ZooKeeperClient.scala:255) at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply(ZooKeeperClient.scala:255) at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:253) at kafka.zookeeper.ZooKeeperClient.kafka$zookeeper$ZooKeeperClient$$waitUntilConnected(ZooKeeperClient.scala:255) at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:113) at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1857) at kafka.server.KafkaServer.kafka$server$KafkaServer$$createZkClient$1(KafkaServer.scala:374) at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:399) at kafka.server.KafkaServer.startup(KafkaServer.scala:207) at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44) at kafka.Kafka$.main(Kafka.scala:84) at kafka.Kafka.main(Kafka.scala)
原因
错误很明显,就是超时了,仔细看,里面报的错误是ZooKeeperClientTimeoutException
异常,这就是代表kafka在连接zk的时候超时了;一开始以为是zk挂了,为了解决这个问题,我特意去看了zookeeper,发现是正在运行的,并没有出问题;后来才发现是kafka的配置错了;在kafka的server.properties配置文件中,有一个配置为:zookeeper.connect=123.57.207.179:2181
,这个配置将kafka添加zk的节点上,但是我这里配的是外网ip(我用的是阿里云服务器),要解决这个问题其实很简单,只需要将外网ip改为内网ip即可;
解决步骤
1、获取内网ip
通过以下命令即可查看内网ip地址
ifconfig # 运行后,得到的内网ip为: 172.21.227.173
2、重新设置kafka的server.properties配置文件
接下来,重新设置kafka的server.properties配置文件,将公网ip改为内网ip,使用编辑器打开server.properties
vim /usr/local/kafka/kafka_2.11-2.4.1/server.properties
找到zookeeper.connect=123.57.207.179:2181
,将其改为内网ip
zookeeper.connect=172.21.227.173:2181
3、重新启动kafka
大功告成
java连接kafka超时 ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
遇到这种问题有如下解决方案
1、 检查zk是否运行
2、 检查linux是否开放了9092端口(kafka默认端口)
3、 检查阿里云服务器是否配置了安全组(我的就是这个问题导致的),安全组可以配置能被外网访问的端口以及黑白名单;安全组配置如下:
这篇关于kafka部署时出现的天坑的文章就介绍到这儿,希望我们推荐的文章对大家有所帮助,也希望大家多多支持为之网!
- 2024-12-21MQ-2烟雾传感器详解
- 2024-12-09Kafka消息丢失资料:新手入门指南
- 2024-12-07Kafka消息队列入门:轻松掌握Kafka消息队列
- 2024-12-07Kafka消息队列入门:轻松掌握消息队列基础知识
- 2024-12-07Kafka重复消费入门:轻松掌握Kafka消费的注意事项与实践
- 2024-12-07Kafka重复消费入门教程
- 2024-12-07RabbitMQ入门详解:新手必看的简单教程
- 2024-12-07RabbitMQ入门:新手必读教程
- 2024-12-06Kafka解耦学习入门教程
- 2024-12-06Kafka入门教程:快速上手指南