Flume对接HBASE报错:NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
2021/5/18 18:55:22
本文主要是介绍Flume对接HBASE报错:NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration,对大家解决编程问题具有一定的参考价值,需要的程序猿们随着小编来一起学习吧!
方案一:手动拷贝需要的相关jar包
flume对接HBASE报错,找不到相关的类:
报错hbase相关日志:
18 五月 2021 17:25:50,633 ERROR [lifecycleSupervisor-1-2] (org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:251) - Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@4b591129 counterGroup:{ name:null counters:{} } } - Exception follows. java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/protobuf/generated/MasterProtos$MasterService$BlockingInterface
则需要将hbase lib下的相关包拷贝到flume lib目录下:
scp hbase*.jar xxx.xx.x.xx:/home/root/programs/flume/lib/ scp metrics*.jar xxx.xx.x.xx:/home/root/programs/flume/lib/ scp netty*.jar xxx.xx.x.xx:/home/root/programs/flume/lib/ scp protobuf*.jar xxx.xx.x.xx:/home/root/programs/flume/lib/
报错zk相关日志:
18 五月 2021 16:55:13,343 ERROR [lifecycleSupervisor-1-2] (org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:251) - Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@4e3d6202 counterGroup:{ name:null counters:{} } } - Exception follows. java.lang.NoClassDefFoundError: org/apache/zookeeper/Watcher at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:756) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) at java.net.URLClassLoader.access$100(URLClassLoader.java:74) at java.net.URLClassLoader$1.run(URLClassLoader.java:369) at java.net.URLClassLoader$1.run(URLClassLoader.java:363) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:362) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at org.hbase.async.HBaseClient.<init>(HBaseClient.java:539) at org.apache.flume.sink.hbase.AsyncHBaseSink.initHBaseClient(AsyncHBaseSink.java:448) at org.apache.flume.sink.hbase.AsyncHBaseSink.start(AsyncHBaseSink.java:438) at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:45) at org.apache.flume.SinkRunner.start(SinkRunner.java:79) at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:249) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.Watcher at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 25 more
则需要把zk相关jar(zookeeper-3.4.6.jar、zookeeper-jute-3.5.8.jar)包拷贝到flume lib下 ;
方案二:
配置相关服务的环境变量
ps:本质这种框架一般都是指定入口类,调动相关jar包进行程序服务,缺少哪些类 报错日志一般会详细指明,还是要仔细观察相关报错日志内容
参考:
hbase连接测试时的常见错误
java.lang.ClassNotFoundException: org.apache.hadoop.hbase.HBaseConfiguration
https://www.fatalerrors.org/a/java.lang.noclassdeffounderror-io-netty-channel-eventloopgroup.html
这篇关于Flume对接HBASE报错:NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration的文章就介绍到这儿,希望我们推荐的文章对大家有所帮助,也希望大家多多支持为之网!
- 2024-06-26终极指南:Scrum中如何设置需求优先级
- 2024-06-26AI大模型企业应用实战(25)-为Langchain Agent添加记忆功能
- 2024-06-26小白家庭 nas 搭建方案-icode9专业技术文章分享
- 2024-06-23AI大模型企业应用实战(14)-langchain的Embedding
- 2024-06-23AI大模型企业应用实战(15)-langchain核心组件
- 2024-06-23AI大模型企业应用实战(16)-langchain核心组件
- 2024-06-23AI 大模型企业应用实战(06)-初识LangChain
- 2024-06-19EntBot.ai: AI Website Chatbot for Product Guides and Development Doc
- 2024-06-17zero-shot-learning-definition-examples-comparison
- 2024-06-06Package Easy(基于 NSIS 的打包exe安装包工具)使用方法-icode9专业技术文章分享