Hudi编译(0.10.1版本)
2022/2/16 23:45:14
本文主要是介绍Hudi编译(0.10.1版本)，对大家解决编程问题具有一定的参考价值，需要的程序猿们随着小编来一起学习吧！
一、编译
1.下载源码
https://www.apache.org/dyn/closer.lua/hudi/0.10.1/hudi-0.10.1.src.tgz2.编译
mvn clean install -DskipTests -DskipITs -Dscala-2.11 -Dspark3 报以下错误[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project hudi-integ-test: Compilation failure: Compilation failure: [ERROR] /Users/FengZhen/Desktop/Hadoop/hudi/hudi-0.10.1/hudi-integ-test/src/main/java/org/apache/hudi/integ/testsuite/helpers/ZookeeperServiceProvider.java:[23,52] 程序包org.apache.hudi.common.testutils.minicluster不存在 [ERROR] /Users/FengZhen/Desktop/Hadoop/hudi/hudi-0.10.1/hudi-integ-test/src/main/java/org/apache/hudi/integ/testsuite/helpers/ZookeeperServiceProvider.java:[39,7] 找不到符号 [ERROR] 符号: 类 ZookeeperTestService [ERROR] 位置: 类 org.apache.hudi.integ.testsuite.helpers.ZookeeperServiceProvider [ERROR] /Users/FengZhen/Desktop/Hadoop/hudi/hudi-0.10.1/hudi-integ-test/src/main/java/org/apache/hudi/integ/testsuite/helpers/ZookeeperServiceProvider.java:[39,55] 找不到符号 [ERROR] 符号: 类 ZookeeperTestService [ERROR] 位置: 类 org.apache.hudi.integ.testsuite.helpers.ZookeeperServiceProvider [ERROR] /Users/FengZhen/Desktop/Hadoop/hudi/hudi-0.10.1/hudi-integ-test/src/main/java/org/apache/hudi/integ/testsuite/helpers/ZookeeperServiceProvider.java:[46,7] 找不到符号 [ERROR] 符号: 类 ZookeeperTestService [ERROR] 位置: 类 org.apache.hudi.integ.testsuite.helpers.ZookeeperServiceProvider [ERROR] /Users/FengZhen/Desktop/Hadoop/hudi/hudi-0.10.1/hudi-integ-test/src/main/java/org/apache/hudi/integ/testsuite/helpers/ZookeeperServiceProvider.java:[46,55] 找不到符号解决：注释掉pom.xml文件中的
<module>hudi-integ-test</module> <module>packaging/hudi-integ-test-bundle</module>再次编译即可。
[INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Hudi ............................................... SUCCESS [ 4.854 s] [INFO] hudi-common ........................................ SUCCESS [ 29.059 s] [INFO] hudi-aws ........................................... SUCCESS [ 4.438 s] [INFO] hudi-timeline-service .............................. SUCCESS [ 2.899 s] [INFO] hudi-client ........................................ SUCCESS [ 0.146 s] [INFO] hudi-client-common ................................. SUCCESS [ 11.748 s] [INFO] hudi-hadoop-mr ..................................... SUCCESS [ 4.333 s] [INFO] hudi-spark-client .................................. SUCCESS [ 29.215 s] [INFO] hudi-sync-common ................................... SUCCESS [ 0.876 s] [INFO] hudi-hive-sync ..................................... SUCCESS [ 4.108 s] [INFO] hudi-spark-datasource .............................. SUCCESS [ 0.142 s] [INFO] hudi-spark-common_2.12 ............................. SUCCESS [ 14.039 s] [INFO] hudi-spark3_2.12 ................................... SUCCESS [ 9.784 s] [INFO] hudi-spark_2.12 .................................... SUCCESS [ 46.734 s] [INFO] hudi-utilities_2.12 ................................ SUCCESS [ 6.831 s] [INFO] hudi-utilities-bundle_2.12 ......................... SUCCESS [ 17.024 s] [INFO] hudi-cli ........................................... SUCCESS [ 14.301 s] [INFO] hudi-java-client ................................... SUCCESS [ 1.872 s] [INFO] hudi-flink-client .................................. SUCCESS [ 6.390 s] [INFO] hudi-spark2_2.12 ................................... SUCCESS [ 20.415 s] [INFO] hudi-dla-sync ...................................... SUCCESS [ 1.836 s] [INFO] hudi-sync .......................................... SUCCESS [ 0.068 s] [INFO] hudi-hadoop-mr-bundle .............................. SUCCESS [ 3.679 s] [INFO] hudi-hive-sync-bundle .............................. SUCCESS [ 1.424 s] [INFO] hudi-spark3.1.2-bundle_2.12 ........................ SUCCESS [ 7.476 s] [INFO] hudi-presto-bundle ................................. SUCCESS [ 4.860 s] [INFO] hudi-timeline-server-bundle ........................ SUCCESS [ 3.708 s] [INFO] hudi-hadoop-docker ................................. SUCCESS [ 0.693 s] [INFO] hudi-hadoop-base-docker ............................ SUCCESS [ 0.484 s] [INFO] hudi-hadoop-namenode-docker ........................ SUCCESS [ 0.071 s] [INFO] hudi-hadoop-datanode-docker ........................ SUCCESS [ 0.057 s] [INFO] hudi-hadoop-history-docker ......................... SUCCESS [ 0.057 s] [INFO] hudi-hadoop-hive-docker ............................ SUCCESS [ 0.694 s] [INFO] hudi-hadoop-sparkbase-docker ....................... SUCCESS [ 0.130 s] [INFO] hudi-hadoop-sparkmaster-docker ..................... SUCCESS [ 0.062 s] [INFO] hudi-hadoop-sparkworker-docker ..................... SUCCESS [ 0.072 s] [INFO] hudi-hadoop-sparkadhoc-docker ...................... SUCCESS [ 0.067 s] [INFO] hudi-hadoop-presto-docker .......................... SUCCESS [ 0.303 s] [INFO] hudi-examples ...................................... SUCCESS [ 10.590 s] [INFO] hudi-flink_2.12 .................................... SUCCESS [ 34.163 s] [INFO] hudi-kafka-connect ................................. SUCCESS [ 13.265 s] [INFO] hudi-flink-bundle_2.12 ............................. SUCCESS [ 13.849 s] [INFO] hudi-kafka-connect-bundle .......................... SUCCESS [ 11.594 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 05:39 min [INFO] Finished at: 2022-02-08T11:31:28+08:00 [INFO] Final Memory: 340M/2823M [INFO] ------------------------------------------------------------------------
3.测试 cd /Users/FengZhen/Desktop/Hadoop/hudi/hudi-0.10.1/hudi-cli ./hudi-cli.sh
SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/Users/FengZhen/Desktop/Hadoop/hudi/hudi-0.10.1/hudi-cli/target/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/Users/FengZhen/Desktop/Hadoop/hudi/hudi-0.10.1/hudi-cli/target/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 0 [main] INFO org.springframework.beans.factory.xml.XmlBeanDefinitionReader - Loading XML bean definitions from URL [jar:file:/Users/FengZhen/Desktop/Hadoop/hudi/hudi-0.10.1/hudi-cli/target/hudi-cli-0.10.1.jar!/META-INF/spring/spring-shell-plugin.xml] 424 [main] INFO org.springframework.context.support.GenericApplicationContext - Refreshing org.springframework.context.support.GenericApplicationContext@69ea3742: startup date [Tue Feb 08 14:02:51 CST 2022]; root of context hierarchy 649 [main] INFO org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor - JSR-330 'javax.inject.Inject' annotation found and supported for autowiring Table command getting loaded HoodieSplashScreen loaded =================================================================== * ___ ___ * * /\__\ ___ /\ \ ___ * * / / / /\__\ / \ \ /\ \ * * / /__/ / / / / /\ \ \ \ \ \ * * / \ \ ___ / / / / / \ \__\ / \__\ * * / /\ \ /\__\ / /__/ ___ / /__/ \ |__| / /\/__/ * * \/ \ \/ / / \ \ \ /\__\ \ \ \ / / / /\/ / / * * \ / / \ \ / / / \ \ / / / \ /__/ * * / / / \ \/ / / \ \/ / / \ \__\ * * / / / \ / / \ / / \/__/ * * \/__/ \/__/ \/__/ Apache Hudi CLI * * * =================================================================== Welcome to Apache Hudi CLI. Please type help if you are looking for help. hudi->version 1.0/Users/FengZhen/Desktop/Hadoop/hudi/hudi-0.10.1/packaging此目录下就是hudi与各组件集成的jar包 Success.
这篇关于Hudi编译(0.10.1版本)的文章就介绍到这儿，希望我们推荐的文章对大家有所帮助，也希望大家多多支持为之网！
- 2024-05-15鸿蒙生态设备数量超8亿台
- 2024-05-13TiDB + ES：转转业财系统亿级数据存储优化实践
- 2024-05-09“2024鸿蒙零基础快速实战-仿抖音App开发（ArkTS版）”实战课程已上线
- 2024-05-09聊聊如何通过arthas-tunnel-server来远程管理所有需要arthas监控的应用
- 2024-05-09log4j2这么配就对了
- 2024-05-09nginx修改Content-Type
- 2024-05-09Redis多数据源，看这篇就够了
- 2024-05-09Google Chrome驱动程序 124.0.6367.62（正式版本）去哪下载？
- 2024-05-09有没有大佬知道这种数据应该怎么抓取呀？
- 2024-05-09这种运行结果里的10.100000001，怎么能最快改成10.1？
