Hudi 系列(二)- 编译 hudi
程序员文章站
2022-07-14 20:33:39
...
编译 hudi
如果是第一次编译 hudi 会花很长时间,maven 本地仓库为空的情况下,我花了40分钟(视网络环境)。编译步骤如下:
- 下载 releace 源代码,https://hudi.apache.org/releases,现在的最新版本是0.7.0 。
- 安装 maven ,maven 的版本选择 3.6.3 。
- 配置 maven 的 setting.xml 为*仓库增加里面增加 aliyun 的 mirror 仓库 。
- 修改 hudi 项目 pom.xml 里面增加 aliyun repository 。
- 删除项目 pom.xml 中和 repo.spring.io 相关的 repository ,不然编译会出错,详见:https://github.com/apache/hudi/pull/2481,https://github.com/apache/hudi/pull/2481/commits/f67acf4d51010704188977b5090ef9c20325860f 。
- 执行 mvn package -DskipTests 开始编译 。
编译成功后,会看到如下信息:
[INFO] Reactor Summary for Hudi 0.7.0:
[INFO]
[INFO] Hudi ............................................... SUCCESS [ 5.237 s]
[INFO] hudi-common ........................................ SUCCESS [ 22.062 s]
[INFO] hudi-timeline-service .............................. SUCCESS [ 2.297 s]
[INFO] hudi-client ........................................ SUCCESS [ 0.136 s]
[INFO] hudi-client-common ................................. SUCCESS [ 4.842 s]
[INFO] hudi-hadoop-mr ..................................... SUCCESS [ 4.118 s]
[INFO] hudi-spark-client .................................. SUCCESS [ 9.274 s]
[INFO] hudi-sync-common ................................... SUCCESS [ 1.117 s]
[INFO] hudi-hive-sync ..................................... SUCCESS [ 3.633 s]
[INFO] hudi-spark-datasource .............................. SUCCESS [ 0.147 s]
[INFO] hudi-spark-common .................................. SUCCESS [ 5.556 s]
[INFO] hudi-spark2_2.11 ................................... SUCCESS [ 3.694 s]
[INFO] hudi-spark3_2.12 ................................... SUCCESS [ 5.367 s]
[INFO] hudi-spark_2.11 .................................... SUCCESS [ 15.311 s]
[INFO] hudi-utilities_2.11 ................................ SUCCESS [ 6.966 s]
[INFO] hudi-utilities-bundle_2.11 ......................... SUCCESS [ 21.478 s]
[INFO] hudi-cli ........................................... SUCCESS [ 12.717 s]
[INFO] hudi-java-client ................................... SUCCESS [ 2.497 s]
[INFO] hudi-flink-client .................................. SUCCESS [ 7.377 s]
[INFO] hudi-dla-sync ...................................... SUCCESS [ 3.680 s]
[INFO] hudi-sync .......................................... SUCCESS [ 0.069 s]
[INFO] hudi-hadoop-mr-bundle .............................. SUCCESS [ 20.377 s]
[INFO] hudi-hive-sync-bundle .............................. SUCCESS [ 4.636 s]
[INFO] hudi-spark-bundle_2.11 ............................. SUCCESS [ 18.165 s]
[INFO] hudi-presto-bundle ................................. SUCCESS [ 11.876 s]
[INFO] hudi-timeline-server-bundle ........................ SUCCESS [ 11.303 s]
[INFO] hudi-hadoop-docker ................................. SUCCESS [ 4.424 s]
[INFO] hudi-hadoop-base-docker ............................ SUCCESS [ 1.532 s]
[INFO] hudi-hadoop-namenode-docker ........................ SUCCESS [ 1.259 s]
[INFO] hudi-hadoop-datanode-docker ........................ SUCCESS [ 1.476 s]
[INFO] hudi-hadoop-history-docker ......................... SUCCESS [ 1.917 s]
[INFO] hudi-hadoop-hive-docker ............................ SUCCESS [ 7.409 s]
[INFO] hudi-hadoop-sparkbase-docker ....................... SUCCESS [ 1.876 s]
[INFO] hudi-hadoop-sparkmaster-docker ..................... SUCCESS [ 1.319 s]
[INFO] hudi-hadoop-sparkworker-docker ..................... SUCCESS [ 1.461 s]
[INFO] hudi-hadoop-sparkadhoc-docker ...................... SUCCESS [ 1.776 s]
[INFO] hudi-hadoop-presto-docker .......................... SUCCESS [ 1.577 s]
[INFO] hudi-integ-test .................................... SUCCESS [ 35.448 s]
[INFO] hudi-integ-test-bundle ............................. SUCCESS [03:10 min]
[INFO] hudi-examples ...................................... SUCCESS [ 28.104 s]
[INFO] hudi-flink_2.11 .................................... SUCCESS [ 13.175 s]
[INFO] hudi-flink-bundle_2.11 ............................. SUCCESS [ 24.294 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 08:42 min
[INFO] Finished at: 2021-01-27T16:35:12+08:00
[INFO] ------------------------------------------------------------------------