快速配置本地hadoop集群(1namenode4datanodes)
2021/11/29 17:08:48
本文主要是介绍快速配置本地hadoop集群(1namenode4datanodes),对大家解决编程问题具有一定的参考价值,需要的程序猿们随着小编来一起学习吧!
安装docker
不多说
clone github仓库
https://github.com/big-data-europe/docker-hadoop
修改其中的docker-compose.yml
修改为如下
version: "3" services: namenode: image: bde2020/hadoop-namenode:2.0.0-hadoop3.1.3-java8 container_name: namenode ports: - 9870:9870 volumes: - hadoop_namenode:/hadoop/dfs/name environment: - CLUSTER_NAME=test env_file: - ./hadoop.env datanode1: image: bde2020/hadoop-datanode:2.0.0-hadoop3.1.3-java8 container_name: datanode1 volumes: - hadoop_datanode1:/hadoop/dfs/data environment: SERVICE_PRECONDITION: "namenode:9870" env_file: - ./hadoop.env depends_on: - namenode datanode2: image: bde2020/hadoop-datanode:2.0.0-hadoop3.1.3-java8 container_name: datanode2 volumes: - hadoop_datanode2:/hadoop/dfs/data environment: SERVICE_PRECONDITION: "namenode:9870" env_file: - ./hadoop.env depends_on: - namenode datanode3: image: bde2020/hadoop-datanode:2.0.0-hadoop3.1.3-java8 container_name: datanode3 volumes: - hadoop_datanode3:/hadoop/dfs/data environment: SERVICE_PRECONDITION: "namenode:9870" env_file: - ./hadoop.env depends_on: - namenode datanode4: image: bde2020/hadoop-datanode:2.0.0-hadoop3.1.3-java8 container_name: datanode4 volumes: - hadoop_datanode4:/hadoop/dfs/data environment: SERVICE_PRECONDITION: "namenode:9870" env_file: - ./hadoop.env depends_on: - namenode resourcemanager: image: bde2020/hadoop-resourcemanager:2.0.0-hadoop3.1.3-java8 container_name: resourcemanager environment: SERVICE_PRECONDITION: "namenode:9870 datanode1:9864 datanode2:9864 datanode3:9864 datanode4:9864" env_file: - ./hadoop.env depends_on: - namenode - datanode1 - datanode2 - datanode3 - datanode4 nodemanager1: image: bde2020/hadoop-nodemanager:2.0.0-hadoop3.1.3-java8 container_name: nodemanager environment: SERVICE_PRECONDITION: "namenode:9870 datanode1:9864 datanode2:9864 datanode3:9864 datanode4:9864 resourcemanager:8088" env_file: - ./hadoop.env depends_on: - namenode - datanode1 - datanode2 - datanode3 - datanode4 historyserver: image: bde2020/hadoop-historyserver:2.0.0-hadoop3.1.3-java8 container_name: historyserver environment: SERVICE_PRECONDITION: "namenode:9870 datanode1:9864 datanode2:9864 datanode3:9864 datanode4:9864 resourcemanager:8088" volumes: - hadoop_historyserver:/hadoop/yarn/timeline env_file: - ./hadoop.env depends_on: - namenode - datanode1 - datanode2 - datanode3 - datanode4 volumes: hadoop_namenode: hadoop_datanode1: hadoop_datanode2: hadoop_datanode3: hadoop_datanode4: hadoop_historyserver:
运行docker
docker-compose up
这篇关于快速配置本地hadoop集群(1namenode4datanodes)的文章就介绍到这儿,希望我们推荐的文章对大家有所帮助,也希望大家多多支持为之网!
- 2023-05-13Windows下hadoop环境搭建之NameNode启动报错
- 2023-04-14hadoop伪分布式集群的安装(不是单机版)
- 2022-12-05Hadoop生态系统—数据仓库Hive的安装
- 2022-11-02Win10搭建Hadoop环境
- 2022-10-19Hadoop生态系统(数据仓库Hive的安装)
- 2022-10-03Hadoop、storm和Spark Streaming简单介绍
- 2022-10-03胖虎的Hadoop笔记——Hadoop的伪分布式部署
- 2022-09-11Ubuntu搭建全分布式Hadoop
- 2022-09-11Ubuntu搭建全分布式Hadoop
- 2022-09-09Ubuntu下安装伪分布式HADOOP遇到的一些问题