Hadoop安装
[*]下载“hadoop-3.0.0-alpha4.tar.gz”，放到/home/hadoop目录下。下载地址：http://hadoop.apache.org
[*]解压，输入命令，tar -xzvf hadoop-3.0.0-alpha4.tar.gz
[*]在/home/hadoop目录下创建数据存放的文件夹，tmp、hdfs、hdfs/data、hdfs/name
[*]配置/home/hadoop/hadoop-3.0.0-alpha4/etc/hadoop目录下的core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://172.17.0.2:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>file:/home/hadoop/tmp</value>
</property>
<property>
<name>io.file.buffer.size</name>
<value>131702</value>
</property>
</configuration>配置/home/hadoop/hadoop-3.0.0-alpha4/etc/hadoop目录下的hdfs-site.xml
<configuration>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/home/hadoop/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/home/hadoop/dfs/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>172.17.0.2:9001</value>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
</configuration>配置/home/hadoop/hadoop-3.0.0-alpha4/etc/hadoop目录下的mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>172.17.0.2:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>172.17.0.2:19888</value>
</property>
</configuration>配置/home/hadoop/hadoop-3.0.0-alpha4/etc/hadoop目录下的yarn-site.xml
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>172.17.0.2:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>172.17.0.2:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>172.17.0.2:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>172.17.0.2:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>172.17.0.2:8088</value>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>768</value>
</property>
</configuration>配置/home/hadoop/hadoop-2.7.0/etc/hadoop目录下hadoop-env.sh、yarn-env.sh的JAVA_HOME，不设置的话，启动不了，
export JAVA_HOME=/home/java/jdk1.7.0_79
配置/home/hadoop/hadoop-2.7.0/etc/hadoop目录下的workers(注意，老版本中这里似乎 是slaves)，删除默认的localhost，增加2个从节点：
172.17.0.3
172.17.0.4配置用户，在start-dfs.sh与stop-dfs.sh文件中增加：
HDFS_DATANODE_USER=root
HADOOP_SECURE_DN_USER=hdfs
HDFS_NAMENODE_USER=root
HDFS_SECONDARYNAMENODE_USER=root如果不加如上配置，则会出现如下错误：
# sbin/start-all.sh
Starting namenodes on
ERROR: Attempting to launch hdfs namenode as root
ERROR: but there is no HDFS_NAMENODE_USER defined. Aborting launch.
Starting datanodes
ERROR: Attempting to launch hdfs datanode as root
ERROR: but there is no HDFS_DATANODE_USER defined. Aborting launch.
Starting secondary namenodes
ERROR: Attempting to launch hdfs secondarynamenode as root
ERROR: but there is no HDFS_SECONDARYNAMENODE_USER defined. Aborting launch.
页:
[1]
