這篇文章主要為大家展示了“centos6.5怎么安裝hadoop1.2.1”,內(nèi)容簡而易懂,條理清晰,希望能夠幫助大家解決疑惑,下面讓小編帶領(lǐng)大家一起研究并學(xué)習(xí)一下“centos6.5怎么安裝hadoop1.2.1”這篇文章吧。
創(chuàng)新互聯(lián)建站專注為客戶提供全方位的互聯(lián)網(wǎng)綜合服務(wù),包含不限于網(wǎng)站設(shè)計(jì)、成都網(wǎng)站設(shè)計(jì)、蕭縣網(wǎng)絡(luò)推廣、微信平臺(tái)小程序開發(fā)、蕭縣網(wǎng)絡(luò)營銷、蕭縣企業(yè)策劃、蕭縣品牌公關(guān)、搜索引擎seo、人物專訪、企業(yè)宣傳片、企業(yè)代運(yùn)營等,從售前售中售后,我們都將竭誠為您服務(wù),您的肯定,是我們大的嘉獎(jiǎng);創(chuàng)新互聯(lián)建站為所有大學(xué)生創(chuàng)業(yè)者提供蕭縣建站搭建服務(wù),24小時(shí)服務(wù)熱線:028-86922220,官方網(wǎng)址:muchs.cn本篇只簡單介紹安裝步驟
1. 角色分配
10.11.84.4 web-crawler--1.novalocal master/slave 10.11.84.5 web-crawler--2.novalocal slave 10.11.84.6 web-crawler--3.novalocal slave
2.安裝目錄 /usr/local/hadoop (HADOOP_HOME)
3,創(chuàng)建group hadoop
groupadd hadoop
4.創(chuàng)建用戶hadoop
useradd -d /home/hadoop -g hadoop hadoop chown -R hadoop:hadoop /usr/local/hadoop
5. 每臺(tái)機(jī)器安裝jdk(安裝在相同的目錄)
1)查看java 版本,可以看到j(luò)dk為openjdk,最好更換為sun jdk
[hadoop@hadoop2 ~]$ java -version openjdk version "1.8.0_131" OpenJDK Runtime Environment (build 1.8.0_131-b12) OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode) [hadoop@hadoop2 ~]$ rpm -qa | grep java java-1.8.0-openjdk-headless-1.8.0.131-11.b12.el7.x86_64 javapackages-tools-3.4.1-11.el7.noarch tzdata-java-2017b-1.el7.noarch java-1.7.0-openjdk-headless-1.7.0.141-2.6.10.5.el7.x86_64 java-1.7.0-openjdk-1.7.0.141-2.6.10.5.el7.x86_64 java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64 python-javapackages-3.4.1-11.el7.noarch
2)卸載openjdk
rpm -qa | grep java | xargs yum -y remove
3)安裝sunjdk
/usr/local 目錄下解壓jdk-8u181-linux-i586.tar.gz 并命名為jdk
配置環(huán)境變量如下
vim /etc/profile 末尾添加如下內(nèi)容
export JAVA_HOME=/usr/local/jdk export JRE_HOME=${JAVA_HOME}/jre export PATH=${JAVA_HOME}/bin:${PATH} export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib:${CLASSPATH}
4)查看jdk是否安裝成功
[root@hadoop2 local]# java -version java version "1.8.0_151" Java(TM) SE Runtime Environment (build 1.8.0_151-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode)
6. master 到 slave ssh 免密碼登錄
在每臺(tái)機(jī)器hadoop用戶下執(zhí)行如下命令
[hadoop@hadoop1 ~]$ ssh-keygen -t rsa -P '' #一路直接回車 發(fā)送master 機(jī)器~/.ssh/id_rsa.pub 到所有slave節(jié)點(diǎn) Scp id_rsa.pub hadoop@10.11.84.5:~ Scp id_rsa.pub hadoop@10.11.84.6:~ 讓后在每臺(tái)slave節(jié)點(diǎn)hadoop 用戶下執(zhí)行如下命令 Cd ~/.ssh Cat ../id_rsa.pub >> authorized_keys Chmod 644 authorized_keys
7.關(guān)閉所有機(jī)器防火墻、selinux
1)centos 7 以下
防火墻
查看防火墻狀態(tài)
service iptables status
臨時(shí)關(guān)閉防火墻
service iptables stop
永久關(guān)閉防火墻
chkconfig iptables off
selinux
setenforce 0 #臨時(shí)關(guān)閉
永久關(guān)閉需要修改如下文件
vim /etc/selinux/config
將SELINUX=enforcing改為SELINUX=disabled
2)centos 7
firewall-cmd --reload #重啟firewall
systemctl stop firewalld.service #停止firewall
systemctl disable firewalld.service #禁止firewall開機(jī)啟動(dòng)
firewall-cmd --state #查看默認(rèn)防火墻狀態(tài)(關(guān)閉后顯示notrunning,開啟后顯示running)
selinux 同上
8.修改hadoop配置文件
Core-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl" rel="external nofollow" rel="external nofollow" rel="external nofollow" ?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>hadoop.tmp.dir</name> <value>/home/hadoop/tmp</value> <description>Abase for other temporary directories.</description> </property> <property> <name>fs.default.name</name> <value>hdfs://web-crawler--1.novalocal:9000</value> </property> </configuration> hadoop-env.sh 添加java_home export JAVA_HOME=/usr/local/jdk hdfs-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl" rel="external nofollow" rel="external nofollow" rel="external nofollow" ?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>dfs.replication</name> <value>2</value> </property> <property> <name>dfs.name.dir</name> <value>/home/hadoop/name</value> </property> <property> <name>dfs.data.dir</name> <value>/home/hadoop/data</value> </property> </configuration> mapred-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl" rel="external nofollow" rel="external nofollow" rel="external nofollow" ?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>mapred.job.tracker</name> <value>http://web-crawler--1.novalocal:9001</value>;;; </property> <property> <name>mapred.system.dir</name> <value>/home/hadoop/mapred/system</value> </property> <property> <name>mapred.local.dir</name> <value>/home/hadoop/mapred/local</value> </property> </configuration>
5)masters
web-crawler--1.novalocal
6)slaves
web-crawler--1.novalocal web-crawler--2.novalocal web-crawler--3.novalocal
9. 修改環(huán)境變量,添加HADOOP_HOME 和JAVA_HOME
Vim /etc/profile (root) export ANT_HOME=/usr/local/ant export PATH=${PATH}:${ANT_HOME}/bin export HADOOP_HOME=/usr/local/hadoop-1.2.1 export JAVA_HOME=/usr/local/jdk export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH export CLASSPATH=$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$CLASSPATH export PATH=/usr/local/lib/phantomjs/bin:$PATH export ANT_HOME=/usr/local/ant export PATH=${PATH}:${ANT_HOME}/bin export PATH=$PATH:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin
10.啟動(dòng)hadoop
1)在master上format namenode
hadoop namenode -format
2)啟動(dòng)hadoop
sh /user/local/hadoop/bin/start-all.sh
11.進(jìn)行簡單測(cè)試
Hadoop fs -mkdir -p /user/hadoop/private/in Hadoop fs -put /user/local/hadoop/bin/start-all.sh /user/hadoop/private/in Cd /usr/local/hadoop Hadoop jar hadoop-examples-1.2.1.jar wordcount /user/hadoop/private/in /user/hadoop/private/out
以上是“centos6.5怎么安裝hadoop1.2.1”這篇文章的所有內(nèi)容,感謝各位的閱讀!相信大家都有了一定的了解,希望分享的內(nèi)容對(duì)大家有所幫助,如果還想學(xué)習(xí)更多知識(shí),歡迎關(guān)注創(chuàng)新互聯(lián)行業(yè)資訊頻道!
新聞名稱:centos6.5怎么安裝hadoop1.2.1-創(chuàng)新互聯(lián)
新聞來源:http://muchs.cn/article20/deggjo.html
成都網(wǎng)站建設(shè)公司_創(chuàng)新互聯(lián),為您提供定制網(wǎng)站、網(wǎng)站收錄、軟件開發(fā)、微信小程序、全網(wǎng)營銷推廣、網(wǎng)頁設(shè)計(jì)公司
聲明:本網(wǎng)站發(fā)布的內(nèi)容(圖片、視頻和文字)以用戶投稿、用戶轉(zhuǎn)載內(nèi)容為主,如果涉及侵權(quán)請(qǐng)盡快告知,我們將會(huì)在第一時(shí)間刪除。文章觀點(diǎn)不代表本網(wǎng)站立場(chǎng),如需處理請(qǐng)聯(lián)系客服。電話:028-86922220;郵箱:631063699@qq.com。內(nèi)容未經(jīng)允許不得轉(zhuǎn)載,或轉(zhuǎn)載時(shí)需注明來源: 創(chuàng)新互聯(lián)
猜你還喜歡下面的內(nèi)容