Bii o ṣe le Fi sori ẹrọ ati Tunto Hadoop Afun lori Node Kan ni CentOS 7
Apache Hadoop jẹ ilana Orisun Orisun ti a kọ fun ibi ipamọ data nla Big ati data ṣiṣe kọja awọn iṣupọ kọnputa. Ise agbese na da lori awọn paati atẹle:
- Hadlip wọpọ - o ni awọn ile-ikawe Java ati awọn ohun elo ti o nilo nipasẹ awọn modulu Hadoop miiran.
- HDFS - Eto Faili Pinpin Hadoop - Eto faili ti o ni iwọn ti Java ti o pin kaakiri awọn apa pupọ.
- MapReduce - ilana YARN fun afiwe data nla ti o jọra.
- YARN Hadoop: Ilana fun iṣakoso iṣupọ iṣupọ.
Nkan yii yoo ṣe itọsọna fun ọ lori bawo ni o ṣe le fi Apoo Hadoop sori ẹrọ ni iṣupọ node kan ni CentOS 7 (tun ṣiṣẹ fun awọn ẹya RHEL 7 ati Fedora 23 +). Iru iṣeto yii tun tọka si bi Ipo-pinpin Pinpin Hadoop.
Igbesẹ 1: Fi Java sori CentOS 7
1. Ṣaaju ki o to tẹsiwaju pẹlu fifi sori Java, buwolu wọle akọkọ pẹlu olumulo gbongbo tabi olumulo kan pẹlu awọn anfaani root lati seto orukọ olupin ẹrọ rẹ pẹlu aṣẹ atẹle.
# hostnamectl set-hostname master
Pẹlupẹlu, ṣafikun igbasilẹ tuntun ninu faili awọn ọmọ ogun pẹlu ẹrọ tirẹ FQDN lati tọka si Adirẹsi IP eto rẹ.
# vi /etc/hosts
Ṣafikun laini isalẹ:
192.168.1.41 master.hadoop.lan
Rọpo orukọ olupin ti o wa loke ati awọn igbasilẹ FQDN pẹlu awọn eto tirẹ.
2. Nigbamii, lọ si Oracle Java oju-iwe igbasilẹ ki o mu ẹya tuntun ti Java SE Development Kit 8 lori ẹrọ rẹ pẹlu iranlọwọ ti aṣẹ curl:
# curl -LO -H "Cookie: oraclelicense=accept-securebackup-cookie" “http://download.oracle.com/otn-pub/java/jdk/8u92-b14/jdk-8u92-linux-x64.rpm”
3. Lẹhin igbasilẹ Java alakomeji pari, fi sori ẹrọ package nipasẹ ipinfunni aṣẹ isalẹ:
# rpm -Uvh jdk-8u92-linux-x64.rpm
Igbesẹ 2: Fi Framework Hadoop sii ni CentOS 7
4. Itele, ṣẹda iroyin olumulo tuntun lori eto rẹ laisi awọn agbara gbongbo eyiti a yoo lo fun ọna fifi sori ẹrọ Hadoop ati agbegbe iṣẹ. Iwe itọsọna ile akọọlẹ tuntun yoo gbe inu /opt/hadoop
itọsọna.
# useradd -d /opt/hadoop hadoop # passwd hadoop
5. Ni igbesẹ ti n bọ ibewo Apache Hadoop oju-iwe lati le gba ọna asopọ fun ẹya iduroṣinṣin tuntun ati ṣe igbasilẹ iwe-akọọlẹ lori ẹrọ rẹ.
# curl -O http://apache.javapipe.com/hadoop/common/hadoop-2.7.2/hadoop-2.7.2.tar.gz
6. Fa jade ile-iwe ẹda ẹda ẹda akoonu itọsọna si ọna ile hadoop ile. Pẹlupẹlu, rii daju pe o yi awọn igbanilaaye awọn faili ti o dakọ ni ibamu.
# tar xfz hadoop-2.7.2.tar.gz # cp -rf hadoop-2.7.2/* /opt/hadoop/ # chown -R hadoop:hadoop /opt/hadoop/
7. Nigbamii, buwolu wọle pẹlu olumulo hadoop ati tunto Hadoop ati Awọn oniyipada Ayika Java lori ẹrọ rẹ nipa ṣiṣatunkọ faili .bash_profile
.
# su - hadoop $ vi .bash_profile
Fi awọn ila wọnyi si ni opin faili naa:
## JAVA env variables export JAVA_HOME=/usr/java/default export PATH=$PATH:$JAVA_HOME/bin export CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar ## HADOOP env variables export HADOOP_HOME=/opt/hadoop export HADOOP_COMMON_HOME=$HADOOP_HOME export HADOOP_HDFS_HOME=$HADOOP_HOME export HADOOP_MAPRED_HOME=$HADOOP_HOME export HADOOP_YARN_HOME=$HADOOP_HOME export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native" export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
8. Nisisiyi, bẹrẹ awọn oniyipada ayika ati ṣayẹwo ipo wọn nipa ipinfunni awọn ofin isalẹ:
$ source .bash_profile $ echo $HADOOP_HOME $ echo $JAVA_HOME
9. Lakotan, tunto ijẹrisi orisun ssh ti o da lori ijẹrisi hadoop nipa ṣiṣe awọn ofin isalẹ (rọpo orukọ olupin tabi FQDN lodi si ssh-copy-id
aṣẹ ni ibamu).
Paapaa, fi iwe aṣẹ kukuru silẹ ni òfo lati le buwolu wọle laifọwọyi nipasẹ ssh.
$ ssh-keygen -t rsa $ ssh-copy-id master.hadoop.lan