Bii o ṣe le Fi sori ẹrọ ati Tunto Oluṣakoso Cloudera lori CentOS/RHEL 7 - Apá 3

Ninu àpilẹkọ yii, a ṣe apejuwe igbesẹ nipasẹ ilana igbesẹ lati fi Oluṣakoso Cloudera sori ẹrọ gẹgẹbi fun awọn iṣe ile-iṣẹ. Ni Apá 2, a ti kọja tẹlẹ nipasẹ Cloudera Pre-requisites, rii daju pe gbogbo awọn olupin ti pese daradara.

Awọn adaṣe ti o dara julọ fun Ṣiṣe olupin Server Hadoop lori CentOS/RHEL 7 - Apá 1
Ṣiṣeto Awọn ohun ti o nilo ṣaaju Hadoop ati Ikunkun Aabo - Apá 2

Nibi a yoo ni iṣupọ ipade 5 nibiti awọn oluwa 2 ati awọn oṣiṣẹ 3 ṣe. Mo ti lo awọn iṣẹlẹ 5 AWS EC2 lati ṣe afihan ilana fifi sori ẹrọ. Mo ti darukọ awọn olupin 5 wọnyẹn ni isalẹ.

master1.linux-console.net
master2.linux-console.net
worker1.linux-console.net
worker2.linux-console.net
worker3.linux-console.net

Oluṣakoso Cloudera jẹ irinṣẹ iṣakoso ati irinṣẹ ibojuwo fun gbogbo CDH. A n ṣakoso igbagbogbo n pe ni ọpa iṣakoso fun Cloudera Hadoop. A le fi ranṣẹ, ṣe atẹle, ṣakoso, ati ṣe awọn ayipada iṣeto pẹlu lilo ọpa yii. Eyi jẹ pataki pupọ lati ṣakoso gbogbo iṣupọ naa.

Ni isalẹ ni awọn lilo pataki ti Cloudera Manager.

Firanṣẹ ati tunto awọn iṣupọ Hadoop ni ọna adaṣe.
Atẹle ilera iṣupọ
Tunto awọn itaniji
Laasigbotitusita Laasigbotitusita
Riroyin
Ṣiṣe Iroyin Iṣamulo iṣupọ
Awọn atunto Awọn orisun ni agbara

Igbesẹ 1: Fifi Olupin Wẹẹbu Afun lori CentOS

A yoo lo master1 bi oluṣakoso wẹẹbu fun awọn ibi ipamọ Cloudera. Pẹlupẹlu, Cloudera Manager ni WebUI, nitorinaa a nilo lati fi Apache sori ẹrọ. Tẹle awọn igbesẹ isalẹ lati fi sori ẹrọ olupin ayelujara apache.

# yum -y install httpd

Lọgan ti o fi sori ẹrọ httpd, bẹrẹ rẹ ki o muu ṣiṣẹ ki o yoo bẹrẹ lori bata.

# systemctl start httpd
# systemctl enable httpd

Lẹhin ti o bẹrẹ httpd, rii daju ipo naa.

# systemctl status httpd

Lẹhin ti o bẹrẹ httpd, ṣii ẹrọ lilọ kiri ayelujara kan ninu eto agbegbe rẹ ki o lẹẹ mọ adirẹsi IP ti master1 ninu ọpa wiwa, o yẹ ki o gba oju-iwe idanwo yii lati rii daju pe httpd n ṣiṣẹ daradara.

Igbesẹ 2: Tunto DNS Agbegbe lati yanju IP ati Orukọ Ile-iṣẹ

A nilo lati ni olupin DNS kan tabi tunto/ati be be lo/awọn ogun lati yanju IP ati orukọ olupin. Nibi a n ṣatunṣe/ati be be lo/awọn ogun, ṣugbọn ni akoko gidi, olupin DNS ifiṣootọ yoo wa nibẹ fun agbegbe iṣelọpọ.

Tẹle awọn igbesẹ isalẹ lati ṣe titẹsi fun gbogbo awọn olupin rẹ ni/ati be be lo/awọn ogun.

# vi /etc/hosts

Eyi yẹ ki o wa ni tunto ni gbogbo awọn olupin.

13.235.27.144   master1.linux-console.net     master1
13.235.135.170  master2.linux-console.net     master2
15.206.167.94   worker1.linux-console.net     worker1
13.232.173.158  worker2.linux-console.net     worker2
65.0.182.222    worker3.linux-console.net     worker3

Igbesẹ 3: Tunto Wiwọle Wiwọle Ọrọigbaniwọle SSH

Cloudera Manager ti wa ni fifi sori ẹrọ lori master1 ninu ifihan yii. A nilo lati tunto ọrọigbaniwọle-kere ssh lati master1 si gbogbo awọn apa miiran. Nitori Cloudera Manager yoo lo ssh lati ṣe ibaraẹnisọrọ gbogbo awọn apa miiran lati fi awọn idii sii.

Tẹle awọn igbesẹ isalẹ lati tunto ssh ọrọigbaniwọle-lati master1 si gbogbo awọn olupin to ku. A yoo ni olumulo ‘tecmint’ lati tẹsiwaju siwaju.

Ṣẹda olumulo 'tecmint' gbogbo awọn olupin 4 nipa lilo pipaṣẹ useradd bi o ti han.

# useradd -m tecmint

Lati fun anfaani gbongbo si olumulo 'tecmint', ṣafikun laini isalẹ sinu/ati be be lo/faili sudoers. O le ṣafikun laini yii labẹ gbongbo bi fifun ni sikirinifoto.

tecmint   ALL=(ALL)    ALL

Yipada si olumulo 'tecmint' ki o ṣẹda bọtini ssh ninu master1 ni lilo pipaṣẹ isalẹ.

# sudo su tecmint
$ ssh-keygen

Bayi daakọ bọtini ti o ṣẹda si gbogbo awọn olupin 4 nipa lilo pipaṣẹ ssh-copy-id bi o ti han.

$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email 
$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email 
$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email  
$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email

Bayi o yẹ ki o ni anfani lati ssh lati master1 lati ku gbogbo awọn olupin laisi ọrọigbaniwọle bi o ti han.

$ ssh master2
$ ssh worker1
$ ssh worker2
$ ssh worker3

Igbesẹ 4: Fifi sori ati tito leto Oluṣakoso Cloudera

A le lo ibi ipamọ olutaja (Cloudera) lati fi gbogbo awọn idii sii nipa lilo awọn irinṣẹ iṣakoso package ni RHEL/CentOS. Ni akoko gidi, ṣiṣẹda ibi ipamọ ti ara wa jẹ iṣe ti o dara julọ nitori a le ma ni iraye si intanẹẹti ninu awọn olupin iṣelọpọ.

Nibi a yoo fi ifilọlẹ Cloudera Manager 6.3.1 silẹ. Niwọn igba ti a yoo lo master1 bi olupin repo, a n ṣe igbasilẹ awọn idii ni ọna ti a darukọ ni isalẹ.

Ṣẹda awọn ilana ti a darukọ ni isalẹ lori olupin master1 .

$ sudo mkdir -p /var/www/html/cloudera-repos/cm6

A le lo ohun elo wget lati ṣe igbasilẹ awọn idii lori http. Nitorinaa, fi wget sii nipa lilo pipaṣẹ isalẹ.

$ sudo yum -y install wget

Nigbamii, ṣe igbasilẹ faili oda Cloudera Manager nipa lilo pipaṣẹ wget atẹle.

$ wget https://archive.cloudera.com/cm6/6.3.1/repo-as-tarball/cm6.3.1-redhat7.tar.gz

Jade faili oda sinu/var/www/html/Cloudera-repos/cm6, tẹlẹ a ti ṣe master1 bi webserver nipa fifi http sori ati pe a ti ni idanwo lori ẹrọ lilọ kiri ayelujara naa.

$ sudo tar xvfz cm6.3.1-redhat7.tar.gz -C /var/www/html/cloudera-repos/cm6 --strip-components=1

Bayi, rii daju pe gbogbo awọn faili Cloudera rpm wa nibẹ ni/var/www/html/Cloudera-repos/cm6/RPMS/x86_64 liana.

$ cd /var/www/html/cloudera-repos/cm6
$ ll

Ṣẹda awọn faili /etc/yum.repos.d/cloudera-manager.repo lori gbogbo awọn olupin ninu awọn ogun iṣupọ pẹlu akoonu atẹle, nibi master1 (65.0.101.148) ni olupin Wẹẹbu naa.

[cloudera-repo]
name=cloudera-manager
baseurl=http:///cloudera-repos/cm6/
enabled=1
gpgcheck=0

Bayi a ti fi ibi ipamọ sii, ṣiṣe aṣẹ ni isalẹ lati wo awọn ibi ipamọ ti o ṣiṣẹ.

$ yum repolist

Ṣiṣe aṣẹ ni isalẹ lati wo gbogbo awọn idii ti o jọmọ Cloudera ti o wa ninu ibi ipamọ.

$ yum list available | grep cloudera*

Fi olupin awọsanma-oluṣakoso-ẹrọ sori ẹrọ, Cloudera-manager-agent, oluṣakoso awọsanma-daemons Cloudera-manager-server-db-2.

$ sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server cloudera-manager-server-db-2

Ṣiṣe aṣẹ isalẹ lati wo gbogbo awọn idii Cloudera ti a fi sii.

$ yum list installed | grep cloudera*

Ṣiṣe aṣẹ ti o wa ni isalẹ lati bẹrẹ Cloudera-scm-server-db eyiti o jẹ ipilẹ data lati tọju Cloudera Manager ati awọn iṣẹ metadata miiran.

Nipa aiyipada, Cloudera n bọ pẹlu postgre-sql eyiti o wa ni ifibọ ninu Cloudera Manager. A n fi ọkan ti a fi sii sii, ni ibi ipamọ data ita gidi kan ti o le ṣee lo. O le jẹ Oracle, MySQL, tabi PostgreSQL.

$ sudo systemctl start cloudera-scm-server-db

Ṣiṣe aṣẹ isalẹ lati ṣayẹwo ipo ti ibi ipamọ data.

$ sudo systemctl status cloudera-scm-server-db

Tunto awọn db.properties fun olupin Manager Cloudera.

$ vi /etc/cloudera-scm-server/db.properties

Ṣe atunto iye ti o wa ni isalẹ jẹ EMBEDDED lati ṣe Cloudera Manager lo Awọn aaye data ifibọ.

com.cloudera.cmf.db.setupType=EMBEDDED

Ṣiṣe aṣẹ ti o wa ni isalẹ lati bẹrẹ olupin Oluṣakoso Cloudera.

$ sudo systemctl start cloudera-scm-server

Ṣiṣe aṣẹ ti o wa ni isalẹ lati ṣayẹwo ipo olupin olupin Cloudera.

$ sudo systemctl status cloudera-scm-server

Ṣiṣe aṣẹ ti o wa ni isalẹ lati bẹrẹ ati ṣayẹwo ipo ti oluṣakoso Manager Cloudera.

$ sudo systemctl start cloudera-scm-agent
$ sudo systemctl status cloudera-scm-agent

Ni kete ti Oluṣakoso Oluṣakoso Cloudera ṣaṣeyọri ati ṣiṣe dara, o le wo WebUI (Oju-iwe Wiwọle) ninu ẹrọ lilọ kiri ayelujara nipa lilo adiresi IP ati nọmba ibudo 7180 eyiti o jẹ nọmba ibudo ti Oluṣakoso Cloudera.

https://65.0.101.148:7180

Ninu àpilẹkọ yii, a ti rii igbesẹ nipasẹ ilana igbesẹ fun fifi sori Cloudera Manager lori CentOS 7. A yoo wo CDH ati awọn fifi sori ẹrọ iṣẹ miiran ni nkan ti n bọ.