Bii o ṣe le Fi sii ati Tunto Hive pẹlu Wiwa giga - Apá 7
Ile Agbon jẹ awoṣe ile ipamọ data ni Hadoop Eco-System. O le ṣe bi ohun elo ETL lori oke Hadoop. Muu Wiwa Giga (HA) lori Ile Agbon ko jọra bi a ṣe ni Awọn iṣẹ Titunto si bi Oluṣakoso Oro.
Failover aifọwọyi kii yoo ṣẹlẹ ni Ile Agbon (Hiveserver2). Ti eyikeyi Hiveserver2 (HS2) ba kuna, ṣiṣe awọn iṣẹ lori HS2 ti o kuna yoo kuna. A nilo lati tun fi iṣẹ naa pada ki iṣẹ naa le ṣiṣẹ lori HiveServer2 miiran. Nitorinaa, ṣiṣe HA lori HS2 kii ṣe nkankan ṣugbọn, jijẹ nọmba awọn paati HS2 ni Iṣupọ.
Ninu nkan yii, a yoo rii awọn igbesẹ lati fi sori ẹrọ ati mu Wiwa Ga ti Ile Agbon ṣiṣẹ.
- Awọn adaṣe ti o dara julọ fun Ṣiṣe olupin Server Hadoop lori CentOS/RHEL 7 - Apá 1
- Ṣiṣeto Awọn ohun ti o nilo ṣaaju Hadoop ati Ikunkun Aabo - Apá 2 Bii a ṣe le Fi sii ati Tunto Oluṣakoso Cloudera lori CentOS/RHEL 7 - Apá 3
- Bii o ṣe le Fi sii CDH ati Tunto Awọn ibi Iṣẹ lori CentOS/RHEL 7 - Apá 4
- Bii o ṣe le Ṣeto Wiwa giga fun Namenode - Apá 5
- Bii o ṣe le Ṣeto Wiwa Giga fun Oluṣakoso Oro - Apá 6
Jẹ ki a bẹrẹ…
Fifi sori Ile Agbon ati Iṣeto ni
1. Buwolu wọle si Oluṣakoso Cloudera ni URL ti o wa ni isalẹ ki o lọ kiri si Oluṣakoso Cloudera -> Ṣafikun Iṣẹ.
http://13.233.129.39:7180/cmf/home
2. Yan iṣẹ ‘Ile Agbon’.
3. Fi awọn iṣẹ naa si awọn apa.
- Ẹnubode - O jẹ iṣẹ alabara nibiti olumulo le wọle si Ile Agbon. Nigbagbogbo, iṣẹ yii yoo gbe sinu Awọn apa Edge ti a fiṣootọ si awọn olumulo.
- Ile Agbon Metastore - O jẹ ibi ipamọ ti aarin fun titoju Metadata Ile Agbon.
- Olupin WebHCat - O jẹ oju opo wẹẹbu API fun Katalogi ati Awọn Iṣẹ Hadoop miiran.
- Hiveserver2 - O jẹ wiwo ti awọn alabara fun ipaniyan ibeere lori Ile Agbon.
Lọgan ti o yan awọn olupin, tẹ 'Tẹsiwaju' lati tẹsiwaju.
4. Ile Agbon Metastore nilo ipilẹ data ipilẹ fun titoju Metadata. Nibi a nlo aiyipada ipilẹ PostgreSQL eyiti o jẹ inbuilt pẹlu CDH.
Ni isalẹ awọn alaye ibi ipamọ data ti a mẹnuba yoo wa ni titẹ laifọwọyi, ‘Asopọ Idanwo’ yoo foju bi a ti ṣẹda ipilẹ data ti a mẹnuba lori fifo. Ni akoko gidi, a nilo lati ṣẹda aaye data ni ibi ipamọ data ita ati idanwo asopọ lati tẹsiwaju siwaju. Lọgan ti o ti ṣe, jọwọ tẹ 'Tẹsiwaju'.
5. Tunto liana Ile-iṣẹ Ile Agbon,/olumulo/Ile Agbon/ile-itaja ni ọna itọsọna aiyipada fun titoju awọn tabili Ile Agbon. Tẹ 'Tẹsiwaju'.
6. Fifi sori ẹrọ ti Ile Agbon ti bẹrẹ.
7. Lọgan ti fifi sori ẹrọ pari, o le gba ipo ‘Pari’. Tẹ 'Tẹsiwaju' lati tẹsiwaju siwaju.
8. Fifi sori ẹrọ Hive ati iṣeto ni ti pari ni aṣeyọri. Tẹ 'Pari' lati pari ilana fifi sori ẹrọ.
9. O le wo iṣẹ Ile Agbon ti a ṣafikun ni Iṣupọ nipasẹ Dasibodu Oluṣakoso Cloudera.
10. O le wo Hiveserver2 ni Awọn apeere ti Ile Agbon. A ti ṣafikun Hiveserver2 ni master1.
Oluṣakoso Cloudera -> Ile Agbon -> Awọn apeere -> Hiveserver2.
Muu Wiwa Giga lori Ile Agbon
11. Atẹle ṣafikun ipa Ile Agbon nipasẹ lilọ si Oluṣakoso Cloudera -> Ile Agbon -> Awọn iṣe -> Ṣafikun Awọn Apejuwe Ipa.
12. Yan awọn olupin nibiti o fẹ gbe afikun Hiveserver2. O le ṣafikun diẹ sii ju meji lọ, ko si opin. Nibi a n ṣe afikun afikun Hiveserver2 kan ni master2.
13. Lọgan ti o yan olupin, tẹ 'Tẹsiwaju'.
14. A yoo fi Hiverserver2 kun sinu Awọn apejọ Ile Agbon, o nilo lati bẹrẹ nipasẹ lilọ si Cloudera Manager -> Ile Agbon -> Awọn apeere -> (Yan Hiveserver2 ti a ṣafikun tuntun) -> Iṣe fun Ti yan -> Ibẹrẹ.
15. Lọgan ti Hiveserver2 ti bẹrẹ lori master2, iwọ yoo gba ipo ‘Ti pari’. Tẹ Sunmọ.
16. O le wo, mejeeji Hiveerver2s n ṣiṣẹ.
Ṣiṣayẹwo Wiwa Ile Agbon
A le sopọ Hiveserver2 nipasẹ ọna ila ti o jẹ alabara tinrin ati laini aṣẹ. O nlo awakọ JDBC lati fi idi asopọ mulẹ.
17. Buwolu wọle si olupin nibiti Hive Gateway n ṣiṣẹ.
[[email ~]$ beeline
18. Tẹ okun asopọ JDBC sii lati so Hiveserver2 pọ. Ni asopọ yii, okun ti a n mẹnuba Hiverserver2 (master2) pẹlu nọmba ibudo aiyipada rẹ 10000. Okun asopọ yii yoo sopọ nikan si Hiveserver2 eyiti o nṣiṣẹ lori master2.
beeline> !connect "jdbc:hive2://master1.linux-console.net:10000"
19. Ṣiṣe ibeere ayẹwo kan.
0: jdbc:hive2://master1.linux-console.net:10000> show databases;
Eyi ni ipilẹ data aiyipada ti o wa ni inbuilt.
20. Lo pipaṣẹ ti o wa ni isalẹ lati fopin si akoko Ile Agbon.
0: jdbc:hive2://master1.linux-console.net:10000> !quit
21. O le lo ọna kanna lati sopọ Hiveserver2 ti n ṣiṣẹ lori master2.
beeline> !connect "jdbc:hive2://master2.linux-console.net:10000"
23. A le sopọ Hiveserver2 ni ipo Awari Zookeeper. Ni ọna yii, a ko nilo lati darukọ Hiveserver2 ninu okun asopọ dipo a nlo Zookeeper lati ṣe iwari Hiveserver2 ti o wa.
Nibi a le lo iwọntunwọnsi fifuye ẹnikẹta lati ṣe iwọnwọn ẹru laarin Hiverserver2 to wa. Iṣeto ni isalẹ nilo lati mu Ipo Awari Zookeeper ṣiṣẹ nipa lilọ si Oluṣakoso Cloudera -> Ile Agbon -> Iṣeto ni.
24. Itele, wa ohun-ini\"Snippet iṣeto-ilọsiwaju HiveServer2" ki o tẹ aami +
lati ṣafikun ohun-ini isalẹ.
Name : hive.server2.support.dynamic.service.discovery Value : true Description : <any description>
25. Lọgan ti o tẹ ohun-ini naa, tẹ 'Fipamọ Awọn ayipada'.
26. Bii a ṣe awọn ayipada si Iṣeto ni, nilo lati tun bẹrẹ awọn iṣẹ ti o kan nipa tite lori Aami Awọ Osan lati tun bẹrẹ awọn iṣẹ naa.
27. Tẹ awọn iṣẹ ‘Tun bẹrẹ Stale’.
28. Awọn aṣayan meji wa. Ti iṣupọ naa ba wa ni iṣelọpọ laaye, a nilo lati fẹran yiyi bẹrẹ lati dinku iṣẹku. Bi a ṣe n fi sii tuntun, a le yan aṣayan keji ‘Tun-fi ranṣẹ Iṣeto Onibara’, ki o tẹ ‘Tun bẹrẹ Bayi’.
29. Lọgan ti atunbere ti pari ni aṣeyọri, iwọ yoo gba ipo ‘Ti pari’. Tẹ 'Pari' lati pari ilana naa.
30. Bayi a yoo sopọ Hiveserver2 ni lilo Ipo Awari Zookeeper. Ninu asopọ JDBC, okun ti a nilo lati lo awọn olupin Zookeeper pẹlu nọmba ibudo rẹ 2081. Gba awọn olupin Zookeeper nipasẹ lilọ si Oluṣakoso Cloudera -> Zookeeper -> Awọn apeere -> (Akiyesi awọn orukọ olupin).
Iwọnyi ni awọn olupin mẹta ti o ni Zookeeper, 2181 jẹ nọmba ibudo naa.
master1.linux-console.net:2181 master2.linux-console.net:2181 worker1.linux-console.net:2181
31. Bayi wa sinu beeline.
[[email ~]$ beeline
32. Tẹ okun asopọ JDBC sii bi darukọ ni isalẹ. A ni lati mẹnuba Ipo Awari Iṣẹ ati Orukọ Orukọ Zookeeper. 'Hiveserver2' ni aaye orukọ aiyipada ti Hiveserver2.
beeline>!connect "jdbc:hive2://master1.linux-console.net:2181,master2.linux-console.net:2181,worker1.linux-console.net:2181/;serviceDiscoveryMode=zookeeper;zookeeperNamespace=hiveserver2"
33. Bayi igba naa ti sopọ si Hiveserver2 ti n ṣiṣẹ lori master1. Ṣiṣe ibeere apẹẹrẹ lati jẹrisi. Lo pipaṣẹ ti o wa ni isalẹ lati ṣẹda ibi ipamọ data kan.
0: jdbc:hive2://master1.linux-console.net:2181,mast> create database tecmint;
34. Lo pipaṣẹ ti o wa ni isalẹ lati ṣe atokọ ibi ipamọ data.
0: jdbc:hive2://master1.linux-console.net:2181,mast> show databases;
35. Bayi a yoo jẹrisi Wiwa to gaju ni Ipo Awari Zookeeper. Lọ si Oluṣakoso Cloudera ki o da Hiveserver2 duro lori master1 ti a ti danwo loke.
Oluṣakoso Cloudera -> Ile Agbon -> Awọn apeere -> (yan Hiveserver2 lori master1) -> Iṣe fun yiyan -> Duro.
36. Tẹ 'Duro' naa. Lọgan ti o duro, iwọ yoo gba ipo ‘Ti pari’. Ṣayẹwo Hiveserver2 lori master1 nipa lilọ kiri si Ile Agbon -> Awọn apeere.
37. Wọle pẹpẹ ki o so Hiveserver2 pọ nipa lilo okun asopọ asopọ JDBC kanna pẹlu Ipo Awari Zookeeper bi a ti ṣe ni awọn igbesẹ ti o wa loke.
[[email ~]$ beeline beeline>!connect "jdbc:hive2://master1.linux-console.net:2181,master2.linux-console.net:2181,worker1.linux-console.net:2181/;serviceDiscoveryMode=zookeeper;zookeeperNamespace=hiveserver2"
Bayi o yoo ni asopọ si Hiveserver2 ti n ṣiṣẹ lori master2.
38. Ṣafẹri pẹlu ibeere ayẹwo.
0: jdbc:hive2://master1.linux-console.net:2181,mast> show databases;
Ninu àpilẹkọ yii, a ti kọja nipasẹ awọn igbesẹ alaye lati ni awoṣe Ile ipamọ data Ile Agbon ni Iṣupọ wa pẹlu Wiwa to gaju. Ni agbegbe iṣelọpọ gidi-akoko, diẹ sii ju Hiveserver2 mẹta ni yoo gbe pẹlu Ipo Awari Zookeeper ṣiṣẹ.
Nibi, gbogbo awọn Hiveserver2's n forukọsilẹ pẹlu Zookeeper labẹ Orukọ Orukọ wọpọ. Zookeeper Dynamically ṣe iwari Hiveserver2 ti o wa ati ṣeto igba Ile Agbon.