In this article we have described the step-by-step process of installing Cloudera Manager in accordance with industry practice. In the second part, we have already met the preliminary requirements for Cloudera, so that all servers are fully prepared.
Here we will have 5 node groups with 2 assistants and 3 staff members. I used 5 copies of the AWS EC2 to demonstrate the installation procedure I called these 5 servers, as shown below.
master2.tecmint.com
workers1.tecmint.com
workers2.tecmint.com
workers3.tecmint.com
Cloudera Manager is a management and monitoring tool for the entire HRC. We, as administrators, generally call it the Cloudera Hadoop management tool. This tool allows us to introduce, monitor, check and modify the configuration. It is very important to manage the entire cluster.
The important functions of the Cloudera Manager are listed below.
- Automatic deployment and configuration of Hadoop clusters.
- Cluster Health Monitor
- Setting alarms
- Troubleshooting
- Reports
- Writing a report on the use of clusters
- Dynamic adjustment of resources
Step 1: Installation of the Apache web server on CentOS
We will use master1 as webserver for the cloudera repositories. In addition, Cloudera Manager is a web user interface, so we need to install Apache. Follow these steps to install the Apache web server.
# yum -y install httpd
After installing httpd, start it up and turn it on to work at startup.
# systemctl start httpd
# systemctl enable httpd
Check the status after starting httpd.
# systemctl status httpd
Status of the Apache
After running httpd, opening a browser on your local system and entering the IP address of master1 in the search field, you should receive this test page to make sure httpd works properly.
Check the Apache website
Step 2: Setting up local DNS to resolve IP address and host name
We need a DNS server or we need to configure /etc/hosts to resolve the IP and hostname. Here we configure /etc/hosts, but in real time for the production environment there will be a dedicated DNS server.
Follow these steps to create a listing for all your servers in /etc/hosts.
# vi /etc/hosts
It must be configured on all servers.
13 235 27 144 master1.tecmint.com master1
13 235 135 170 master2.tecmint.com master2
15 206 167 94 worker1.tecmint.com worker1
13 232 173 158 worker2.tecmint.com worker2
65 0 182 222 worker3.tecmint.com worker3.
Local DNS configuration
Step 3: Login settings without SSH password
In this demonstration the Cloudera manager is installed on master1. You must configure ssh without password from master1 to all other nodes. Because Cloudera Manager uses ssh to connect all other nodes to install packages.
Follow these steps to set up a passwordless ssh from master1 to all other servers. We have the tecmint user for further work.
Create a custom tecmint of the 4 servers using the useradd command, as shown in the figure.
# useradd -m tecmint
To give the tecmint user the privilege of root, add the following line in /etc/sudoers You can add this line under root, as shown in the screenshot.
tecmint ALL=(ALL) ALL
Adding a user to Sudo
Go to the tecmint user site and create the ssh key in master1 using the command below.
# sudo su tecmint
$ ssh-keygen
Now copy the key created with the ssh-copy-id command to the 4 servers as shown in the figure.
ssh-copy-id -i ~/.ssh/id_rsa.pub [protected by email]$ ssh-copy-id -i ~/.ssh/id_rsa.pub [protected by email]$ ssh-copy-id -i ~/.ssh/id_rsa.pub [protected by email]$ ssh-copy-id -i ~/.ssh/id_rsa.pub [protected by email]$ ssh-copy-id -i ~/.ssh/id_rsa.pub [protected via email]$ ssh-copy-id -i ~/.ssh/id_rsa.pub [protected via email]$ ssh-copy-id -i [protected via email].
You should now be able to switch from master1 to the other servers without a password, as shown in the picture.
ssh-Master2
sshwerker1
sshwerker2
sshwerker3
SSH Login Servers without password
Step 4: Installation and configuration of Cloudera Manager
We can use the vendor repository (Cloudera) to install all packages using the package management tools in RHEL/CentOS. Creating our own real-time repository is a best practice because we may not have internet access on the production servers.
Here we will install version 6.3.1 of the Cloudera manager. Since we will use master1 as a repo server, we will download the packages in the following path.
Create the following folders on your Master1 server.
$ sudo mkdir -p /var/www/html/cloudera-repos/cm6
We can use the wget tool to download packages via http. So install wget with the following command.
$ ship’s yum – wget installation
Then download the tar file from the Cloudera Manager with the following wget command.
wget https://archive.cloudera.com/cm6/6.3.1/repo-as-tarball/cm6.3.1-redhat7.tar.gz
Unzip the tar file in /var/www/html/cloudera-repos/cm6, we already created master1 as webserver by installing http and testing it in the browser.
$ sudo tar xvfz cm6.3.1-redhat7.tar.gz -C /var/www/html/cloudera-repos/cm6 –strip components=1
Now make sure all Cloudera rpm files are in /var/www/html/cloudera-repos/cm6/RPMS/x86_64.
cd /var/www/html/cloudera-repos/cm6
$ ll
List of Cloud Files
Create files /etc/yum.repos.d/cloudera-manager.repo on all the cluster’s host servers with the following content, where master1 (65.0.101.148) is the web server
name=cloudera-manager
baseurl=http:///cloudera-repos/cm6/
enabled=1
gpgcheck=0.
Now that a repository has been added, run the command below to see which repositories have been activated.
Actor
Cloud list directory
Run the following command to display all available cloud related packets in the repository.
The $ brummel list is available at | grep cloudera*.
List of Cloud Packages
Install the cloud manager server, cloud manager agent, cloud managerdb-2.
$ sudo yum installation cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server cloudera-manager-db-2
Installation of a cloud in CentOS
To view all installed Cloudera packages, execute the following command.
The $ yum list is set |grep cloudera*.
List of installed Cloud Service Packages
Run the following command to start Cloudera Server-BD, the basic database for storing Cloudera Manager and metadata from other services.
Cloudera comes standard with postgre-sql, which is integrated in the Cloudera manager. We install an integrated external database that can be used in real time. This can be Oracle, MySQL or PostgreSQL.
$ sudo systemctl start cloudera-scm-server-db
To check the status of the database, execute the following command.
$ sudo systemctl status cloudera-scm-server-db
Check the status of the cloud database
Configuration of db.properties for the Cloudera Manager server
$ vi /etc/cloudera-scm-server/db.properties
Set the value under EMBEDDED to force the Cloud Manager to use the integrated database.
com.cloudera.cmf.db.setupType=EMBEDDED
Configuring the cloud database properties
Run the following command to start the Cloudera Manager server.
$ sudo systemctl start cloudera-scm server
Run the following command to check the status of the Cloudera Manager server.
$ state sudo systemctl cloudera-scm server
Check the status of the cloud manager
To start and check the status of the Cloudera Manager agent, execute the following command.
sudo systemctl start cloudera-scm agent
$ sudo systemctl status cloudera-scm agent
Checking the agent status of the storage manager
Once the server of Cloudera Manager has been successfully launched, you can view the web user interface (login page) in your browser using the IP address and port number 7180, which is the port number of Cloudera Manager.
https://65.0.101.148:7180
Cloud Manager Login
Summary
In this article we have followed step by step the process of installing Cloudera Manager on CentOS 7. We will see the HRC and other services in the next article.
Related Tags:
cloudera cdh 6 installation,etc/cloudera-scm-server/db properties,repo cloudera,cloudera express,cloudera cluster installation step by step,cloudera manager server configuration file,download cdh,cloudera installation,cloudera ipv6,automated installation by cloudera manager,cloudera 5.16 installation,download cloudera repo file,cloudera manager download,usr/share/cmf/schema scm_prepare_database sh,install cloudera manager on centos 7,cloudera manager installation,cloudera quickstart vm,install cdh using cloudera manager,cloudera installation steps,install cdh on ubuntu
