Are you prepared in attending an interview? Then do not worry. If you are preparing for Hadoop Administration job interview and don’t know how to crack interview and what level or difficulty of questions to be asked in job interviews then go through Wisdomjobs Hadoop Administration interview questions and answers page to crack your job interview. A Hadoop Administrator is one who administers and manages hadoop clusters and all other resources in the entire Hadoop ecosystem. The role of a Hadoop Admin is mainly associated with tasks that involve installing and monitoring hadoop clusters. He is responsible for having those clusters safe. Below is the list of frequently asked Hadoop Administration interview questions and answers which gets you ready to face the interviews:
Fair Scheduling is the process in which resources are assigned to jobs such that all jobs get to share equal number of resources over time.
Fair Scheduler can be used under the following circumstances:
i) If you wants the jobs to make equal progress instead of following the FIFO order then you must use Fair Scheduling.
ii) If you have slow connectivity and data locality plays a vital role and makes a significant difference to the job runtime then you must use Fair Scheduling.
iii) Use fair scheduling if there is lot of variability in the utilization between pools.
Capacity Scheduler allows runs the hadoop mapreduce cluster as a shared, multi-tenant cluster to maximize the utilization of the hadoop cluster and throughput.
Capacity Scheduler can be used under the following circumstances:
i) If the jobs require scheduler detrminism then Capacity Scheduler can be useful.
ii) CS's memory based scheduling method is useful if the jobs have varying memory requirements.
iii) If you want to enforce resource allocation because you know very well about the cluster utilization and workload then use Capacity Scheduler.
NameNode, DataNode, TaskTracker and JobTracker
The easiest way of doing this is to run the command to stop running shell script i.e. click on stop-all.sh. Once this is done, restarts the NameNode by clicking on start-all.sh.
FIFO Scheduler – This scheduler does not consider the heterogeneity in the system but orders the jobs based on their arrival times in a queue.
COSHH- This scheduler considers the workload, cluster and the user heterogeneity for scheduling decisions.
Fair Sharing-This Hadoop scheduler defines a pool for each user. The pool contains a number of map and reduce slots on a resource. Each user can use their own pool to execute the jobs.
jps command is used to verify whether the daemons that run the Hadoop cluster are working or not. The output of jps command shows the status of the NameNode, Secondary NameNode, DataNode, TaskTracker and JobTracker.
Memory-System’s memory requirements will vary between the worker services and management services based on the application.
Operating System - a 64-bit operating system avoids any restrictions to be imposed on the amount of memory that can be used on worker nodes.
Storage- It is preferable to design a Hadoop platform by moving the compute activity to data to achieve scalability and high performance.
Capacity- Large Form Factor (3.5”) disks cost less and allow to store more, when compared to Small Form Factor disks.
Network - Two TOR switches per rack provide better redundancy.
Computational Capacity- This can be determined by the total number of MapReduce slots available across all the nodes within a Hadoop cluster.
The file system goes offline whenever the NameNode is down.
This file provides an environment for Hadoop to run and consists of the following variables-HADOOP_CLASSPATH, JAVA_HOME and HADOOP_LOG_DIR. JAVA_HOME variable should be set for Hadoop to run.
Use the command -/etc/init.d/hadoop-0.20-namenode status.
Question 12. In A Mapreduce System, If The Hdfs Block Size Is 64 Mb And There Are 3 Files Of Size 127mb, 64k And 65mb With Fileinputformat. Under This Scenario, How Many Input Splits Are Likely To Be Made By The Hadoop Framework.?
2 splits each for 127 MB and 65 MB files and 1 split for the 64KB file.
Hadoop FSCK (File System Check) command is used to check missing blocks.
Text Mining, Graph Analysis, Semantic Analysis, Sentiment Analysis, Recommendation Systems.
Hadoop job –kill jobID
Using the command – Hadoop job –list, gives the list of jobs running in a Hadoop cluster.
Yes, it is possible to copy files across multiple Hadoop clusters and this can be achieved using distributed copy. DistCP command is used for intra or inter cluster copying.
Ubuntu or Linux is the most preferred operating system to run Hadoop. Though Windows OS can also be used to run Hadoop but it will lead to several problems and is not recommended.
Question 20. The Mapred.output.compress Property Is Set To True, To Make Sure That All Output Files Are Compressed For Efficient Space Usage On The Hadoop Cluster. In Case Under A Particular Condition If A Cluster User Does Not Require Compressed Data For A Job. What Would You Suggest That He Do?
If the user does not want to compress the data for a particular job then he should create his own configuration file and set the mapred.output.compress property to false. This configuration file then should be loaded as a resource into the job.
It is always better to deploy a secondary NameNode on a separate standalone machine. When the secondary NameNode is deployed on a separate machine it does not interfere with the operations of the primary node.
The NameNode should never be reformatted. Doing so will result in complete data loss. NameNode is formatted only once at the beginning after which it creates the directory structure for file system metadata and namespace ID for the entire file system.
The task will be started again on a new TaskTracker and if it fails more than 4 times which is the default setting (the default value can be changed), the job will be killed.
Nothing could have actually wrong, if there is huge volume of data because data replication usually takes times based on data size as the cluster has to copy the data and it might take a few hours.
The configuration files are located in “conf” sub directory. Hadoop has 3 different Configuration files- hdfs-site.xml, core-site.xml and mapred-site.xml.
Hadoop Administration Related Tutorials
|Informatica Tutorial||Teradata Tutorial|
|Hadoop Tutorial||Java Tutorial|
|Hadoop MapReduce Tutorial||Apache Pig Tutorial|
|HBase Tutorial||MongoDB Tutorial|
Hadoop Administration Related Interview Questions
|Informatica Interview Questions||Teradata Interview Questions|
|Hadoop Interview Questions||Java Interview Questions|
|Hadoop MapReduce Interview Questions||Apache Pig Interview Questions|
|Machine learning Interview Questions||NoSQL Interview Questions|
|HBase Interview Questions||MongoDB Interview Questions|
|Data Science R Interview Questions|
The Hadoop Distributed Filesystem
Developing A Mapreduce Application
How Mapreduce Works
Setting Up A Hadoop Cluster
All rights reserved © 2020 Wisdom IT Services India Pvt. Ltd
Wisdomjobs.com is one of the best job search sites in India.