Friday, October 3, 2014

Hive Setup on Debian

2014/10/04 Saturday 11:45

A fine day , but nothing to do, no plans to go out.
So thought of uploading the manual for hive setup.

Having manual handy is quite helpful, when you have any idea to implement, but you are stuck because of environment. It takes time to search for setup manuals, then installing the stuff.
And when you find its finished, you get some issues of xxx not installed, yyy not present.
And then there starts the series of problems wherein you either give up your idea, or get so much involved in setting up things that you almost forget what was the main thing with which you started all this. There are few highly motivated people who do the setup and then implement their idea.

Well lets see how to install hive right from the scratch.

1) Install Virtual Box

I prefer to install on virtual box, because it is quite useful to understand how to setup from beginning, right from empty OS.

So first download the virtual box depending on your OS.
https://www.virtualbox.org/wiki/Downloads

2) Install Debian on Virtual Box

Download Debian ISO image
http://ftp.acc.umu.se/debian-cd/7.6.0/

Mount this Debian ISO image in your virtual box and install Debian OS on virtual box.
After installing, start your OS.

If you want, you can share the network of Virtual Box and your local PC. Also install SSH, so that you can connect from your local PC to your virtual debian os via any terminal.

3) Setting up JAVA

Download Java JDK
http://www.oracle.com/technetwork/java/javase/downloads/index.html

-----
## check OS version
file /sbin/init

## remove all old versions of java
apt-get purge openjdk-\*


## download the corresponding java 
cd /home/xxx/
wget "java download URL"

## extract the contents
## replace this file with yours
tar -xzvf "jdk-7u65-linux-x64.tar.gz" 

ls -la jdk-7u65-linux-x64

## move it to common location
mkdir -pv /usr/local/java/
mv  jdk-7u65-linux-x64  /usr/local/java/

## setup env variables
vi /etc/profile
----
JAVA_HOME=/usr/local/java/jdk1.7.0_65
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin

JRE_HOME=/usr/local/java/jdk1.7.0_65
PATH=$PATH:$HOME/bin:$JRE_HOME/bin

export JAVA_HOME
export JRE_HOME
export PATH
----

## import the env variables
source /etc/profile

## update the symbolic links for java
update-alternatives --install "/usr/bin/java" "java" "/usr/local/java/jdk1.7.0_65/bin/java" 1
update-alternatives --install "/usr/bin/javac" "javac" "/usr/local/java/jdk1.7.0_65/bin/javac" 1
update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/local/java/jdk1.7.0_65/bin/javaws" 1

update-alternatives --set java /usr/local/java/jdk1.7.0_65/bin/java
update-alternatives --set javac /usr/local/java/jdk1.7.0_65/bin/javac
update-alternatives --set javaws /usr/local/java/jdk1.7.0_65/bin/javaws

## confirm your java version
java -version


4) Setting up Hadoop (pre-requirement for Hive)


http://www.apache.org/dyn/closer.cgi/hadoop/common/

cd /home/xxx/
wget "hadoop download URL"

## extract the contents
## replace this file with yours
tar -xzvf "hadoop-2.5.1.tar.gz" 

ls -la hadoop-2.5.1

## move it to common location
mkdir -pv /usr/local/hadoop/
mv  hadoop-2.5.1  /usr/local/hadoop/

## setup env variables
vi /etc/profile
----
HADOOP_PREFIX=/usr/local/hadoop/hadoop-2.5.1
PATH=$PATH:$HOME/bin:$HADOOP_PREFIX/bin

export HADOOP_PREFIX
export PATH
----

## confirm your hadoop version
hadoop version


5) Setting up Hive

http://www.apache.org/dyn/closer.cgi/hive/

cd /home/xxx/
wget "hive download URL"

## extract the contents
## replace this file with yours
tar -xzvf "apache-hive-0.13.1-bin.tar.gz" 

ls -la apache-hive-0.13.1-bin

## move it to common location
mkdir -pv /usr/local/hive/
mv  apache-hive-0.13.1-bin  /usr/local/hive/

## setup env variables
vi /etc/profile
----
HIVE_HOME=/usr/local/hive/apache-hive-0.13.1-bin
PATH=$PATH:$HOME/bin:$HIVE_HOME/bin

export HIVE_HOME
export PATH
----

## confirm your hive version
hive


And there you go, your basic setup is complete.
Now implement anything.
For reference of commands, I think this Hive confluence is quite useful.
https://cwiki.apache.org/confluence/display/Hive/GettingStarted

2014/10/04 Saturday 1:11