安装Hive

  |   0 评论   |   1,688 浏览

解压Hive压缩包

[root@namenode ~]# tar -zxvf /soft/hive-0.13.1-cdh5.3.6.tar.gz -C /usr/local/

拷贝MySQL驱动至HiveClassPath目录下

[root@namenode ~]# cp /soft/mysql-connector-java-5.1.40.jar /usr/local/hive-0.13.1-cdh5.3.6/lib/

修改配置文件hive-site.xml

[root@namenode ~]# vim /usr/local/hive-0.13.1-cdh5.3.6/conf/
hive-site.xml


<configuration>
        <property>
                <name>javax.jdo.option.ConnectionURL</name>
                <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true&amp;autoReconnect=true&amp;characterEncoding=UTF-8</value>
        </property>
        <property>
                <name>javax.jdo.option.ConnectionDriverName</name>
                <value>com.mysql.jdbc.Driver</value>
        </property>
        <property>
                <name>javax.jdo.option.ConnectionUserName</name>
                <value>root</value>
        </property>
        <property>
                <name>javax.jdo.option.ConnectionPassword</name>
                <value>root</value>
        </property>
        <property>
                <name>datanucleus.autoCreateSchema</name>
                <value>true</value>
        </property>
        <property>
                <name>datanucleus.fixedDatastore</name>
                <value>false</value>
        </property>
        <property>
                <name>hive.metastore.uris</name>
                <value>thrift://namenode:9083</value>
        </property>
 
</configuration>

修改环境变量并加载

vim /etc/profile

添加

export HIVE_HOME=/usr/local/hive-0.13.1-cdh5.3.6
export PATH=$PATH:$HIVE_HOME/bin

加载

[root@namenode ~]# source /etc/profile

 

启动metastore

[root@namenode ~]# nohup hive --service metastore &

[1] 30512

[root@namenode ~]# Starting Hive Metastore Server

验证metastore服务

[root@namenode ~]# netstat -antp|grep 9083

tcp        0      0 0.0.0.0:9083                0.0.0.0:*                   LISTEN      30512/java

Hive测试表所需的数据上传至HDFS当中

[root@namenode ~]# hadoop fs -mkdir /customer

创建测试表文件test.txt

vim /sample/test.txt

1       a

2       b

3       c

4       d

[root@namenode ~]# hadoop fs -put 
/sample/test.txt
 /customer

启动hive命令行并验证

[root@namenode ~]# hive

创建test数据库

Hive> create database test;
hive> use test;
OK
Time taken: 0.489 seconds
hive> CREATE EXTERNAL TABLE `customer`(
    > `id` int,
    > `name` string)
    > ROW FORMAT DELIMITED
    > FIELDS TERMINATED BY '\t'
    > LINES TERMINATED BY '\n'
    > LOCATION
    > 'hdfs://namenode:9000/customer';
OK
Time taken: 0.275 seconds
hive> select * from customer;
OK
1       a
2       b
3       c
4       d
Time taken: 0.417 seconds, Fetched: 4 row(s)
hive> select count(*) from customer;
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapreduce.job.reduces=<number>
Starting Job = job_1481942242964_0002, Tracking URL = http://namenode:8041/proxy/application_1481942242964_0002/
Kill Command = /usr/local/hadoop-2.5.0-cdh5.3.6/bin/hadoop job  -kill job_1481942242964_0002
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2016-12-17 11:22:49,108 Stage-1 map = 0%,  reduce = 0%
2016-12-17 11:22:55,392 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.18 sec
2016-12-17 11:23:05,822 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 3.29 sec
MapReduce Total cumulative CPU time: 3 seconds 290 msec
Ended Job = job_1481942242964_0002
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1  Reduce: 1   Cumulative CPU: 3.29 sec   HDFS Read: 205 HDFS Write: 2 SUCCESS
Total MapReduce CPU Time Spent: 3 seconds 290 msec
OK
4
Time taken: 28.174 seconds, Fetched: 1 row(s)


读后有收获可以支付宝请作者喝咖啡