安装Sqoop2

  |   0 评论   |   1,506 浏览

解压sqoop2压缩包至/usr/local目录

[root@namenode ~]# tar -zxvf /soft/sqoop2-1.99.4-cdh5.3.6.tar.gz -C /usr/local/

修改环境变量

vim /etc/profile

export SQOOP_HOME=/usr/local/sqoop2-1.99.4-cdh5.3.6

export PATH=$PATH:$SQOOP_HOME/bin:$SQOOP_HOME/server/bin

source /etc/profile

修改ClassPath

[root@namenode ~]# vim /usr/local/sqoop2-1.99.4-cdh5.3.6/server/conf/catalina.properties

修改

common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,${catalina.home}/../lib/*.jar,/usr/local/hadoop-2.5.0-cdh5.3.6/share/hadoop/common/*.jar,/usr/local/hadoop-2.5.0-cdh5.3.6/share/hadoop/common/lib/*.jar,/usr/local/hadoop-2.5.0-cdh5.3.6/share/hadoop/hdfs/*.jar,/usr/local/hadoop-2.5.0-cdh5.3.6/share/hadoop/hdfs/lib/*.jar,/usr/local/hadoop-2.5.0-cdh5.3.6/share/hadoop/mapreduce/*.jar,/usr/local/hadoop-2.5.0-cdh5.3.6/share/hadoop/mapreduce/lib/*.jar,/usr/local/hadoop-2.5.0-cdh5.3.6/share/hadoop/yarn/*.jar,/usr/local/hadoop-2.5.0-cdh5.3.6/share/hadoop/yarn/lib/*.jar,/usr/local/hadoop-2.5.0-cdh5.3.6/lib/*.jar

添加Hadoop配置文件

[root@namenode ~]# vim /usr/local/sqoop2-1.99.4-cdh5.3.6/server/conf/sqoop.properties

替换如下配置

org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/hadoop-2.5.0-cdh5.3.6/etc/hadoop/

拷贝MySQL驱动

创建lib目录:

mkdir –p /usr/local/sqoop2-1.99.4-cdh5.3.6/lib/

[root@namenode ~]# cp /soft/mysql-connector-java-5.1.40.jar /usr/local/sqoop2-1.99.4-cdh5.3.6/lib/

启动SqoopServer

添加可执行权限:

chmod +x /usr/local/sqoop2-1.99.4-cdh5.3.6/*.sh

chmod +x /usr/local/sqoop2-1.99.4-cdh5.3.6/server/*.sh

[root@namenode ~]# sqoop.sh server start

Sqoop home directory: /usr/local/sqoop2-1.99.4-cdh5.3.6

Setting SQOOP_HTTP_PORT: 12000

Setting SQOOP_ADMIN_PORT: 12001

Using CATALINA_OPTS:

Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001

Using CATALINA_BASE: /usr/local/sqoop2-1.99.4-cdh5.3.6/server

Using CATALINA_HOME: /usr/local/sqoop2-1.99.4-cdh5.3.6/server

Using CATALINA_TMPDIR: /usr/local/sqoop2-1.99.4-cdh5.3.6/server/temp

Using JRE_HOME: /usr/local/jdk1.7.0_79/

Using CLASSPATH: /usr/local/sqoop2-1.99.4-cdh5.3.6/server/bin/bootstrap.jar

[root@namenode ~]# tail -f /usr/local/sqoop2-1.99.4-cdh5.3.6/server/logs/catalina.out

log4j: setFile ended

log4j: Parsed “defaultAppender” options.

log4j: Handling log4j.additivity.default=[null]

log4j: Finished configuring.

Dec 18, 2016 10:07:39 AM org.apache.catalina.startup.HostConfig deployDirectory

INFO: Deploying web application directory ROOT

Dec 18, 2016 10:07:39 AM org.apache.coyote.http11.Http11Protocol start

INFO: Starting Coyote HTTP/1.1 on http-12000

Dec 18, 2016 10:07:39 AM org.apache.catalina.startup.Catalina start

INFO: Server startup in 11067 ms

查看监听端口

[root@namenode ~]# netstat -antp|grep 12001

tcp 0 0 127.0.0.1:12001 0.0.0.0:* LISTEN 18861/java

[root@namenode ~]# netstat -antp|grep 12000

tcp 0 0 0.0.0.0:12000 0.0.0.0:* LISTEN 18861/java

拷贝MySQL驱动至各个节点的Hadoop的classpath目录下

[root@namenode sqoop2-1.99.4-cdh5.3.6]# scp /soft/mysql-connector-java-5.1.40.jar datanode1:/usr/local/hadoop-2.5.0-cdh5.3.6/share/hadoop/common/lib/

mysql-connector-java-5.1.40.jar 100% 968KB 967.7KB/s 00:00

[root@namenode sqoop2-1.99.4-cdh5.3.6]# scp /soft/mysql-connector-java-5.1.40.jar datanode2:/usr/local/hadoop-2.5.0-cdh5.3.6/share/hadoop/common/lib/

mysql-connector-java-5.1.40.jar 100% 968KB 967.7KB/s 00:00

[root@namenode sqoop2-1.99.4-cdh5.3.6]# scp /soft/mysql-connector-java-5.1.40.jar namenode:/usr/local/hadoop-2.5.0-cdh5.3.6/share/hadoop/common/lib/

mysql-connector-java-5.1.40.jar

读后有收获可以支付宝请作者喝咖啡