Apache Sqoop Codegen Command With Example | Praveen Deshmane

posted on Nov 20th, 2016

Apache Sqoop

Apache Sqoop is a command-line interface application for transferring data between relational databases and Hadoop. It supports incremental loads of a single table or a free form SQL query as well as saved jobs which can be run multiple times to import updates made to a database since the last import. Imports can also be used to populate tables in Hive or HBase. Exports can be used to put data from Hadoop into a relational database. Sqoop got the name from sql+hadoop. Sqoop became a top-level Apache project in March 2012.

Pre Requirements

1) A machine with Ubuntu 14.04 LTS operating system.

2) Apache Hadoop pre installed (How to install Hadoop on Ubuntu 14.04)

3) MySQL Database pre installed (How to install MySQL Database on Ubuntu 14.04)

4) Apache Sqoop pre installed (How to install Apache Sqoop on Ubuntu 14.04)

Sqoop Codegen Command With Example

This post describes the importance of 'codegen' tool. From the viewpoint of object-oriented application, every database table has one DAO class that contains 'getter' and 'setter' methods

It generates DAO class in Java, based on the Table Schema structure. The Java definition is instantiated as a part of the import process. The main usage of this tool is to check if Java lost the Java code. If so, it will create a new version of Java with the default delimiter between fields.

Step 1 - Change the directory to /usr/local/hadoop/sbin

$ cd /usr/local/hadoop/sbin

Step 2 - Start all hadoop daemons.

$ start-all.sh

Step 3 - The JPS (Java Virtual Machine Process Status Tool) tool is limited to reporting information on JVMs for which it has the access permissions.

$ jps

Step 4 - Change the directory to /usr/local/sqoop/bin

$ cd /usr/local/sqoop/bin

CODEGEN (See, How to use Sqoop Import command)

$ sqoop codegen \
--connect jdbc:mysql://localhost/userdb \
--username root \
--password root \
--table employee

Apache Sqoop Codegen Command With Example

If the command executes successfully, then it will produce the following output on the terminal.

$ /tmp/sqoop-hduser/compile/0607f608c40d5860029a2132208e5912

Verification

Let us take a look at the output. The path, which is in bold, is the location that the Java code of the emp table generates and stores. Let us verify the files in that location using the following commands.

$ cd /tmp/sqoop-hduser/compile/0607f608c40d5860029a2132208e5912

$ ls

Apache Sqoop Codegen Command With Example

Please share this blog post and follow me for latest updates on

facebook             google+             twitter             feedburner

Previous Post                                                                                          Next Post

Labels : Apache Sqoop Installation on Ubuntu   Apache Sqoop Import Command Example   Apache Sqoop Export Command Example   Apache Sqoop Job Command Example   Apache Sqoop Eval Command Example   Apache Sqoop List-tables Command Example   Apache Sqoop List-tables Command Example