How to validate the missing records in Hive after sqooping the data from Oracle

Wanted to know how to check missing records in Hive when data is loaded from Oracle due to some issue.

Suppose:

Oracle no.of records : 2000
Hive no.of records : 1990

How to check the 10 missing records in Hive.

Copyright Notice：Content Author:「Gudala Sandeep」，Reproduced under the CC 4.0 BY-SA copyright license with a link to the original source and this disclaimer.
Link to original article：https://stackoverflow.com/questions/51027311/how-to-validate-the-missing-records-in-hive-after-sqooping-the-data-from-oracle

Answers

Harold 2018-06-26T07:15:45

Export from Oracle to a CSV file with a sort by primary Key. Same with HIVE. And a small UNIX diff. Bottom line, Sqoop cannot miss records, so there's probably something wrong in your integration.",

Products recommended by Amazon

More >>>

SanDisk 128GB microSDXC-Card, Licensed for Nintendo-Switch - SDSQXAO-128G-GNCZN

HUANUO Dual Monitor Stand - Adjustable Spring Monitor Desk Mount Swivel Vesa Bracket with C Clamp, Grommet Mounting Base for 13 to 27 Inch Computer Screens - Each Arm Holds 4.4 to 14.3lbs

AMD Ryzen 5 5600X 6-core, 12-Thread Unlocked Desktop Processor with Wraith Stealth Cooler

Timberland Men's 6-Inch Premium Waterproof Boot

More about “How to validate the missing records in Hive after sqooping the data from Oracle” related questions

How to validate the missing records in Hive after sqooping the data from Oracle

Wanted to know how to check missing records in Hive when data is loaded from Oracle due to some issue. Suppose: Oracle no.of records : 2000 Hive no.of records : 1990 How to check the 10 missing

Show Detail

Special characters are not proper after sqooping data into Hive from teradata

I'm trying to sqoop the teradata table into Hive using below "sqoop-import" command. sqoop tdimport -Dtdch.output.hdfs.avro.schema.file=/tmp/data/country.avsc --connect jdbc:teradata://tdserver/

Show Detail

Sqoooing data to HBase and integrated with Hive is having performance impact

We are sqooping the oracle tables to HBase and created mapping tables in Hive, and it seems while we are sqooping, all the data is going to one region, and hence when I query on hive only one map t...

Show Detail

Sqooping same table for different schema in parallal is failing

We are having different data base schemas in Oracle. We are planning to sqoop some of the tables from oracle to Hive ware house. But If we put sqooping of tables of an oltp is sequential it is work...

Show Detail

Sqooping from oracle to hive is failing with an error

I tried to sqoop a table from oracle to Hive, But when I issue a sqoop command it is able to connect the db and fetch the reocds, but while it is getting the meta data for creating a table in Hive ...

Show Detail

How to copy data from an Hive table to a HBase table which is already loaded with some data

We have an Hive table which is partitioned on date (yyyy-MM-dd), and this table is daily loaded with new data for latest date partition. And also this table has last 2 years of data(partitioned). ...

Show Detail

Hive throws error after sqooping data

I want to import data from database to HDFS in a parquet format then populate the hive table. I can't use sqoop import --hive-import because sqoop moves data from the --target-dir to the hive meta...

Show Detail

How to validate file data from HDFS and Hive table?

I have Requirement in my current project, I am retrieving data from Oracle source Database and load in to HDFS destination directory as a CSV file through Oracle goldengate data replication. Each...

Show Detail

Error Message when sqooping oracle table into hive

I was looking to find out how I can fix the following error message that I keep getting when I am sqooping a data table into oracle. I was able to sqoop another table this morning but every attempt...

Show Detail

How to create an external Hive table if the field value has comma separated values

I had used sqoop-import command to sqoop the data into Hive from teradata. Sqoop-import command is creating a text file with comma(,) as the delimiter. After Sqooping, I had created an external ta...

Show Detail