hadoop-streaming example failed to run - Type mismatch in key from map
NickName:johnsam Ask DateTime:2011-09-28T08:09:05

hadoop-streaming example failed to run - Type mismatch in key from map

I was running  $HADOOP_HOME/bin/hadoop  jar $HADOOP_HOME/hadoop-streaming.jar \
    -D stream.map.output.field.separator=. \
    -D stream.num.map.output.key.fields=4 \
    -input myInputDirs \
    -output myOutputDir \
    -mapper org.apache.hadoop.mapred.lib.IdentityMapper \
    -reducer org.apache.hadoop.mapred.lib.IdentityReducer 
What hould be the input file when IdentityMapper is the mapper?

I was hoping to see it can sort on certain selected keys and not the entire keys. My input file is simple "aa bb". "cc dd" Not sure what did I miss? I always get this error java.lang.Exception: java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.LongWritable at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:371) Caused by: java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.LongWritable

Copyright Notice：Content Author:「johnsam」，Reproduced under the CC 4.0 BY-SA copyright license with a link to the original source and this disclaimer.
Link to original article：https://stackoverflow.com/questions/7576985/hadoop-streaming-example-failed-to-run-type-mismatch-in-key-from-map

Answers
Praveen Sripati 2011-09-28T02:35:23

This is a known bug and here is the JIRA. The bug has been identified in Hadoop 0.21.0, but I don't think it's in any of the Hadoop release version. If you are really interested to fix this, you can\n\n\ndownload the source code for Hadoop (for the release you are working)\ndownload the patch from JIRA and apply it\nbuild and test Hadoop\n\n\nHere are the instructions on how to apply a patch.\n\nOr instead of using an IdentityMapper and the IdentityReducder, use a python/perl scripts which will read the k/v pairs from STDIN and then write the same k/v pairs to the STDOUT without any processing. It's like creating your own IdentityMapper and the IdentityReducder not using Java.",

Products recommended by Amazon
More >>>
SanDisk 128GB microSDXC-Card, Licensed for Nintendo-Switch - SDSQXAO-128G-GNCZN
SanDisk 128GB microSDXC-Card, Licensed for Nintendo-Switch - SDSQXAO-128G-GNCZN
 HUANUO Dual Monitor Stand - Adjustable Spring Monitor Desk Mount Swivel Vesa Bracket with C Clamp, Grommet Mounting Base for 13 to 27 Inch Computer Screens - Each Arm Holds 4.4 to 14.3lbs
HUANUO Dual Monitor Stand - Adjustable Spring Monitor Desk Mount Swivel Vesa Bracket with C Clamp, Grommet Mounting Base for 13 to 27 Inch Computer Screens - Each Arm Holds 4.4 to 14.3lbs
 AMD Ryzen 5 5600X 6-core, 12-Thread Unlocked Desktop Processor with Wraith Stealth Cooler
AMD Ryzen 5 5600X 6-core, 12-Thread Unlocked Desktop Processor with Wraith Stealth Cooler
 Timberland Men's 6-Inch Premium Waterproof Boot
Timberland Men's 6-Inch Premium Waterproof Boot
More about “hadoop-streaming example failed to run - Type mismatch in key from map” related questions
hadoop-streaming example failed to run - Type mismatch in key from map

I was running $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/hadoop-streaming.jar \ -D stream.map.output.field.separator=. \ -D stream.num.map.output.key.fields=4 \ -input myInputDirs \ -

Show Detail
Type mismatch in key from map In Hadoop Sorting

I am creating a simple hadoop sorting example, I have following code. I am using ready-made InverseMaper and Identity reducer FileInputFormat.setInputPaths(conf, new Path(args[0])); ...

Show Detail
MapReduce with Hadoop: Type mismatch in key from map

I'm running a simple wordcount program, and I get the following error: Type mismatch in key from map: expected org.apache.hadoop.io.Text, received org.apache.hadoop.io.LongWritable What is the me...

Show Detail
Type mismatch in key from map even though i setMapOutputKey class

I am getting type mismatch from map even though i have set map output key class and map output value class. This is my sample code. public class NgramCount{ protected final static String RAWCOUN...

Show Detail
Type mismatch in key from map as it identified as LongWriteable

I wrote a MapReduce program to process the data from a text file. But when I run it locally (Linux VM), it hits an error that it identified the key from the map as a LongWriteable not Text as requi...

Show Detail
Task fails in hadoop because of "Type mismatch in key from map"

I have been folowing this tutorial for setting up hadoop on a single windows machine. Both the NameNode and JobTracker services are working respectively on http://localhost:50070 and http://localhost:

Show Detail
Hibernate: mapping with a map and getting argument type mismatch

I've got this class DepartmentPerson in Java where I've mapped a Map in Hibernate as following: <map cascade="save-update" name="acceptByPeriod" table="

Show Detail
Type mismatch in value from map

I am running this code using oozie workflow and getting type mismatch error: public static class mapClass extends Mapper<Object, Text, LongWritable, LongWritable> { public void map(...

Show Detail
MapReduce Key type mismatch

I am new to MapReduce, in my MapReduce program I got a key type mismatch error: Error: java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.io.LongWritable, received org.a...

Show Detail
Type mismatch in key from map

I'm getting this error: java.lang.Exception: java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.io.Text, received org.apache.hadoop.io.LongWritable at org.apache.ha...

Show Detail