Cannot run Java class files with hadoop streaming
NickName:Shrish Bajpai Ask DateTime:2011-07-23T01:35:04

Cannot run Java class files with hadoop streaming

Whenever I am trying to use Java class files as my mapper and/or reducer I am getting the following error:

java.io.IOException: Cannot run program "MapperTst.class": java.io.IOException: error=2, No such file or directory

I executed the following command on the terminal:

hadoop@ubuntu:/usr/local/hadoop$ bin/hadoop jar contrib/streaming/hadoop-streaming-0.20.203.0.jar -file /home/hadoop/codes/MapperTst.class -mapper /home/hadoop/codes/MapperTst.class -file /home/hadoop/codes/ReducerTst.class -reducer /home/hadoop/codes/ReducerTst.class  -input gutenberg/* -output gutenberg-outputtstch27


Copyright Notice：Content Author:「Shrish Bajpai」，Reproduced under the CC 4.0 BY-SA copyright license with a link to the original source and this disclaimer.
Link to original article：https://stackoverflow.com/questions/6793856/cannot-run-java-class-files-with-hadoop-streaming

Answers
surfer 2011-12-19T13:32:23

I had the same problem. The solution was for me to put the java mapper/reducer/combiner in a specified package. With the default package it won't work. It will give you the error you had.",

arun_suresh 2011-07-26T19:46:26

Assuming your fully qualified Mapper class name (including the package) is codes.MapperTest and the reducer class name is codes.ReducerTst,\n\nPackage your Map and reduce classes into a jar file say /home/hadoop/test.jar\nYour command should work if you modify it to :\n\n\nhadoop@ubuntu:/usr/local/hadoop$ bin/hadoop jar \\\n contrib/streaming/hadoop-streaming-0.20.203.0.jar \\\n -libjars /home/hadoop/test.jar \\\n -mapper codes.MapperTst \\\n -reducer codes.ReducerTst \\\n -input gutenberg/* -output gutenberg-outputtstch27\n",

Products recommended by Amazon
More >>>
SanDisk 128GB microSDXC-Card, Licensed for Nintendo-Switch - SDSQXAO-128G-GNCZN
SanDisk 128GB microSDXC-Card, Licensed for Nintendo-Switch - SDSQXAO-128G-GNCZN
 HUANUO Dual Monitor Stand - Adjustable Spring Monitor Desk Mount Swivel Vesa Bracket with C Clamp, Grommet Mounting Base for 13 to 27 Inch Computer Screens - Each Arm Holds 4.4 to 14.3lbs
HUANUO Dual Monitor Stand - Adjustable Spring Monitor Desk Mount Swivel Vesa Bracket with C Clamp, Grommet Mounting Base for 13 to 27 Inch Computer Screens - Each Arm Holds 4.4 to 14.3lbs
 AMD Ryzen 5 5600X 6-core, 12-Thread Unlocked Desktop Processor with Wraith Stealth Cooler
AMD Ryzen 5 5600X 6-core, 12-Thread Unlocked Desktop Processor with Wraith Stealth Cooler
 Timberland Men's 6-Inch Premium Waterproof Boot
Timberland Men's 6-Inch Premium Waterproof Boot
More about “Cannot run Java class files with hadoop streaming” related questions
Cannot run Java class files with hadoop streaming

Whenever I am trying to use Java class files as my mapper and/or reducer I am getting the following error: java.io.IOException: Cannot run program "MapperTst.class": java.io.IOException: error=2...

Show Detail
Problem with Hadoop Streaming -file option for Java class files

I am struggling with a very basic issue in hadoop streaming in the "-file" option. First I tried the very basic example in streaming: hadoop@ubuntu:/usr/local/hadoop$ bin/hadoop jar contrib/strea...

Show Detail
Unable to find files in hadoop streaming

I have a similar issue to Hadoop Streaming - Unable to find file error . However none of the solutions presented there are working. My command line is: hadoop jar /mnt/shared/hadoop-streaming-1...

Show Detail
Run java MapReduce using Hadoop Streaming API

I have developed my own mapper.java and reducer.java and want to run them as a hadoop job. I have configured a single-node hadoop cluster and run MapReduce like this: $ bin/hadoop jar contrib/stre...

Show Detail
Hadoop Streaming Problems

I ran into these issues while using Hadoop Streaming. I'm writing code in python 1) Aggregate library package According to the hadoop streaming docs ( http://hadoop.apache.org/common/docs/r0.20.0/

Show Detail
Hadoop Streaming with Java Mapper/Reducer

I'm trying to run a hadoop streaming job with a java Mapper/Reducer over some wikipedia dumps (in compressed bz2 form). I'm trying to use WikiHadoop, which is an interface released by Wikimedia rec...

Show Detail
Hadoop Streaming in .NET

I am running hadoop in Pseudo-Distributed mode and using hadoop streaming to do my map-reduce operations. But the problem is I keep getting Streaming Job Failed error message. Following is the log:

Show Detail
hadoop streaming python windows cannot run mapreduce subprocess failed with code 64

I'm trying to run a simple mapreduce word-count program with python on Windows and I got this error: 2021-08-07 23:30:14,670 INFO mapreduce.Job: Task Id : attempt_1628353447352_0001_m_000001_0, St...

Show Detail
Hadoop streaming with mongo-hadoop connector fails

I created this job that reads a bunch of JSON files from HDFS and tries to load them into MongoDB. It's just the map script because I don't require any additional processing on the reduce step. I'm

Show Detail
Structured streaming writes Parquet files to Hadoop

I was able to write the results of structured streaming to Parquet files. The thing is that those files are in local file system, and now I want to write them to Hadoop file system. Is there any wa...

Show Detail