Where is the classpath set for hadoop
NickName:Bourne Ask DateTime:2015-02-01T15:53:33

Where is the classpath set for hadoop

Where is the classpath for hadoop set? When I run the below command it gives me the classpath. Where is the classpath set?

  bin/hadoop classpath

I'm using hadoop 2.6.0

Copyright Notice:Content Author:「Bourne」,Reproduced under the CC 4.0 BY-SA copyright license with a link to the original source and this disclaimer.
Link to original article:https://stackoverflow.com/questions/28260653/where-is-the-classpath-set-for-hadoop

Answers
Siva Krishnan 2017-12-28T10:17:35

Open your bash profile (~/.profile or ~/.bash_profile) for editing and add the following:\n\n\nexport HADOOP_HOME=\"/usr/local/Cellar/hadoop\" then Replace with your own path\nexport HADOOP_CLASSPATH=$(find $HADOOP_HOME -name '*.jar' | xargs echo | tr ' ' ':')\nSave the changes and reload.\nsource ~/.profile\n",


Ashrith 2015-02-01T08:28:52

As said by almas shaikh it's set in hadoop-config.sh, but you could add more jars to it in hadoop-env.sh\n\nHere is a relevant code from hadoop-env.sh which adds additional jars like capacity-scheduler and aws jar's.\n\nexport HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-\"/etc/hadoop\"}\n\n# Extra Java CLASSPATH elements. Automatically insert capacity-scheduler.\nfor f in $HADOOP_HOME/contrib/capacity-scheduler/*.jar; do\n if [ \"$HADOOP_CLASSPATH\" ]; then\n export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$f\n else\n export HADOOP_CLASSPATH=$f\n fi\ndone\n\n# ... some other lines omitted\n\n# Add Aws jar\nexport HADOOP_CLASSPATH=$HADOOP_CLASSPATH:share/hadoop/tools/lib/*\n",


SMA 2015-02-01T08:29:27

When you run hadoop command, it sources a file hadoop-config.sh that resides in $HADOOP_HDFS_HOME/libexec which sets your classpath (CLASSPATH) by picking jars residing in various directories viz. \n\n$HADOOP_HDFS_HOME/share/hadoop/mapreduce \n$HADOOP_HDFS_HOME/share/hadoop/common\n$HADOOP_HDFS_HOME/share/hadoop/hdfs etc.\n",


Ming 2015-02-01T08:29:24

As per this blog post, it is in an environment variable named HADOOP_CLASSPATH. You can set it as you would any other environment variable, the specifics of which depend on which shell you use. If you use bash, then you can call like export HADOOP_CLASSPATH=/path/to/wherever:/path/to/wherever/else.",


More about “Where is the classpath set for hadoop” related questions

Where is the classpath set for hadoop

Where is the classpath for hadoop set? When I run the below command it gives me the classpath. Where is the classpath set? bin/hadoop classpath I'm using hadoop 2.6.0

Show Detail

Where and How to set HADOOP_CLASSPATH

I have written a maven program and then I build it and put 3rd party jars on target/lib folder. with which command and where I can put them in my HADOOP_CLASSPATH? The location of setting

Show Detail

How to set HADOOP_CLASSPATH for Oozie MapReduce action

I can run my MapReduce job manually to load data into Hbase without any issue, HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HADOOP_MAPRED_HOME/../hbase/* HADOOP_CLASSPATH=$HADOOP_CLASSPATH:./* export

Show Detail

Hadoop HADOOP_CLASSPATH issues

This question doesn't refer to distributing jars in the whole cluster for the workers to use them. It refers to specifying a number of additional libraries on the client machine. To be more speci...

Show Detail

Hadoop User Classpath - Beeline

Is there any way I can set HADOOP_USER_CLASSPATH_FIRST for beeline MapReduce and spark engines export HADOOP_USER_CLASSPATH_FIRST=true export HADOOP_CLASSPATH=/xx/jackson/jackson-core-2.6.5.jar:/xx/

Show Detail

Setting additional classpath for a hadoop tool

How can I dynamically set up an additional classpath so that jars are accessible from a hadoop tool class? I'm running my job via "hadoop jar". I need something like the "-libjars" switch which

Show Detail

Where is the classpath for hadoop mapreduce?

I've been looking for the class path for the Hadoop mapreduce classes. I'm using Hortonworks version 2.2.4 sandbox. I need the classpath for running my javac compiler as such: javac -cp (CLASS_PAT...

Show Detail

Classpath to use for MapR/Hadoop/Hive

I'm trying to compile some java code for hadoop and need to know what classpath I need to specify. For cloudera I use this below but what do I use for a MapR installation? Surprisingly I could only...

Show Detail

Hadoop classpath psudo-distributed mode

I'm running some simple tests of Hadoop locally in psudo-distributed mode. But the reducer is giving me a ClassNotFoundException. I've got the HADOOP_CLASSPATH set as such: export HADOOP_CLASSPATH=/

Show Detail

Hadoop DistributedCache classpath resolution

I am facing an issue using a Hadoop's DistributedCache. I am getting java.lang.ClassNotFoundException for the jars which were successfully added to cache. I revealed the problem: - The client is

Show Detail