getting started with the latest version of hadoop and ec2
NickName:Alex Gordon Ask DateTime:2013-04-30T00:49:21

getting started with the latest version of hadoop and ec2

I am following Hadoop in Action to get started with hadoop with ec2. I'm running on ubuntu and have downloaded and installed the latest version of Hadoop. I am hitting a road block at this command:

hadoop-ec2 launch-cluster mycluster 2

The book says "The Hadoop EC2 tools are in the directory src/contrib/ec2/bin under your Hadoop installation. Recall that our ec2-init.sh script has already added that directory to your system PATH. Within that directory is hadoop-ec2, which is a meta-command for executing other commands. To launch a Hadoop Cluster on ec2 use:

hadoop-ec2 launch-cluster < cluster-name> < number-of-slaves>"


The response I get is: hadoop-ec2: command not found

I noticed that the variable $HADOOP_HOME is not set.

It looks like this book is out-dated.

  1. can someone direct me to a tutorial that was created in the last couple of months on how to set up hadoop with ec2?
  2. After some quick googling, it seems that HADOOP_HOME is deprecated. Is this true?
  3. I am able to without problems execute ec2-describe-images. and get all the available images that I can use. Why doesn't hadoop-ec2 command work?

Thank you for your guidance.

Copyright Notice:Content Author:「Alex Gordon」,Reproduced under the CC 4.0 BY-SA copyright license with a link to the original source and this disclaimer.
Link to original article:https://stackoverflow.com/questions/16284028/getting-started-with-the-latest-version-of-hadoop-and-ec2

More about “getting started with the latest version of hadoop and ec2” related questions

getting started with the latest version of hadoop and ec2

I am following Hadoop in Action to get started with hadoop with ec2. I'm running on ubuntu and have downloaded and installed the latest version of Hadoop. I am hitting a road block at this command:

Show Detail

Getting started with Hadoop and Eclipse

I'm following a couple of tutorials for setting up Hadoop with Eclipse. This one is from Cloudera : http://v-lad.org/Tutorials/Hadoop/05%20-%20Setup%20SSHD.html But this seems to focus on checking...

Show Detail

Hadoop showing old version despite latest version installation

I am trying to install hadoop in my ubuntu OS. I followed each and every step exactly from this link Hadoop Install Tutorial and everything was going as expected until i tried to run $ start-dfs.s...

Show Detail

Latest compatible version of hadoop and hbase

I have to install hadoop and hbase on 4 machines. I have found latest version of hadoop-2.6.0 and hbase-0.98.9-hadoop2-bin. 1)Are they compatible? 2)How I should install hbase and hadoop on 4 mac...

Show Detail

Hadoop latest version in binary?

when I was about to download the latest stable version of hadoop as of today (06/27/2014), I find that there is no bin.tar.gz available. I see the following. I am not sure which one I should downlo...

Show Detail

Script to set up Hadoop on EC2

The script to setup Hadoop on EC2 as described in https://wiki.apache.org/hadoop/AmazonEC2 has been removed from recent hadoop release. Google points me to an alternative http://whirr.apache.org/...

Show Detail

Cascalog Hadoop version support

I notice that the Cascalog getting started guide specifies a version of Hadoop :profiles { :dev {:dependencies [[org.apache.hadoop/hadoop-core "1.0.3"]]}} If my group uses a different version of ...

Show Detail

Hadoop version layout error

I am getting this error in one or two nodes in a Hadoop cluster, where rest of the nodes are running fine: ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: ***********

Show Detail

Codebuild latest node version mismatched with EC2

Has anyone run into the issue that Codebuild's latest version for node 14 is 14.19.2 but EC2's latest supported version of node 14 is 14.19.1. Since Codebuild doesn't support specifying minor versi...

Show Detail

Hadoop Data node is not getting started

I am using Hadoop version 1.2.1. on single node. When I try to start all nodes using bin/start-all.sh on linux. Data node does not start. In the log file of data node: " 2014-02-19 12:27:41,085 IN...

Show Detail