Hadoop on Amazon EC2 : Job tracker not starting properly

Posted by Algorist on Stack Overflow See other posts from Stack Overflow or by Algorist
Published on 2010-05-05T06:03:08Z Indexed on 2010/05/05 6:08 UTC
Read the original article Hit count: 393

Filed under:

amazon-ec2

|

amazon-web-services

|

hadoop

|

amazon

|

aws

Hi,

We are running Hadoop on Amazon EC2 cluster. We start the master, slaves and attach the ebs volumes and finally waiting for hadoop jobtracker, tasktracker etc to start and we have timeout of 3600 seconds. We are noticing 50% of the time that job tracker is not able to start before the timeout. Reason being, hdfs is not initialized properly and still in safemode and job tracker is unable to start. I noticed few connectivity issues between nodes on EC2 as I tried manually pinging slaves.

Did anyone face similar issue and know how to solve this?

Thank you
Bala

© Stack Overflow or respective owner

Related posts about amazon-ec2

Amazon EC2 spot instances - is there a catch ?

as seen on Server Fault - Search for 'Server Fault'
I needed to start a new EC2 instance today and decided to try out the new spot instances, where you can reduce your instance cost by bidding on the maximum per-hour price you're prepared to pay. Since today's spot price was only 35c / hour, compared with 85c / hour for an on-demand instance, I was… >>> More
Amazon EC2 spot instances - is there a catch ?

as seen on Server Fault - Search for 'Server Fault'
I needed to start a new EC2 instance today and decided to try out the new spot instances, where you can reduce your instance cost by bidding on the maximum per-hour price you're prepared to pay. Since today's spot price was only 3.5c / hour, compared with 8.5c / hour for an on-demand instance, I… >>> More
Amazon EC2 master node hanging

as seen on Server Fault - Search for 'Server Fault'
Hi, I am using cloudera setup to launch a cluster with hadoop on Amazon. Sometimes, the master hadoop node hangs and we have to restart the job from the job. Did anyone face similar problem and resolve the issue. Thank you. >>> More
Can't Connect to IIS Ftp Site under Amazon EC2

as seen on Server Fault - Search for 'Server Fault'
IIS 7.5: Ftp Firewall Suport: Data Ranges 49152-65535 using external Ip of Amazon EC2 static IP Ftp IPv4 Restriction: allow: Amazon EC2 static IP Ftp Authentication: Anonymous: Enabled, Basic: Disabled, IISMgr: Enabled Ftp Authorization: Allow All Users: Read/Write Windows Firewall (Inbound): Open… >>> More
Amazon EC2 EBS volume scheduled backup/snapshots using puppet

as seen on Server Fault - Search for 'Server Fault'
I am not a Linux admin, although I wish I was, and I have seen these questions Amazon EC2 Backup Strategy Amazon EC2 + EBS:: Regular backup plan? Simple Backup Strategy for Amazon EC2 instances / volumes? And this suggestion http://alestic.com/2009/09/ec2-consistent-snapshot I tried using… >>> More

Related posts about amazon-web-services

amazon web services and sql server support

as seen on Server Fault - Search for 'Server Fault'
Hi All, I have built my application using sql server 2008 and .net framework 3.5 I am looking for a sclable hosting service and have come to think of amazon web services. Does amazon also support hosting of sql server 2008 databases? What hosting services do you advise Thank you. >>> More
Unable to list owned images and running instances from Amazon Web Services using Zend Framework

as seen on Stack Overflow - Search for 'Stack Overflow'
I am using Zend Framework's library to manage EC2 instances and AMI. However I can't list the AMI's I own and can't list existing EC2 instances. $ec2Instance = new Zend_Service_Amazon_Ec2_Instance($awsAccessKey, $awsSecretKey); $instances = $ec2Instance ->describe(); $ec2Instance -describe()… >>> More
Amazon Web services - retrieving a wishlist

as seen on Stack Overflow - Search for 'Stack Overflow'
I've been tinkering with Yahoo Pipes and the Amazon E-Commerce Service (ECS) SDK to retrieve my wishlist. The problem is that although I can get all the items on my wishlist just fine, it seems to include items that I've deleted too. Has anyone else used this API and noticed this? Is there a way… >>> More
Amazon Web Services: A Developer Primer

as seen on Internet.com - Search for 'Internet.com'
With Amazon Web Services (AWS), developers get both scalable services that they can use to architect their applications and the flexibility to run any software on Amazon's compute cloud. >>> More
amazon web services

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, Has anyone of you worked on amazon web services? I just wanted to retrieve the citations of a given book. Is this possible using the aws? Is aws free? Thanks. >>> More