hadoop-0.18.3 Could not create the Java virtual machine

Installed hadoop on a VM, and needed to set the java heap size, -Xmx1000m, lower than the default 1000 to get it to work.  I set the HADOOP_HEAPSIZE var in the conf/hadoop-env.sh dir to the lower value, but hadoop continued to spit out this error:

# hadoop -help
Could not create the Java virtual machine.
Exception in thread "main" java.lang.NoClassDefFoundError: Could_not_reserve_enough_space_for_object_heap
Caused by: java.lang.ClassNotFoundException: Could_not_reserve_enough_space_for_object_heap
        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
Could not find the main class: Could_not_reserve_enough_space_for_object_heap.  Program will exit.

Didn’t matter what I set the HADOOP_MAXHEAP to, the problem persisted. I never did find the answer online, so figured I do the world a favor today and make a note about how to fix it. Maybe I’ll save someone else the 2 hours it took me to figure this out!

THE SOLUTION:

Change the bin/hadoop file. Here’s what to change:

--- hadoop.orig 2009-04-22 22:07:58.000000000 -0700
+++ hadoop      2009-04-22 21:57:00.000000000 -0700
@@ -233,7 +233,7 @@
 # setup 'java.library.path' for native-hadoop code if necessary
 JAVA_LIBRARY_PATH=''
 if [ -d "${HADOOP_HOME}/build/native" -o -d "${HADOOP_HOME}/lib/native" ]; then
-  JAVA_PLATFORM=`CLASSPATH=${CLASSPATH} ${JAVA} org.apache.hadoop.util.PlatformName | sed -e "s/ /_/g"`
+  JAVA_PLATFORM=`CLASSPATH=${CLASSPATH} ${JAVA} ${JAVA_HEAP_MAX} org.apache.hadoop.util.PlatformName | sed -e "s/ /_/g"`
   if [ -d "$HADOOP_HOME/build/native" ]; then
     JAVA_LIBRARY_PATH=${HADOOP_HOME}/build/native/${JAVA_PLATFORM}/lib
   fi

JAVA_HEAP_MAX needs to be used when executing JAVA to get the JAVA_PLATFORM variable: On my VM, I can’t just call java, too much memory required; I must pass a -Xmx128m any time I call the java executable. With the JAVA_PLATFORM variable set incorrectly, the java execution of hadoop was failing.

This is a Virtozzo (aka Parrellels, aka OpenVZ) VM, by the way. I’ll send a note to Apache, maybe the already know!? The solution is so simple!

About Dave Koopman

Started with Go Daddy in 2002 as a software developer in Email. In a team environment, we built Web-Based Email and Online Group Calendar. Since, positions include: Manager of Dedicated Hosting Development, Systems Architect, Manager of Mysql Operations, Email Architect, Chief Technology Officer, VP Product Development, Chief Scientist and most recently Principal Engineer - Hosting Performance Team. When not innovating solutions at GoDaddy, hobbies include skiing, offroading, boating and camping. This is a personal blog and opinions are of my own and not necessarily that of my employer. I also author on Inside.GoDaddy.com. Connect with me on Google.
This entry was posted in Code, Linux and tagged . Bookmark the permalink.

11 Responses to hadoop-0.18.3 Could not create the Java virtual machine

  1. DaveK says:

    The developers already know. I downloaded verion 0.20.0 and checked it:

    JAVA_PLATFORM=`CLASSPATH=${CLASSPATH} ${JAVA} -Xmx32m org.apache.hadoop.util.PlatformName | sed -e “s/ /_/g”`

    Here they are just hard coding it to -Xmx32m, which is better. If I have my HADOOP_HEAPSIZE cranked way up on a big server, I don’t want to use this value for getting the JAVA_PLATFORM!

  2. DaveK says:

    I’m just going to use the 0.20.0 version, even though 0.18.3 is the latest version marked stable.

  3. Martinus says:

    Hi,

    I’m using hadoop-0.20.3 and I have the same problem, when I start-all.sh. I changed the bin/hadoop file as above, but it still gave me the same error.

    Thanks.

  4. hello!,I really like your writing so much! proportion we keep up a correspondence extra approximately
    your article on AOL? I require a specialist in this space
    to solve my problem. May be that is you! Taking a look ahead
    to see you.

  5. Nice post. I used to be checking continuously this blog and I am impressed!

    Extremely useful information particularly the final section :)
    I take care of such information much. I used to be looking for this certain info for a long time.
    Thanks and best of luck.

  6. Greetings I am so excited I found your blog, I really found you by
    mistake, while I was searching on Google for something else, Anyways I am here now and would
    just like to say thank you for a incredible post and
    a all round thrilling blog (I also love the theme/design),
    I don’t have time to go through it all at the minute but I
    have bookmarked it and also included your RSS feeds, so
    when I have time I will be back to read a lot more, Please do
    keep up the excellent work.

  7. Howdy! I realize this is sort of off-topic but I
    needed to ask. Does managing a well-established blog like
    yours take a massive amount work? I am completely new to blogging but I do write in my diary on a daily basis.
    I’d like to start a blog so I will be able to
    share my personal experience and thoughts online.
    Please let me know if you have any recommendations or tips
    for new aspiring bloggers. Thankyou!

  8. Hey are using WordPress for your site platform? I’m new to
    the blog world but I’m trying to get started and set up my own. Do you require any coding knowledge to make your own blog?
    Any help would be really appreciated!

  9. I don’t know if it’s just me or if everyone else encountering issues with your site.
    It appears as if some of the text in your posts are running off the screen. Can someone else please comment and let me
    know if this is happening to them too? This might be a problem
    with my web browser because I’ve had this happen before.
    Appreciate it

  10. not fake says:

    You’ll want the promissory note of the previous
    home loan, a copy from the loan statement,
    proof income, tax data, insurance information, therefore
    on not fake brick and mortar lenders
    will expect you to write a check payable in their mind and still have it dated for just two weeks after you receive
    the credit.

  11. I was curious if you ever thought of changing the layout of your website?
    Its very well written; I love what youve got to say. But
    maybe you could a little more in the way of content so people could connect with it better.
    Youve got an awful lot of text for only having 1 or two images.

    Maybe you could space it out better?

Leave a Reply

Your email address will not be published. Required fields are marked *

* Copy This Password *

* Type Or Paste Password Here *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>