When builds fail for no reason: Feeding Maven memory

Ever experience a wonderfully fantastic “green bar!” in Eclipse, NetBeans, or IntelliJ, only to find that when you run your ANT or Maven build, you get an equally catastrophic build failure? If you have, you’ve probably tried what most of us tried, and you’ve attempted to increase Maven’s heap capacity using MVN_OPTS:

export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=512m"

That would normally work fine if the error we received is something like:

java.lang.OutOfMemoryError: PermGen space

Unfortunately, though, this doesn’t solve the problem because Maven actually uses a separate JVM for each JUnit test execution! So while we have successfully enabled Maven to be a hog, our tests are still running in a constrained environment. What we need to do instead is increase the memory capacity of the Maven Surefire launcher:

Increasing JUnit Test Memory in Maven

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <argLine>-Xms256m -Xmx512m -XX:MaxPermSize=512m</argLine>
  </configuration>
</plugin>

Our solution

This will expand your test memory capacity significantly, and if it’s still not enough, you can play with these numbers for your environment; however, to all you Maven users out there who’ve run into these problems, may the Force be with you, because sometimes it’s not obvious when a memory issue is the root of your problems. However, if you are using Arquillian in your test suite, chances are that you are going to run into a memory issue at some point, because the memory required to start up a container like Jetty or Tomcat in your test suite is significant. If you are using JBoss AS7 or GlassFish, then your server container will run in a different JVM and this will not typically be an issue.

A very confusing example

Here is an interesting JVM core dump in the [[Rewrite]] and [[PrettyFaces]] test suites that was actually being caused by an out-of-memory error leading to native code execution failure, but nothing in the error trace would lead us to think it! (Unimportant bits omitted:)

Core dump caused by Out-of-memory error

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGBUS (0x7) at pc=0x00007fac3c24e3bf, pid=32027, tid=140377731290880
#
# JRE version: 7.0_03-b04
# Java VM: Java HotSpot(TM) 64-Bit Server VM (22.1-b02 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libc.so.6+0x1313bf]  __nss_hosts_lookup+0x1117f
#
# Core dump written. Default location: ~/Projects/ocpsoft/prettyfaces/annotations/core or core.32027
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

---------------  T H R E A D  ---------------

Current thread (0x0000000000bef800):  JavaThread "main" [_thread_in_native, id=32030, stack(0x00007fac3cbed000,0x00007fac3ccee000)]

siginfo:si_signo=SIGBUS: si_errno=0, si_code=2 (BUS_ADRERR), si_addr=0x00007fac3534c000

Registers:
...

Top of Stack: (sp=0x00007fac3cce7b68)
...

Instructions: (pc=0x00007fac3c24e3bf)
...

Register to memory mapping:
...


Stack: [0x00007fac3cbed000,0x00007fac3ccee000],  sp=0x00007fac3cce7b68,  free space=1002k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libc.so.6+0x1313bf]  __nss_hosts_lookup+0x1117f
C  [libzip.so+0x5057]  ZIP_GetNextEntry+0x57
j  java.util.zip.ZipFile.getNextEntry(JI)J+0
j  java.util.zip.ZipFile.access$500(JI)J+2
j  java.util.zip.ZipFile$1.nextElement()Ljava/util/zip/ZipEntry;+54
j  java.util.zip.ZipFile$1.nextElement()Ljava/lang/Object;+1
j  java.util.jar.JarFile$1.nextElement()Ljava/util/jar/JarFile$JarFileEntry;+4
j  java.util.jar.JarFile$1.nextElement()Ljava/lang/Object;+1
j  sun.misc.URLClassPath$JarLoader.validIndex(Ljava/lang/String;)Z+42

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  java.util.zip.ZipFile.getNextEntry(JI)J+0
j  java.util.zip.ZipFile.access$500(JI)J+2
j  java.util.zip.ZipFile$1.nextElement()Ljava/util/zip/ZipEntry;+54
j  java.util.zip.ZipFile$1.nextElement()Ljava/lang/Object;+1
j  java.util.jar.JarFile$1.nextElement()Ljava/util/jar/JarFile$JarFileEntry;+4
j  java.util.jar.JarFile$1.nextElement()Ljava/lang/Object;+1
j  sun.misc.URLClassPath$JarLoader.validIndex(Ljava/lang/String;)Z+42
J  sun.misc.URLClassPath$JarLoader.getResource(Ljava/lang/String;ZLjava/util/Set;)Lsun/misc/Resource;
J  sun.misc.URLClassPath$JarLoader.getResource(Ljava/lang/String;Z)Lsun/misc/Resource;
j  sun.misc.URLClassPath$JarLoader.findResource(Ljava/lang/String;Z)Ljava/net/URL;+3
j  sun.misc.URLClassPath.findResource(Ljava/lang/String;Z)Ljava/net/URL;+17
j  java.net.URLClassLoader$2.run()Ljava/net/URL;+12
j  java.net.URLClassLoader$2.run()Ljava/lang/Object;+1
v  ~StubRoutines::call_stub
j  java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;Ljava/security/AccessControlContext;)Ljava/lang/Object;+0
j  java.net.URLClassLoader.findResource(Ljava/lang/String;)Ljava/net/URL;+13
j  java.lang.ClassLoader.getResource(Ljava/lang/String;)Ljava/net/URL;+30
j  java.net.URLClassLoader.getResourceAsStream(Ljava/lang/String;)Ljava/io/InputStream;+2

Conclusion

It takes a lot of work to crash the JVM like this, so here’s to more memory, more memory for Maven, more memory for all!

Posted in OpenSource

Tom Jenkinson says:

April 26, 2012 at 3:46 am

Thanks for the pro-tip, it came about 30 minutes too late! I have just got round to resolving a very similar (intermittent) issue in one of our builds with native code crashing.

Wish I had seen this yesterday!

[reply]