jvm tuning

Better performance in production servers is possible with proper configuration of JVM parameters, particularily those related to memory usage and garbage collection.

The allocation of memory for the JVM is specified using -X options when starting Resin (the exact options may depend upon the JVM that you are using, the examples here are for the Sun JVM).

unix> bin/httpd.sh -Xmn100M -Xms500M -Xmx500M
win> bin/httpd.exe -Xmn100M -Xms500M -Xmx500M
install win service> bin/httpd.exe -Xmn100M -Xms500M -Xmx500M -install

It is good practice with server-side Java applications like Resin to set the minimum -Xms and maximum -Xmx heap sizes to the same value.

For efficient garbage collection, the -Xmn value should be lower than the -Xmx value.

(thanks to Rob Lockstone for his comments)

There are essentially two GC threads running. One is a very lightweight thread which does "little" collections primarily on the Eden (a.k.a. Young) generation of the heap. The other is the Full GC thread which traverses the entire heap when there is not enough memory left to allocate space for objects which get promoted from the Eden to the older generation(s).

If there is a memory leak or inadequate heap allocated, eventually the older generation will start to run out of room causing the Full GC thread to run (nearly) continuously. Since this process "stops the world", Resin won't be able to respond to requests and they'll start to back up.

The amount allocated for the Eden generation is the value specified with -Xmn. The amount allocated for the older generation is the value of -Xmx minus the -Xmn. Generally, you don't want the Eden to be too big or it will take too long for the GC to look through it for space that can be reclaimed.

See also:

• Troubleshooting Technique: Garbage Collector
• Sun documentation on garbage collection

Each thread in the VM get's a stack. The stack size will limit the number of threads that you can have, too big of a stack size and you will run out of memory as each thread is allocated more memory than it needs.

The Resin startup scripts (httpd.exe on Windows, wrapper.pl on Unix) will set the stack size to 2048k, unless it is specified explicity. 2048k is an appropriate value for most situations.

-Xss determines the size of the stack: -Xss1024k. If the stack space is too small, eventually you will see an exception java.lang.StackOverflowError.

Some people have reported that it is necessary to change stack size settings at the OS level for Linux. A call to ulimit may be necessary, and is usually done with a command in /etc/profile:

ulimit -s 2048

JDK 5 includes a number of tools that are useful for monitoring the JVM. Documentation for these tools is available from the Sun website. For JDK's prior to 5, Sun provides the jvmstat tools.

The most useful tool is jconsole. Details on using jconsole are provided in the Administration section of the Resin documentation.

win> ./httpd.exe -Dcom.sun.management.jmxremote
unix> bin/httpd.sh -Dcom.sun.management.jmxremote

 ... in another shell window ... 

win> jconsole.exe
unix> jconsole

Choose Resin's JVM from the "Local" list.

jps and jstack are also useful, providing a quick command line method for obtaining stack traces of all current threads. Details on obtaining and interpreting stack traces is in the Troubleshooting section of the Resin documentation.

# jps
12903 Jps
20087 Resin
# jstack 20087
Attaching to process ID 20087, please wait...
Debugger attached successfully.
Client compiler detected.
JVM version is 1.5.0-beta2-b51
Thread 12691: (state = BLOCKED)
 - java.lang.Object.wait(long) (Compiled frame; information may be imprecise)
 - com.caucho.util.ThreadPool.runTasks() @bci=111, line=474 (Compiled frame)
 - com.caucho.util.ThreadPool.run() @bci=85, line=423 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)


Thread 12689: (state = BLOCKED)
 - java.lang.Object.wait(long) (Compiled frame; information may be imprecise)
 - com.caucho.util.ThreadPool.runTasks() @bci=111, line=474 (Compiled frame)
 - com.caucho.util.ThreadPool.run() @bci=85, line=423 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=595 (Interpreted frame)

...