December 19, 2006
Silly ObjectGrid Java 6 vs Java 5 test
I have a real simple kick the tires type ObjectGrid app that just puts a 100k entries in a local Map and then pulls them one one at a time.
With IBMs Java 5, I get 640ms to insert 100k entries in a single transaction and 8.3 secs to pull them out with one pull per transaction. It runs this test in a loop allowing the JIT to warm up etc and these numbers are after several iterations when it basically stablizes. Suns Java 6 takes 1.2s to insert and 11.4 secs to pull them out.
The same command line arguments are used for both, i.e. set the heap to 512M -Xmx512M. I didn't spend much time figuring out why, this whole exercise took 10 minutes. But, interesting considering the out of the box performance claims none the less. Clearly, this isn't a definitive IBM is better type claim, it's just what I saw on an initial lets see how fast Java 6 is, it may indeed be quicker in other tests.
December 19, 2006 | Permalink
How about testing WAS on the sun & ibm JVM's on windows or linux,
ooops! you can't do that, me bad
Posted by: jonathan | Dec 20, 2006 8:53:59 AM
It might be fun to see an IBM Java 5 vs Sun Java 5 test to see if the difference in inherent to the significant differences in the JVM internals rather than a move from 5 to 6. Were there any GC cycles as this tests or where they visible in some of the loops and you discarded them.
I often split apps up and use a JNI call to measure the processor cycle count between certain point you often see differences differences in core functions such as object allocation etc.
If you post the test case I can play with that if you want, I'm sitting in a hotel in Switzerland quite bored.
Posted by: Paul Anderson | Dec 20, 2006 12:12:05 PM
The biggest gotcha's with Sun vs IBM JVM comparisons are:
1. IBM default GC model is optimized mark and sweep while the Sun default model is generational GC. Unfortunately, both of these companies have focused more on their default GC model than the other in recent years, so I'm not sure that configuring one to use the other's default would be a "fair" test. My guess is that IBM has put more into their generational GC algorithm than Sun has put into their mark and sweep algorithm, so I'd probably take that route.
2. The Sun -client vs -server runtime and the dynamic Server level machine detection.
In most tests I've run, enabling the server runtime improves performance significantly, at the cost of longer warmup times. Also, the server VM doesn't appear to be available on Windows for i586, so i586 Linux is probably the best place for the comparison.
Posted by: sbranda | Dec 28, 2006 2:23:22 AM