Having some spare time over the Easter weekend I thought I’d spend a little time trying out a few things, including:

  • Java 8
  • Direct memory allocation with sun.misc.Unsafe

So I made a hash-map implementation that stores map entries on memory allocated outside the typical JVM heap: BinaryOffheapHashMap

First of all: The GC process in Java is a lifesaver, allowing us all to focus on writing functionality rather than obsessing over managing the mundane task of allocating and freeing memory. Yet, it doesn’t mean you shouldn’t be memory-aware. Failing to understand the memory impact of your code can lead to horrible performance, as you’re simply creating too much garbage. You can potentially try to avoid an ill-timed GC pause by having a bigger heap, but you’re just postponing the inevitable.

Then there’s the case when you want to store huge amounts of data inside your JVM app. The lifecycle of this data is well understood and not part of any fancy distributed/parallel algorithm. You push your data in, then get it back out, potentially deleting it at some point. Isn’t it a bit wasteful to allocate a 200G heap, use it mainly for this purpose, and have the GC process scan through all that? For GC, the good die young, and it doesn’t scale that well with increasing heap space.

It’s for these reasons we’re beginning to see more and more focus on off-heap storage structures in Java. Off-heap here relates to memory not managed as part of the standard JVM heap, thereby not being part of what the garbage collector cares to look at. Products like Hazelcast or Oracle Coherence both provide this option for their distributed maps.

Several of these off-heap maps still keep their keys on-heap, for various reasons. The BOHMap as I call it, which stands for Binary Off-heap HashMap, does not. This allows it to keep only a few private fields, 2 ints + 2 longs + 2 object references (Unsafe and a hash function), for the duration of the instance. The rest are short lived or managed off-heap.

Performance testing BOHMap against the standard HashMap is a little tricky. I could simple allocate just enough heap to store the test data, thereby forcing the JVM GC to spend a lot of time doing its job. That would surely make BOHMap look very impressive, but I could just as well argue the heap space wasn’t correctly configured for the test case. One thing is for sure, BOHMap must do more work than the standard HashMap, since it needs to copy data between on-heap and off-heap memory.

If we start by looking at the fantasy case, where the JVM GC doesn’t need to go mental, here’s what I see:

On a AWS r3.8xlarge instance with 244 GiB of system memory, I give the JVM a 200G heap using -Xms200G -Xmx200G when testing the traditional HashMap. This is then able to put/update about 1 million entries per second. If I’d continued the test longer, the GC would likely have impacted this significantly.

However, running with standard heap settings on this machine, forcing the GC to kick in earlier while the test is running, this drops to around 200k-300k per second.

On the BOHMap case I see around 500k+ put/updates per second, with no/little impact on the heap and GC. I set the partition count equal to “expected map size” divided by 5 for those results, but had similar results when dividing by 25 instead of 5 up to about 10 million map entries.

For the record: I was using 64-bit Java 8u5 on Ubuntu 13.10, with each key and value being a 64 byte big Binary object. Their content was random.

So there you go. I hope some of you might find this useful, especially if you need to store a lot of data in memory. I sure enjoyed creating it, playing around with sun.misc.Unsafe 🙂