Java

Around 6 years ago I was experimenting and pushing the practical boundaries of what the JVM could handle with regards to heap size. We had Java 8 at the time, and allocating hundreds of GB of heap space was risky. Lots of time and effort was spend in many IT teams around the world just trying to keep the GC happy with this setup. Some, often myself included, would recommend we aim for an architecture with several smaller JVMs, each dealing with a more manageable heap size. But such a setup does sometimes bring with it a lot of complexity. While a web application might easily scale horizontally like that, not all application have that luxury. ...

Having some spare time over the Easter weekend I thought I’d spend a little time trying out a few things, including: Java 8 Direct memory allocation with sun.misc.Unsafe So I made a hash-map implementation that stores map entries on memory allocated outside the typical JVM heap: BinaryOffheapHashMap First of all: The GC process in Java is a lifesaver, allowing us all to focus on writing functionality rather than obsessing over managing the mundane task of allocating and freeing memory. Yet, it doesn’t mean you shouldn’t be memory-aware. Failing to understand the memory impact of your code can lead to horrible performance, as you’re simply creating too much garbage. You can potentially try to avoid an ill-timed GC pause by having a bigger heap, but you’re just postponing the inevitable. Then there’s the case when you want to store huge amounts of data inside your JVM app. The lifecycle of this data is well understood and not part of any fancy distributed/parallel algorithm. You push your data in, then get it back out, potentially deleting it at some point. Isn’t it a bit wasteful to allocate a 200G heap, use it mainly for this purpose, and have the GC process scan through all that? For GC, the good die young, and it doesn’t scale that well with increasing heap space. ...

For the TLDRs: This blog post presents the reasoning behing a project called ResourcePriorityBlockingQueue which is a blocking queue implementation that: Allows you to assign priority on tasks, just as with a PriorityBlockingQueue. Tasks may belong to task groups where each group may have a different priority. In that case, tasks are prioritized by group first, then task priority second. For a particular instance of a ResourcePriorityBlockingQueue you give it implementations of a ResourcePrioritizer and a ResourceAllocator, which further defines to which resources a particular task is made available. Focus as been put on making the code highly concurrent with as little synchronized code as possible. Lock free code is used where applicable, with efficient use of internal data structures. This gives ResourcePriorityBlockingQueue a performance characteristic which is comparable to that of a pure PriorityBlockingQueue when used in a similar fashion. It’s available on GitHub, so feel free to poke around. Any constructive feedback is always welcome. ...

For the last month or so I’ve been rather busy developing a trading platform for algorithmic trading, connected to LMAX (you should also check out their very interesting Disruptor framework). It’s a rather comprehensive solution with both risk management, position management, back testing, data management on tick level, etc built in. Everything that connects to and communicates with the exchange is event based, basically on a topic based setup. So one event producer may have zero or more event consumers. I’ve really enjoyed finally having a proper concurrent project to work on, poking around the Java concurrency source code in order to find a few tips and tricks as to how I can squeeze out a bit more performance. It’s not that the trading I’ll be doing is that high frequent per say, it’s more about eliminating latency. Now, of course, most of the computational power will be needed in the actual business logic as that’s where the heavy lifting happens. But since the event system that glues everything together is so omnipresent, having it perform well is also very important. Analyzing the spacing between each tick for presumably one of their busiest instruments, EUR/USD, indicates that the most frequent observation is between 1 and 10 milliseconds, so that gives me a rough estimate as to how quick the consumers need to be. If they can’t keep up on average the events will begin queuing up, which is very bad. For the event system itself, delivery of one event from a producer to a consumer is on average 170 microseconds, so that basically leaves the consumer with almost 100% of the time (given that the exchange connected producers are very-to-fairly lightweight). I’m not really sure where to do further optimizations along my critical paths, so I might need to go crazy and do manual memory management or something (with the Unsafe class), but I doubt I’ll need it. I think most of the quick win gains currently is at the network/IO level. Anyway, I digress. Because the point is that while performance testing various parts I got some conflicting evidence as to which concurrent queue implementation to use. So I decided to do a somewhat more comprehensive test while still running it in an isolated environment. There are three factors at play here: Do we have a slow or quick producer, slow or quick consumer, and finally are we using busy wait loops when waiting for events? A slow producer or consumer would typically be slow if it needs to wait for IO, so that’s simulated here with a Thread.sleep(1) call. From JavaDoc we know: A ConcurrentLinkedQueue is an appropriate choice when many threads will share access to a common collection. This queue does not permit null elements. ArrayBlockingQueue is a classic “bounded buffer”, in which a fixed-sized array holds elements inserted by producers and extracted by consumers. This class supports an optional fairness policy for ordering waiting producer and consumer threads LinkedBlockingQueue typically have higher throughput than array-based queues but less predictable performance in most concurrent applications. ...