Busy wait and queue performance

For the last month or so I’ve been rather busy developing a trading platform for algorithmic trading, connected to LMAX (you should also check out their very interesting Disruptor framework). It’s a rather comprehensive solution with both risk management, position management, back testing, data management on tick level, etc built in. Everything that connects to and communicates with the exchange is event based, basically on a topic based setup. So one event producer may have zero or more event consumers. I’ve really enjoyed finally having a proper concurrent project to work on, poking around the Java concurrency source code in order to find a few tips and tricks as to how I can squeeze out a bit more performance. It’s not that the trading I’ll be doing is that high frequent per say, it’s more about eliminating latency. Now, of course, most of the computational power will be needed in the actual business logic as that’s where the heavy lifting happens. But since the event system that glues everything together is so omnipresent, having it perform well is also very important. Analyzing the spacing between each tick for presumably one of their busiest instruments, EUR/USD, indicates that the most frequent observation is between 1 and 10 milliseconds, so that gives me a rough estimate as to how quick the consumers need to be. If they can’t keep up on average the events will begin queuing up, which is very bad. For the event system itself, delivery of one event from a producer to a consumer is on average 170 microseconds, so that basically leaves the consumer with almost 100% of the time (given that the exchange connected producers are very-to-fairly lightweight). I’m not really sure where to do further optimizations along my critical paths, so I might need to go crazy and do manual memory management or something (with the Unsafe class), but I doubt I’ll need it. I think most of the quick win gains currently is at the network/IO level. Anyway, I digress. Because the point is that while performance testing various parts I got some conflicting evidence as to which concurrent queue implementation to use. So I decided to do a somewhat more comprehensive test while still running it in an isolated environment. There are three factors at play here: Do we have a slow or quick producer, slow or quick consumer, and finally are we using busy wait loops when waiting for events? A slow producer or consumer would typically be slow if it needs to wait for IO, so that’s simulated here with a Thread.sleep(1) call. From JavaDoc we know: A ConcurrentLinkedQueue is an appropriate choice when many threads will share access to a common collection. This queue does not permit null elements. ArrayBlockingQueue is a classic “bounded buffer”, in which a fixed-sized array holds elements inserted by producers and extracted by consumers. This class supports an optional fairness policy for ordering waiting producer and consumer threads LinkedBlockingQueue typically have higher throughput than array-based queues but less predictable performance in most concurrent applications. ...

November 13, 2011 · 6 min · Christian Felde

Cloud computing makes your service a commodity

Just a brief blog post as I’m mostly busy working hard on my dissertation (which should be made available here in a few weeks) and applying for jobs. Hopefully the job thing should be signed in a few weeks as well, as there are a few interesting in the pipeline at the moment. Anyway, thought it was time to post something, so here goes. Been following Brightbox for some time. They are a new cloud computing provider (but not a new hosting provider in general), and they give off a good first impression. Yesterday they published their pricing plan for the soon to be non-beta platform. I don’t have any reason for doubting the quality of their service, and I often like to support the “little guys”, but man, cloud computing really have turned servers into a commodity. ...

August 20, 2011 · 2 min · Christian Felde

Excessive Optimism and Analyst Recommendations

This blog post taps into some of the theories outlined by behavioral finance. Specifically I’m going to look at research done on what is called overconfidence. Overconfidence among investors can manifest itself in many ways. One of these forms is excessive optimism (or pessimism) with regards to beliefs in future outcome. Carleton, Chen, and Steiner (1998) and Jegadeesh, and Kim (2003) study the value of analyst recommendations. There are two aspects worth considering specifically in this context. The first is the quality of the recommendations and the second is the market reaction to them. The value of an analyst report can be defined as its impact on the market, thus the quality of the report and market reaction is not mutually exclusive but rather tightly linked. This poses some potential issues as there might be a feedback loop at work here, where analysts with a broader audience could potentially have a bigger impact than more unknown analysts with less exposure. However, given that the topic in question is an anomaly of the efficient market hypothesis, this potential feedback loop is of interest. In an efficient market, any new information in the form of a buy/sell recommendation must only have an immediate impact without any positive serial correlation in future abnormal returns if it represents new information. I will begin by analyzing a number of aspects related to analyst recommendations and proceed to cover market impact. ...

June 27, 2011 · 5 min · Christian Felde

Node.js, is this the next big thing?

Although I did briefly notice node.js a few months back, it didn’t really catch my interest until I saw this video; Node.js: JavaScript on the Server. I’ve been developing JavaScript on the client (browser) side more or less for the last 5 years. And I didn’t enjoy it too much right away, but with libraries like jQuery, Prototype, Scriptaculous, Ext JS, GWT, etc + continued performance improvements in the JS engines this has changed. It’s now at the point that I feel a lot more productive creating “business apps” with HTML, JavaScript and CSS within a browser then I ever felt using things like Swing or the equivalent in the .NET world. ...

August 4, 2010 · 3 min · Christian Felde