For the TLDRs:
This blog post presents the reasoning behing a project called ResourcePriorityBlockingQueue which is a blocking queue implementation that:
- Allows you to assign priority on tasks, just as with a PriorityBlockingQueue.
- Tasks may belong to task groups where each group may have a different priority. In that case, tasks are prioritized by group first, then task priority second.
- For a particular instance of a ResourcePriorityBlockingQueue you give it implementations of a ResourcePrioritizer and a ResourceAllocator, which further defines to which resources a particular task is made available.
- Focus as been put on making the code highly concurrent with as little synchronized code as possible. Lock free code is used where applicable, with efficient use of internal data structures. This gives ResourcePriorityBlockingQueue a performance characteristic which is comparable to that of a pure PriorityBlockingQueue when used in a similar fashion.
- It's available on GitHub, so feel free to poke around. Any constructive feedback is always welcome.
I spend a lot of my time working with Oracle Coherence. If you've never heard of Coherence it can briefly be described as a linearly scalable in-memory HashMap. By linearly scalable I mean a distributed HashMap, where each cluster member is responsible for storing a portion of the complete map. As everything is in-memory you maintain data availability by ensuring there are nodes in the cluster whom also contain a backup for the primary partition. So should a node die, one or more backups are readily available to become primary storage locations.
So as you add nodes you increase the total available storage space, with each node serving a smaler fraction of the total data content, thereby ensuring linear and horisontal scalability for getting and putting data. But getting and putting data is just a simple little use case for Coherence. Instead I find it more interesting to focus on it's parallel aggregation capabilities. As each node owns fractions of the total data population you're able to write aggregation logic which, if fanned out, allow each node to perform a partial aggregation on only its data fraction.
A few days back I was browsing my tweets when I ran across this one from @news.yc:
"Oh, this might be interesting", I though to myself. So I clicked the link and started reading..
"This post will detail what I did to make approx. 500k from high frequency trading from 2009 to 2010." That's the first line of the post.. looks promising, doesn't it?