our solid-state future
Mmmm... hardware..
I’ve never been a hardware guy. Hardware has gotten so fast throughout my professional life that it has just never been a big issue. Also, on wall st we had a robust and annual budget for h/w so I’d routinely sign-off on hundreds of thousands of dollars on all sorts of machines I’d never lay eyes on and somehow they always did the trick.
Before 9/11, they’d be in server racks in the building or down the street, but since then they might also be in increasingly far-flung places like weehawken or long island, tampa, even texas or beyond. The machines always seemed unbelievably overpriced – I remember over the years pretty consistently paying something like $40K for a low-end db server. But that’s what it cost and you could only purchase approved products from approved channels, so nobody spent much thought on it. Now that I don’t have the same kinds of constraints – or budgets! – I increasingly have to think of hardware.
As a software engineer, the hardware itself is also insisting that I pay some uncharacteristic attention to it. The evolution of processors has reached a point where the programming paradigms many of us have fruitfully employed over many years are no longer suited for getting full performance out of today’s machines. The recent introduction of remarkably powerful and inexpensive parallel-computing platforms based on GPUs like nvidia’s cuda also outline a future that even current university training doesn’t address in a fashion practically adapted for institutional application. Cores are multiplying like Tribbles.
The lines between persistent storage and main memory are also blurring as consumer SSDs push up from the ‘low’-end while exotic ioDrives and the like offer a glimpse of a world where the performance gap between the two approaches nil and after their long reign myriad metallic platters will spin no more.
troubling like Tribbles
There have been some steps taken towards taming the core dilemma. Google’s introduction of the distributed map-reduce paradigm and all of the associated plumbing on top of computers in the ‘cloud’ is probably the boldest and most effective reaction thus far, but it’s not always obvious that you want your stuff running in someone else’s cloud amongst many other natural limitations of this approach. This is also a solution ‘in the large’ and sometimes you need performant solutions on a different, smaller scale. Here, the development of functional languages and idioms may be of some help, but there certainly don’t seem to be clear winners yet.
Erlang, Ocaml, Haskell, Scala and others all seem to have very limited impact thus far and all face big challenges before receiving widespread welcome. Worse, any language can be mangled into expressing things poorly so the languages can only be a meaningful aid to programmers who are able to adjust their mindset for a new world’s computing paradigm. This likely won’t be easy for many until there is an established set of usable programming idioms and toolsets for dealing with concurrency on a whole new scale. To me, it seems that functional programming might well be an important part of that, but it’s difficult to imagine it as the complete game changer any time soon. As it is, many of us have already been using functional languages (in my case, sql and R) on a regular or even daily basis for a long time, so it’s difficult to cast this as the revolution, bottled.
As cores proliferate and the bandwidth amongst them increases, new challenges and opportunities are unveiled. Feeding all of those hungry cores can be a chore. If you already had problems with i/o bound processes, then adding cores is sort of like adding liquidity to a debt crisis – not obviously helpful. We faced an issue like this recently while trying to improve the throughput of our backtesting subsystem in particularly poor-performing cases.
In these cases, we use daily data to initialize an intraday strategy. For example, we might look at all equities over a trailing 3 or 5-year period, and perform various analytics – like calculating correlation matrices, beta against various benchmarks, volatility, etc – to determine which names might be candidates for the strategy’s portfolio. We found that an inordinate amount of our time was spent just performing these morning analytics and that the cost of a day’s backtest was significantly spent on this oft-repeated morning exercise. While working through this issue, we noted that we’ve reached a point where a decent, dual nehalem, server (with 16 logical cores) could be built with 48G of RAM for something along the lines of $6K. Seriously. So we stuck all of our daily data into memory and the improvement has been essentially infinite. Maybe not the best example, but hopefully illustrative of the fact that the h/w underneath us is changing qualitatively and that we need to be more active in involving it in our design decisions.
Of course, even a yawning expanse like 48G or even a terabyte (in not too many years, after all, I should be able to buy 1T of RAM for my boxes at a similar price point to what I pay for 48G today), eventually gets consumed and so we can’t hope to employ this solution for all our problems. We continue to develop our historical TAQ infrastructure (most recently discussed here) and this is certainly an example where buying memory isn’t going to get you very far. But SSDs now are getting reasonable and so our current approach uses memory as much as possible but when it needs to get volumes of detailed historical data we’ve placed our indices onto SSDs while the actual stores themselves reside on much more affordable RAID arrays. Reading indices is now much faster and adds minimal i/o overhead to a very i/o-bound problem.
By constantly looking at new languages and concurrency idioms, vigilantly assessing the current and projected costs of ’solid-state’-solutions to our most vexing problems, and just staying active and creative, we hold out some hope that we can transition gracefully to what is looking increasingly like a solid-state future.
