Hack the market

you, hf-trading

There seems to be a developing meme out there suggesting that algorithmic-, and in particular high-frequency, trading is some kind of gold-rush route to easy money which brings to mind…

…this revision of a paper I’d read previously: “Statistical Arbitrage in the US Equities Market” by Avellaneda and Lee.   It’s a detailed and thoroughly worked (and now re-worked) paper illustrating the development and analysis of a US equity stat-arb strategy based on Principal Component Analysis (PCA) and then revised to use ETFs.

I came across this paper as I have still never used PCA in any of my own strategy development work and read Carol Alexander’s excellent Market Models over my summer vacation with an eye towards giving a PCA hedging model a spin in the near-term. Thus, I wanted another look at this paper as a reference point.  Although it’s an excellent paper, I’m not going to urge you to go out and read it immediately unless you have a reasonably pressing practical interest.  Instead, I find it interesting largely because of one of its authors – Professor Avellaneda – and its conclusions in the form of its strategies’ performance.

I’ve seen Prof Avellaneda speak a number of times at a variety of quant meetups organized by the relevant Columbia/NYU financial engineering depts.  His paper reminds me that at least once during my noisome adolescent years, my father intoned darkly that:

the streets are littered with brilliant minds

Implying that any wits I may believe myself to possess wouldn’t by themselves be worth much in life and that I’d need to bring actual tools to the task of solving problems if I wanted to address interesting ones.  Having seen Mr Avellaneda speak, I’m confident that at my peak, my “processor” was never as fast as his.  Much worse, there is no comparison between the tools he can level at a problem compared to me – he’s on an entirely different playing field so far as concrete mathematical/analytical capabilities go.  That’s why I go see him speak and read his papers.

Thus, the results of Avellaneda and Lee’s work are particularly interesting to me as they’re really pretty dull: something like a Sharpe of .9 and degrading briskly.  Now, you don’t expect people to be providing detailed recipes to wildly profitable strategies, and this result isn’t bad, particularly given that they’re describing strategies which likely have significant capacity.  Still, it illustrates that very smart people working with sophisticated mathematical tools even over extended periods are still operating under noteworthy constraints.  Perhaps also: ideas are relatively easy – examining them in the requisite detail is difficult and time consuming, even for (particularly for?) people with the most finely honed toolsets…

I frequently have friends or colleagues who will observe that if you “just write a strategy that foos when bar but yaddas when baz… you should surely make money.”  Maybe.  But the reality is that just putting together the strategy and working through it takes significant time for anything but the simplest strategies.  Once you add genuine complexity to a strategy, you can spend enormous time tuning it.

This, in turn, poses a dilemma I encounter frequently and honestly don’t have a great answer for:

how to find the balance between continuing development on a known good strategy and initiating the development and analysis of unrelated and novel strategies?

the back of the envelope as canvas

This next (de-)reference isn’t directly pertinent to algo trading, but the lessons learned by building BIG distributed systems can surely be applied elsewhere.  And they’re just plain fascinating.

Google’s Jeff Dean gave a recent talk entitled “Designs, Lessons and Advice from Building Large Distributed Systems” at the Large Scale Distributed Systems and Middleware (somehow “LADIS”) workshop and the slides are here.   Go read them.

If bald exhortation doesn’t convince you maybe slide 24 will:

or perhaps what he does with these baseline numbers in slide 27 will pique your interest:

back of the envelope as art form

One that made me (a serial prototype-builder) cringe:

Important skill: ability to estimate performance of a system design
– without actually having to build it!

Ouch.

maybe it is easy after all

Of course, if you’ve studied Avellaneda & Lee’s paper and it held no challenges or surprises and you’ve reviewed Mr Dean’s presentation and it’s old hat to you, too…