Dingbat Kabuki and other structural market hacks

When I first started puppetmaster trading, one of my dearest friends, a Yale-educated economist and professor of same, asked me an important question.  He asked:

In the markets, there are always ‘insiders’ who have the ability to trade on knowledge that you can’t know or with an advantage that you can’t have.  How are you going to compete with these players?

I provided a variety of answers, but at the time my conception of the universe of people with both inside knowledge and the ability to trade on it was limited to cases like that of Mr Rajaratnam.  I believed that cases like these were constrained by clear laws that were duly surveilled and prosecuted by the appropriate authorities.  The problem seemed like a very real one, but constrained in size and not essential to my enterprise.  I still hope that my belief of the time was true, but since then I’ve certainly understood that there’s more than one way to hack the market.

For some, a market hack might consist of some kind of simple (or complex) algorithm(s) applied to some set of markets.  But this really isn’t a hack so much as it’s a trading strategy – like many that have long existed – only that it’s now implemented in software where originally it would have been implemented in wetware.  While implementing trading strategies in software does open up new vistas in terms of the kinds of strategies that you can look to implement – computers are faster than people by a noteworthy amount in many tasks – but, for the most part, you’re really still just trading and when you take on positions, you are still bearing risk.  You might be ‘hacking’ but it’s really not a market hack as I’ve come to appreciate.

hack the market structure

Another area where my perspective has changed substantively since those halcyon days of ‘05 is my appreciation for market (micro) structure.  In futures, market structure is pretty plain as most contracts are effectively monopolies run by their listing exchanges.  There are a few cases of instruments which are tradable across markets, but a rich market microstructure is just not a core identifying characteristic for futures like it is in the fragmented and incredibly dynamic and quickly developing world of equities.  So, while there are some dozens of futures exchanges scattered about the globe, as a futures trader, routing algorithms just don’t enter the picture except in limited and relatively simple cases.

If I have a view on or a hedging need for interest rates that I see effectively expressed through futures, then I go to e.g. the cme and I don’t need to worry my pretty head about getting a great execution because the exchange is in one place and it’s all lit up and filled to the brim with liquidity.  Easy peasy.  But if I have a view on, say, General Electric that I want to express with an equity position and I’m trading in sufficient size or sufficiently close to the market that I really want to ensure best execution, then I might find the need to look around some dozens of lit exchanges and maybe even ping about in a dark pool or twelve.  This is decidedly not easy peasy and is one of the reasons wall st has an insatiable hunger for propeller heads with advanced degrees in seemingly unrelated fields.

Last summer’s brouhaha about ‘flash’ orders first illuminated for many a rich ground for genuine market hacks: the incredibly dynamic terrain of equity market micro structure which changes almost daily with the emergence of new exchanges, ATSes, order types, rebates and pricing structures and all of the many other critical minutia that differentiate the many venues.  Some of these undoubtedly provide meaningful and important services; contrary to populist inclination, dark pools are largely a defense against frothy HF trading strategies.

But the concern (if perhaps not all of the attention) is merited.   Equity microstructural ‘rules’ change so quickly that even independent and upright regulators could hardly be expected to keep up.  This is clearly fertile ground for genuine – if perhaps fleeting – market hacks.

steganography and hacking the market

What characteristics make equity market structure a fertile ground for these kinds of genuine market hacks which I’m describing?  I think the main answer is to be found in the ancient art of steganography – hiding in plain site.  Everything about equity market microstructure is public.  All of the rules for each of the venues are available to the people who might make use of them.  Unraveling what they imply about where you should be executing for any given order is the tricky bit which is hardly revealed through a simple reading of all of the various rules.

Where else do we see this phenomenon?  The US Tax code comes to mind (sort of like the iPhone: “there’s a loophole for that”).  The 2,000 page health care bill comes to mind.  Everybody seems to care about health care, but who’s actually read that bill?  Who could?

Totally public yet it may as well be encrypted.  Steganography.

Elizabeth Warren has also made this observation in the context of contracts made between banks and their retail clientele.  Noted scholar Scott Adams dubbed it a ‘confusopoly.’

a group of companies with similar products who intentionally confuse customers instead of competing on price

So, could people hack the tax code?  Hank and I think so.  So do my friends in the lobbying business.  How about all those recently converted bank holding companies – hack much?  In my last post we read one insider’s view that Madoff had effectively hacked the SEC… perhaps regulatory organizations themselves are also hackable instruments!  How about that health care bill?  …

misdirecting the hack

A lot of attention has focused recently on exchange traded markets and people are up in arms about HF traders and proprietary trading.  I’ve argued before that this appears to be a great ploy to take attention away from the real issues at hand; in the worst case, HFT improprieties might account for no more than 1% of the money disappeared in the last few years as part of the so-called ‘credit crisis’.  It’s easy for non-finance-professionals not to understand that the big bad wolves of the credit crisis essentially all happened off market in essentially unregulated multi-trillion dollar otc markets of ingeniously engineered structured products. These are the real market hackers.  Exchange traded instruments had effectively nothing to do with our current circumstances but remain a convenient scapegoat.

Some argue persuasively that there are fundamental, structural market hacks at the very foundations of our financial system.  I wouldn’t know.  But it is the kind of thing one might think about while watching the highly stylized performance of our leaders last night.

dingbat kabuki and the transparent market hack

To my knowledge, Cal’s (go bears!) Professor Brad DeLong coined the term ‘dingbat kabuki’ back in 2005 in shrill response to a Washington Post article.  He reprised the term this week in response to the latest bit of macroeconomic genius out of washington.  What an inspired phrase.  Yesterday at 2:30 and last night at 9pm the markets voiced their applause and cheered on the performance.

Indeed, it is masterful.

–

A technical note about this post.  While writing it, I seem to have accidentally published it at some point and then realized the error and ‘unpublished’ it.  I apologize if this had any untoward effects on you or your RSS reader.

A reader, Chris P, (following up on this post) recently inquired by email about the specific rules on Market-On-Open orders. In particular, he was interested in the cut-off times for MOO orders. Market-On-Close orders have a cut-off time of 3:40 and 3:50 on the NYSE and Nasdaq, respectively. That is, MOC orders can’t be entered, modified or cancelled after the cut-off.

He pointed me to two documents, one excellent and concise guide to the open and close from the Nasdaq and another relatively useless piece of documentation from the NYSE. The Nasdaq doc indicates that MOO orders must be entered by 9:28am and my own informal experiment confirms this. MOO orders bound for the Nasdaq after 9:28 will be rejected as invalid orders.

The NYSE doc was less precise (didn’t specify any time) and I was able to enter a MOO order after 9:28 which was filled, while orders after the 9:30 open were rejected. If any reader has more precise information or some experiences to share on the topic, kindly use the comments to do so.

I’ve been trading an increasing amount at the open and close of the equity markets using market-on-open (MOO) and market-on-close (MOC) order types and have found that the quality of executions varies enormously between the two types and have spent a bit of time analyzing the differences which I share below.

The quick scoop is that MOC orders almost invariably fill at the exchange’s published closing price, while MOOs vary very substantially from the published open price. Below I quantify my findings in a bit greater depth.

I looked at 846 recent MOO and MOC equity trades made over the past two months. Of all of these trades, only one MOC trade didn’t execute at the published close and the price I got was only off by one penny. Across all of the trades, I received the open or close price 55% of the time.

The remaining 45% of the time I varied from the open by an average of +.04% This means that I actually saw a slight price improvement in the average case. That is, if I was shorting then I executed at a price above the listed open and vice-versa for longs. I’ll take it!

In the below chart I plot the trades against their variance, positive or negative, from the listed open or close.

The biggest difference was a whopping 8.56% but at least it went in my favor. The stdev across all of the trades was .87% so we’re not looking at too disperse a grouping.

This data is a bit skewed as the majority of the MOO orders are going short. This is also a pretty limited universe of trades, so I’ll continue to look at the execution quality I’m getting on these order types and will revisit it if I see any interesting changes.

My interpretation is that my broker is making a best-effort to get a fair open price and they’re doing a creditable job of it. The exchanges are doing a nearly perfect job with MOC orders.

What impact does this have on my strategies? I’m not sure yet, but my first blush impression is that it might be worthwhile to try to get some price improvement over the posted open price as a means of both improving the results and extending the capacity of such strategies. It’s a favorable result as it means that strategies which back-test well on open/close data have a pretty good chance of executing well in reality.

A related issue, which I’m still researching, concerns the capacity of such strategies and may be the topic of a future post…

While doing some research on the quality and volume of executions at the open and close of US equity markets, I came across two topical research reports by Celent, a finance consultancy. The first report is a detailed look at execution quality on nasdaq issues while the second addresses the same topic for the nyse. An abstract of the first report can be found here and of the second here. Both are interesting enough on their own, though I’ve yet to acquire the full reports.

Apart the story told by these two graphics, the articles highlight a few further trends. One is that execution quality is dramatically improving across the board in terms of both speed and price; since 2006, NYSE execution speeds have dropped a remarkable 92%. At the same time, the difference in price quality across venues has diverged, such that everyone will get you filled faster and better than before, but some do so with much more consistently good prices.

If I can get a hold of the original reports, I’ll write a more detailed review of their contents, but for now I just include the links, the catchy speed-charts and these few high-points. For shops dealing with smart order routing algorithms, the reports might be well worth purchasing and studying directly.