photo credit: Jochen Abitz @flickr

This past spring I was compelled to rejoin what one of my former partners had longingly referred to as “civilization.” The process of rejoining the civilized was itself of note in an environment so changed as to be unrecognizable, but I’ll skip that for now.  Instead I have some observations on the interesting spot in which I’ve found myself: writing dark pool aware algos from the inside.  That is, I’m working for a block trading ‘dark pool’ working on the team that develops their quantitative strategies.

While still within the world of algorithmic trading, this is a substantial change from what I’d been doing before and has proven a rich ground for learning, in particular about market structure.  The biggest aspect of the change – besides being civilized – is the change of perspective from the prop trader to, effectively, an execution trader.  As a prop trader you are looking to identify and execute trading opportunities.  Seeking alpha.  Instead, as an execution trader you receive orders and need to execute them with some highly customized sets of constraints.  You want to get things done over some time frame with some appropriate balance of aggressiveness and stealth.  Liquidity seeking.  The ‘what’ has already been decided for you; it’s the ‘how’ you need to worry about.  Thus, there’s some loss of ‘agency’ in going from the former role to the latter and this corresponds precisely and inversely with the notion of agency trading.

Going from alpha-seeking to seeking-liquidity is a change of perspective, but the blocking and tackling are constant.  In the end, you’re trading – managing orders and positions and deluges of market data and analytics; familiar, fun stuff.

What I’ve found most interesting is the new perspective I’m afforded on market structures.

the default structure of a market is a social network

My first job on wall st was on a muni desk in the mid 90′s; there was no exchange and market structure was effectively a social network.  Since then, I’ve always worked around exchange-traded instruments: futures, equities, and options on either.  Exchanges have a lot to be said for them.  We’ve seen the results of too much creativity and too little oversight in the magnificently free world away from exchanges.  But exchange-traded instruments haven’t been standing still.  While futures markets have exploded volume-wise and imploded as players have collapsed into one another through consolidations, US equity markets have fragmented pretty wildly.  This equity fragmentation has been especially interesting for its effects on market structure alongside the parallel ‘rise of the machines’ in the form of high-frequency trading.

Equity markets have fragmented in accordance with Reg ATS which provides guidelines on how Alternative Trading Systems (ATS) must behave in order to stay in good standing with the relevant authorities.  While the number of exchanges has grown slowly, ATSes have risen and fallen like city eateries over the past decade.  The interesting thing about Reg ATS is that while it covers lots of requirements, it also allows quite a bit of latitude in the determination of the microstructure of the ATS.

market structure machines

One, perhaps fanciful, view of dark pools is that they collectively represent a crucible in which market structures are being actively mutated and evolved.  Some experiments yield value, meet a real need and are adopted and copied while others fizzle out or perhaps even look to skirt the rules a bit.  Outside of the pools, you have all sorts of market participants always eager to glean any advantage they might from any information leakage – intentional or otherwise – from within the dynamic ATS ecology.  This is where gaming can come into play.  But, according to me, the most interesting game is played-out in the making of the very definitions of the pools themselves and how they interact with external parties.

Here again, we see the fundamental distinction between roles that trading entities can play.  ATSes are commercial entities; just as any broker dealer can determine if it will do principal trading or remain purely agency, ATSes can decide how they will seek to bring volumes to their venue and they can adjust their offerings at any time.  Keeping up with these changing microstructures can be a full-time job and handily explains the existence of  so-called ‘Smart’ Order Routers (SORs), dark ‘aggregation’ algos and such offerings as Rosenblatt’s excellent Let There Be Light (LTBL) series.  If everybody is constantly changing their prices and adjusting their rules, it’s very non-trivial to figure out where something can best get done at any given moment in time.

It’s possible and illuminating to consider a venue itself as something of a trade (“value proposition”) wherein the venue is trying to add value to the overall marketplace and/or their own flow trading operations.  A market-maker or high-frequency shop might require a steady diet of deliciously uninformed retail order flow to manage their inventories.  A firm which possessed such a flow might trade against it themselves (“internalization”), might “cross” it against other client orders and might “preference” it to other interested parties like our market-maker.  They might do some combination of all three.

Thus, based on their motivations and goals, dark pools will self-organize in a multitude of ways as they try to meet these needs and satisfy their target constituencies.  Dark pools arose to meet a number of needs amongst financial market participants, including block trading, flow trading, utility and branding & marketing purposes.  Block trading has been around forever and used to be a point-to-point social network wherein brokers simply knew which brokers specialized in blocks or might happen to have a relevant block handy.  A lot like the old muni desk.  This model was replaced by the current pioneers of ‘dark pool’ block trading venues.  But by far the biggest dark pools from a volume perspective are those created by, yes, market-makers or hft shops for internalization purposes and large broker dealers for internalization and crossing purposes: flow trading.

A telling measure of the nature of a given venue is its average trade size.  Large sizes are done by block traders, whilst most flow is done in smaller sizes.  Some dark pools have average trade sizes that are even smaller than those on the ‘lit’ exchanges.

structural topologies, too

Given the different motivations of the various constituencies providing and utilizing dark venues, it’s clear that to really understand a pool’s character, one must understand the motivations of those providing it.  You can change the microstructure in any number of different ways, but in the end a market remains fundamentally a social network.

One final aspect of market structure which I’m only now coming to appreciate for its full worth is the topology of market structures.  When you view the flows of orders, etc from the perspective of a venue, routing becomes a surprisingly significant part of your considerations.  When you add-in the fact that there are many inter-relations amongst all these flows, it changes the nature of markets themselves.  You can wind up with all sorts of convoluted situations stemming from ‘topological’ considerations.  Venue A doesn’t take traffic from B, but does from C and E; B wants to get to A and has a relationship with E.  Is there now a route from B to E?  It’s complicated and it’s an area in which it’s hard to find published research.  After all, it is dark.

into a pool, darkly

Plunging into this environment has been quite the fun dunk as all of the aforementioned acts as the background against which my algos must now play.  This has fundamentally enriched my view of the equity markets.  Apart market structure considerations, there have been other lessons as well.

The problem of trading in size is one that I simply hadn’t had to worry about before.  That is, when sizes are denoted as significant percentages (or multiples!) of a given name’s Average Daily Volume (ADV), trading takes on a very different character.  When you’re looking to trade in size in this sense, even your analytical interests will change.   Never before have historical volume curves looked so meaningful!

Another nicety has been that algorithmic ‘execution-quality’ trading, as a practice, is vastly better documented than the alpha-seeking variety, so it’s possible to learn from a very richly developed, quantitatively rigorous, body of knowledge and publication.  From a learning perspective, I’ve dropped into a target-rich environment.

“Y’know, watching government regulators trying to keep up with the world is my favorite sport.” – character L. Bob Rife in Neal Stephenson’s Snow Crash.

From a regulatory perspective also, it’s an interesting environment. As I’ve argued before (notably – before I started working for a dark pool), while these are areas that need to be regulated, they are not even close to the real sources of today’s serious financial issues.  Thus, much of the very animated talk about microstructure – principally regarding HFTs and dark pools is at best misguided and probably diversionary.

I’m finding the dark illuminating.

–

The opinions expressed here may or may not be my own by the time you read them, but are most certainly NOT those of my employer.

Dingbat Kabuki and other structural market hacks

When I first started puppetmaster trading, one of my dearest friends, a Yale-educated economist and professor of same, asked me an important question.  He asked:

In the markets, there are always ‘insiders’ who have the ability to trade on knowledge that you can’t know or with an advantage that you can’t have.  How are you going to compete with these players?

I provided a variety of answers, but at the time my conception of the universe of people with both inside knowledge and the ability to trade on it was limited to cases like that of Mr Rajaratnam.  I believed that cases like these were constrained by clear laws that were duly surveilled and prosecuted by the appropriate authorities.  The problem seemed like a very real one, but constrained in size and not essential to my enterprise.  I still hope that my belief of the time was true, but since then I’ve certainly understood that there’s more than one way to hack the market.

For some, a market hack might consist of some kind of simple (or complex) algorithm(s) applied to some set of markets.  But this really isn’t a hack so much as it’s a trading strategy – like many that have long existed – only that it’s now implemented in software where originally it would have been implemented in wetware.  While implementing trading strategies in software does open up new vistas in terms of the kinds of strategies that you can look to implement – computers are faster than people by a noteworthy amount in many tasks – but, for the most part, you’re really still just trading and when you take on positions, you are still bearing risk.  You might be ‘hacking’ but it’s really not a market hack as I’ve come to appreciate.

hack the market structure

Another area where my perspective has changed substantively since those halcyon days of ’05 is my appreciation for market (micro) structure.  In futures, market structure is pretty plain as most contracts are effectively monopolies run by their listing exchanges.  There are a few cases of instruments which are tradable across markets, but a rich market microstructure is just not a core identifying characteristic for futures like it is in the fragmented and incredibly dynamic and quickly developing world of equities.  So, while there are some dozens of futures exchanges scattered about the globe, as a futures trader, routing algorithms just don’t enter the picture except in limited and relatively simple cases.

If I have a view on or a hedging need for interest rates that I see effectively expressed through futures, then I go to e.g. the cme and I don’t need to worry my pretty head about getting a great execution because the exchange is in one place and it’s all lit up and filled to the brim with liquidity.  Easy peasy.  But if I have a view on, say, General Electric that I want to express with an equity position and I’m trading in sufficient size or sufficiently close to the market that I really want to ensure best execution, then I might find the need to look around some dozens of lit exchanges and maybe even ping about in a dark pool or twelve.  This is decidedly not easy peasy and is one of the reasons wall st has an insatiable hunger for propeller heads with advanced degrees in seemingly unrelated fields.

Last summer’s brouhaha about ‘flash’ orders first illuminated for many a rich ground for genuine market hacks: the incredibly dynamic terrain of equity market micro structure which changes almost daily with the emergence of new exchanges, ATSes, order types, rebates and pricing structures and all of the many other critical minutia that differentiate the many venues.  Some of these undoubtedly provide meaningful and important services; contrary to populist inclination, dark pools are largely a defense against frothy HF trading strategies.

But the concern (if perhaps not all of the attention) is merited.   Equity microstructural ‘rules’ change so quickly that even independent and upright regulators could hardly be expected to keep up.  This is clearly fertile ground for genuine – if perhaps fleeting – market hacks.

steganography and hacking the market

What characteristics make equity market structure a fertile ground for these kinds of genuine market hacks which I’m describing?  I think the main answer is to be found in the ancient art of steganography – hiding in plain site.  Everything about equity market microstructure is public.  All of the rules for each of the venues are available to the people who might make use of them.  Unraveling what they imply about where you should be executing for any given order is the tricky bit which is hardly revealed through a simple reading of all of the various rules.

Where else do we see this phenomenon?  The US Tax code comes to mind (sort of like the iPhone: “there’s a loophole for that”).  The 2,000 page health care bill comes to mind.  Everybody seems to care about health care, but who’s actually read that bill?  Who could?

Totally public yet it may as well be encrypted.  Steganography.

Elizabeth Warren has also made this observation in the context of contracts made between banks and their retail clientele.  Noted scholar Scott Adams dubbed it a ‘confusopoly.’

a group of companies with similar products who intentionally confuse customers instead of competing on price

So, could people hack the tax code?  Hank and I think so.  So do my friends in the lobbying business.  How about all those recently converted bank holding companies – hack much?  In my last post we read one insider’s view that Madoff had effectively hacked the SEC… perhaps regulatory organizations themselves are also hackable instruments!  How about that health care bill?  …

misdirecting the hack

A lot of attention has focused recently on exchange traded markets and people are up in arms about HF traders and proprietary trading.  I’ve argued before that this appears to be a great ploy to take attention away from the real issues at hand; in the worst case, HFT improprieties might account for no more than 1% of the money disappeared in the last few years as part of the so-called ‘credit crisis’.  It’s easy for non-finance-professionals not to understand that the big bad wolves of the credit crisis essentially all happened off market in essentially unregulated multi-trillion dollar otc markets of ingeniously engineered structured products. These are the real market hackers.  Exchange traded instruments had effectively nothing to do with our current circumstances but remain a convenient scapegoat.

Some argue persuasively that there are fundamental, structural market hacks at the very foundations of our financial system.  I wouldn’t know.  But it is the kind of thing one might think about while watching the highly stylized performance of our leaders last night.

dingbat kabuki and the transparent market hack

To my knowledge, Cal’s (go bears!) Professor Brad DeLong coined the term ‘dingbat kabuki’ back in 2005 in shrill response to a Washington Post article.  He reprised the term this week in response to the latest bit of macroeconomic genius out of washington.  What an inspired phrase.  Yesterday at 2:30 and last night at 9pm the markets voiced their applause and cheered on the performance.

Indeed, it is masterful.

–

A technical note about this post.  While writing it, I seem to have accidentally published it at some point and then realized the error and ‘unpublished’ it.  I apologize if this had any untoward effects on you or your RSS reader.