photo credit: Jochen Abitz @flickr

This past spring I was compelled to rejoin what one of my former partners had longingly referred to as “civilization.” The process of rejoining the civilized was itself of note in an environment so changed as to be unrecognizable, but I’ll skip that for now.  Instead I have some observations on the interesting spot in which I’ve found myself: writing dark pool aware algos from the inside.  That is, I’m working for a block trading ‘dark pool’ working on the team that develops their quantitative strategies.

While still within the world of algorithmic trading, this is a substantial change from what I’d been doing before and has proven a rich ground for learning, in particular about market structure.  The biggest aspect of the change – besides being civilized – is the change of perspective from the prop trader to, effectively, an execution trader.  As a prop trader you are looking to identify and execute trading opportunities.  Seeking alpha.  Instead, as an execution trader you receive orders and need to execute them with some highly customized sets of constraints.  You want to get things done over some time frame with some appropriate balance of aggressiveness and stealth.  Liquidity seeking.  The ‘what’ has already been decided for you; it’s the ‘how’ you need to worry about.  Thus, there’s some loss of ‘agency’ in going from the former role to the latter and this corresponds precisely and inversely with the notion of agency trading.

Going from alpha-seeking to seeking-liquidity is a change of perspective, but the blocking and tackling are constant.  In the end, you’re trading – managing orders and positions and deluges of market data and analytics; familiar, fun stuff.

What I’ve found most interesting is the new perspective I’m afforded on market structures.

the default structure of a market is a social network

My first job on wall st was on a muni desk in the mid 90′s; there was no exchange and market structure was effectively a social network.  Since then, I’ve always worked around exchange-traded instruments: futures, equities, and options on either.  Exchanges have a lot to be said for them.  We’ve seen the results of too much creativity and too little oversight in the magnificently free world away from exchanges.  But exchange-traded instruments haven’t been standing still.  While futures markets have exploded volume-wise and imploded as players have collapsed into one another through consolidations, US equity markets have fragmented pretty wildly.  This equity fragmentation has been especially interesting for its effects on market structure alongside the parallel ‘rise of the machines’ in the form of high-frequency trading.

Equity markets have fragmented in accordance with Reg ATS which provides guidelines on how Alternative Trading Systems (ATS) must behave in order to stay in good standing with the relevant authorities.  While the number of exchanges has grown slowly, ATSes have risen and fallen like city eateries over the past decade.  The interesting thing about Reg ATS is that while it covers lots of requirements, it also allows quite a bit of latitude in the determination of the microstructure of the ATS.

market structure machines

One, perhaps fanciful, view of dark pools is that they collectively represent a crucible in which market structures are being actively mutated and evolved.  Some experiments yield value, meet a real need and are adopted and copied while others fizzle out or perhaps even look to skirt the rules a bit.  Outside of the pools, you have all sorts of market participants always eager to glean any advantage they might from any information leakage – intentional or otherwise – from within the dynamic ATS ecology.  This is where gaming can come into play.  But, according to me, the most interesting game is played-out in the making of the very definitions of the pools themselves and how they interact with external parties.

Here again, we see the fundamental distinction between roles that trading entities can play.  ATSes are commercial entities; just as any broker dealer can determine if it will do principal trading or remain purely agency, ATSes can decide how they will seek to bring volumes to their venue and they can adjust their offerings at any time.  Keeping up with these changing microstructures can be a full-time job and handily explains the existence of  so-called ‘Smart’ Order Routers (SORs), dark ‘aggregation’ algos and such offerings as Rosenblatt’s excellent Let There Be Light (LTBL) series.  If everybody is constantly changing their prices and adjusting their rules, it’s very non-trivial to figure out where something can best get done at any given moment in time.

It’s possible and illuminating to consider a venue itself as something of a trade (“value proposition”) wherein the venue is trying to add value to the overall marketplace and/or their own flow trading operations.  A market-maker or high-frequency shop might require a steady diet of deliciously uninformed retail order flow to manage their inventories.  A firm which possessed such a flow might trade against it themselves (“internalization”), might “cross” it against other client orders and might “preference” it to other interested parties like our market-maker.  They might do some combination of all three.

Thus, based on their motivations and goals, dark pools will self-organize in a multitude of ways as they try to meet these needs and satisfy their target constituencies.  Dark pools arose to meet a number of needs amongst financial market participants, including block trading, flow trading, utility and branding & marketing purposes.  Block trading has been around forever and used to be a point-to-point social network wherein brokers simply knew which brokers specialized in blocks or might happen to have a relevant block handy.  A lot like the old muni desk.  This model was replaced by the current pioneers of ‘dark pool’ block trading venues.  But by far the biggest dark pools from a volume perspective are those created by, yes, market-makers or hft shops for internalization purposes and large broker dealers for internalization and crossing purposes: flow trading.

A telling measure of the nature of a given venue is its average trade size.  Large sizes are done by block traders, whilst most flow is done in smaller sizes.  Some dark pools have average trade sizes that are even smaller than those on the ‘lit’ exchanges.

structural topologies, too

Given the different motivations of the various constituencies providing and utilizing dark venues, it’s clear that to really understand a pool’s character, one must understand the motivations of those providing it.  You can change the microstructure in any number of different ways, but in the end a market remains fundamentally a social network.

One final aspect of market structure which I’m only now coming to appreciate for its full worth is the topology of market structures.  When you view the flows of orders, etc from the perspective of a venue, routing becomes a surprisingly significant part of your considerations.  When you add-in the fact that there are many inter-relations amongst all these flows, it changes the nature of markets themselves.  You can wind up with all sorts of convoluted situations stemming from ‘topological’ considerations.  Venue A doesn’t take traffic from B, but does from C and E; B wants to get to A and has a relationship with E.  Is there now a route from B to E?  It’s complicated and it’s an area in which it’s hard to find published research.  After all, it is dark.

into a pool, darkly

Plunging into this environment has been quite the fun dunk as all of the aforementioned acts as the background against which my algos must now play.  This has fundamentally enriched my view of the equity markets.  Apart market structure considerations, there have been other lessons as well.

The problem of trading in size is one that I simply hadn’t had to worry about before.  That is, when sizes are denoted as significant percentages (or multiples!) of a given name’s Average Daily Volume (ADV), trading takes on a very different character.  When you’re looking to trade in size in this sense, even your analytical interests will change.   Never before have historical volume curves looked so meaningful!

Another nicety has been that algorithmic ‘execution-quality’ trading, as a practice, is vastly better documented than the alpha-seeking variety, so it’s possible to learn from a very richly developed, quantitatively rigorous, body of knowledge and publication.  From a learning perspective, I’ve dropped into a target-rich environment.

“Y’know, watching government regulators trying to keep up with the world is my favorite sport.” – character L. Bob Rife in Neal Stephenson’s Snow Crash.

From a regulatory perspective also, it’s an interesting environment. As I’ve argued before (notably – before I started working for a dark pool), while these are areas that need to be regulated, they are not even close to the real sources of today’s serious financial issues.  Thus, much of the very animated talk about microstructure – principally regarding HFTs and dark pools is at best misguided and probably diversionary.

I’m finding the dark illuminating.

–

The opinions expressed here may or may not be my own by the time you read them, but are most certainly NOT those of my employer.

or: scaling with elastic map-reduce

Between a rapidly evolving compute environment in which cores are multiplying like springtime rabbits, and a business domain in which the fecundity of market data is making those same rabbits look downright prudish, we are always looking for ways to scale our efforts.  There are three levels at which this can be typically done:

• “below the processor” with things like CUDA or FPGAs,
• “amongst the cores” with things like TBB, Cilk, &tc, or
• “in the cloud” (or grid) with things likes Amazon’s EC2 and its bewildering and fast-growing coterie of associated technologies and products

Today we’ll look at a simple example which we implement on top of Hadoop and then deploy into Amazon’s cloud to get a back-of-the envelope feel for what kind of scaling we might expect to gain – and at what costs – from using this smorgasboard of technologies.

‘smorgasboard’ is being generous

Probably the hardest thing whenever you look at some new set of technologies is just getting your bearings.  Amazon’s stack is complex and very much of a moving target, so I’ll start by providing a bit of a taxonomy of the beasties we’ll need to at least familiarize ourselves with in order to get anything done.

First, there’s the family of Amazon’s Web Services (AWS).  This has been around for a good few years, so some parts of it are quite mature and well documented and understood where others are bleeding edge with an emphasis on bleeding.  All of them can be played with quite easily by simply signing up (this involves providing a credit card, so head’s up).  I can’t think of any more interesting family of technologies, so if you think there’s any possibility these technologies might help you scale, then I encourage you to sign up and give it a spin.  Very interesting stuff.

Anyway, within AWS we’ll play with Elastic Compute Cloud (EC2), Elastic MapReduce (EMR), Simple Storage Service (S3) and Elastic Block Storage (EBS).

EC2 provides you with direct access to Amazon’s racks of humming servers in the cloud.  Thanks to the miracles of modern virtualization technologies, an entire machine can be bundled up as an image file – an Amazon Machine Image (AMI – think of an OS’s DVD but with software that you specify, configured as you please). Once bundled into an AMI, your computer can be ‘installed’ onto any number of machines within the cloud.  Nifty.

Such machines can be bought for long term use (at quite affordable rates), or they can be used on an as-needed basis for an hourly charge.  Hence the ‘Elastic’ in EC2.  They can also be managed programmatically, and until recently, this was pretty much the only way to develop jobs in the cloud.

Each machine that you use in EC2 has some local storage associated with it, but this is not persistent across sessions.  Once your instance goes down, the data is all gone.  To get around this and to provide an all-around storage solution, AWS provides S3 which is, to me, an overly-simplified distributed file system which is something of a pain to use.  It is also limited to file sizes of 5G which I find to be a significant annoyance.  The good news is that S3 seems to be both highly durable and reasonably performant.

A more recent entry in the AWS world of storage is the EBS facility.  Here, you can create arbitrary sized “disks” in the cloud which can be attached directly to your computer nodes just as though they were local disks.  Very handy.  They don’t have the same level of durability as S3 as they’re not distributed, but I haven’t had any difficulties with them and they’re very useful for putting together AMIs within the cloud.

Finally, EMR brings the Hadoop implementation of Google’s marvelous distributed map-reduce model into the AWS cloud.  If you haven’t read about map-reduce, well perhaps you should – it’s really powerful.  Hadoop is an open-source implementation of it done in Java and seems quite nice though it’s under very active development, so the APIs are still very much a work in progress.  Be prepared to change your code.  Even though it’s written in Java, any language can be used to implement your own applications, and it seems as though both ruby and python are used pretty extensively along with java and c/c++.

what map-reduce buys you

The first thing map-reduce buys you is something of a headache as fitting your application design into the map-reduce model isn’t so obvious.  That said, it may well be worth a couple aspirin as it brings a great deal to the game.

Consider an application we might find interesting which we’d like to place in the cloud – strategy back-testing. It seems to be a very natural fit for parallel processing as each strategy is independent of the others, so theoretically we should be able to run each one on it’s own virtual server without interdependencies.  While this is true, it’s rather easier than it sounds.  In order to implement our distributed backtesting platform, we first need to implement one that works on one node.  Fair enough.  We must then introduce a “chunker” which will break the overall set of simulations into bite-sized chunks.  We must then define an intermediate format for sending the strategies (or instructions) to each node.  And we must define an intermediate format for the results.  And then we need to implement an “assembler” which coalesces all of the results back into one result set which can be reported on or displayed etc.  And if one of the nodes fails, we have to notice it.  And if we notice that a node has failed, we need to figure out which job(s) it was responsible for and reallocate them to another node which presumably hasn’t failed…

And on and on.  The point is that this isn’t something that you’re going to get done in an afternoon.  Indeed, it’s unlikely that you could write a correct and complete specification of the system in an afternoon.

What map-reduce buys you is all of the distribution bits that I mention above.  The costs are that you will have to figure out how to model things such that they’re compliant with the map-reduce model and that the end result might not be as efficient as a hand-coded solution.  But when you can scale easily and cheaply, then raw efficiency may not be such an over-arching concern.

scaling in the cloud with elastic map-reduce

In order to get an idea how difficult it is to actually scale a solution, I implemented a simple program using real data and collecting some timings across a variety of  AWS/EMR configurations. To do this, I first had to code against Hadoop v18.3 which is what is supported in AWS.  (Initially I wrote it to v20.2 and was surprised at how much I had to change to back-port it to 18.3.)

The details of my test program aren’t important – basically counting various features from a set of heterogeneous files.  The files are approximately the same size ~1.8G and there were 10 of them, so the overall dataset is a bit over 18G.  I only do one map-reduce; I’m not chaining them together.  On a local machine – a pre-nehalem, dual-chip w/ quad-cores (8 total) xeon server with hardware raid – the process took 1068s or a touch under 18 minutes.

I was impressed to see that with absolutely no code changes, I was able to run the system on EMR without difficulties.  In fact, selecting the number of nodes to run against and the size of those nodes is just a matter of changing launch parameters.  Very nice.  AWS offers a variety of different specs on their servers.  I tested the classic ‘small’ server which provides a 32-bit O/S, one core and 1.7G of memory as well as the ‘large’ server which comes with a 64bit O/S and 2X2 cores and 7.5Gs of memory.  There are other configurations available, but I didn’t test them.  In all cases I was running centos5.4.

The results are tabulated and graphed below.

performance locally and in various cloud configurations

time to complete vs. # nodes for large & small nodes

Scaling isn’t exactly linear, but it is good and it is simple and it is cheap & on-demand. Clearly for any given use-case, it will make sense to do tests like this to figure out what makes the most sense.  Another dimension to consider is cost.  Even partial hours count as a full hour, so putting big or many boxes against jobs that don’t take long to run isn’t cost effective.  That said, even though I fecklessly ran jobs which took as little as 10 minutes while paying for 60, the entire costs associated with developing and running the example I present here was less than $4US.

One thing I noticed is that the time to initialize instances seems to grow as you increase the number of nodes, though I didn’t quantify this.  In some cases, it was really quite long before real work started getting done – several minutes at least.  Getting this to work for interactive processes will take some thought and, probably, some dedicated nodes.  Another things to note is that although my local server did a fine job for itself, if we were to increase the dataset by a factor of, say 100, we’d reach a point where it simply couldn’t complete the operation whereas the cloud isn’t so strictly bounded.  It’s also pretty interesting that my local machine with 8-cores and 24G of RAM and pretty decent RAID subsystem got soundly spanked by eight of Amazon’s cheapie 1-core boxes.

Given that it is cheap, massively and simply scalable and not terribly difficult, one could say that if you don’t spend some time with your head in the clouds, then it’s possible you’ve got it stuck in the sand.

The example I’ve chosen is, I think, very nice because it’s a portfolio-oriented strategy, which is pretty much the only kind I care to explore; it’s also based around the concept of pairs trading, which is something which most can easily relate to; and finally, it’s already public domain and yet almost certainly has some juice in it for those who care to understand it and extend it intelligently.

The example comes from the blog of a company, Palantir which does (something like?) analytical / decision support software for both finance and intelligence-gathering services (quants and spooks – spooky quants?).  The specific example is here and is described thusly:

In this study we explore a trading strategy that isolates pairs of instruments within a sector that are highly correlated. We enter a trade if the price paths of these instruments diverge, going long one instrument and short the other, with the assumption that their price paths will converge.

We construct our strategy in four parts: (1) isolate the target set of tradable instruments, (2) choose a rule for finding correlated pairs within that set, (3) pick criteria for entering trades, and (4) test the trading strategy.

…

For this study we want to isolate pairs of instruments whose correlation is above a certain threshold. At the beginning of each quarter, we restrict our target stocks to the top 5 in the sector by 252-day correlation to the S&P 500. Then our Strategy uses a metric that, for a given group of stocks, outputs a list of pairs with a correlation above our target threshold of .60 over the same time period.

…

Across every day and every pair in our trading list, we check the 21-day z-score (number of standard deviations from the mean) of the difference between the two series. We classify the stocks as diverging when the absolute value of the z-score is between 1.5 and 3.0. When this condition is met we short the stock that is rising and long the stock that is falling, sizing our position relative to the z-score and the number of pairs currently trading.

~(sectors) ; reframing the strategy

While I’ll implement a strategy broadly based on their description, I will use some poetic license to modify it in ways that seem to me sensible.  First, I have good experience writing equity strategies which deal with sectors and have pretty uniformly found that sectors, as defined by S&P or whomever, are more trouble than they’re worth.  Instead, looking at correlation matrices and determining on that basis who should be paired, is much more fruitful in my experience.  Tr8der has some very nice posts illustrating (very prettily!) a similar point here and here.  (Actually, his point is a bit broader and represents an area of possibly promising development of this strategy which I won’t pursue here.)

So, we’re going to chunk the idea of explicitly using sectors.  Instead, we’ll simply construct, on a monthly basis (Palantir did this quarterly) a TrailDays correlation matrix from which we’ll extract the set of pairs which have a correlation greater than MinCorrelation.  On a daily basis, we will calculate the ZScoreDays z-scores of each of these pairs and sort them by absolute value, tossing those whose values are less than MinZScore or exceed MaxZScore.  Since I don’t like the idea of having an open-ended number of pairs/positions in my portfolio, I will constrain the portfolio to a maximum of N names and will Allocate $1M to the strategy.   Finally, we will look at the effect of trading an EvenlyWeighted portfolio or a z-score weighted portfolio (as Palantir had done).

The initial ‘universe’ of equities across which we will calculate our correlations will be filtered from my database of daily equity data going back to 2005 where the closing price is over $1, the average daily dollar-volume is over $50M and the data is relatively clean (few if any gaps).  This yields an initial universe of ~600 equities and ~180K pairs.

Assuming that we keep N above 20 or so and we keep our allocation constant at $1M, and assume execution at the close price, we should expect our results to be reasonably believable as it’s unlikely that the size we’re trading in would distort the market.  Scaling any strategy presents its own set of challenges which we’re not going to go into for this example, but an obvious direction for scaling the strategy would be to simply run it over intraday data and scale into and out of positions as conditions permit.

Ok, so that’s what our strategy is going to look like.  Now, how do we “make it so”?

structure of a stratpart

Within Stratbox, strategies are represented as composites made-up of ‘stratparts’ each of which has its own metadata descriptor containing parameters.  Stratparts can be composed together within a strategy to enhance or modify the behavior of a strategy.  Thus, one can have a stratpart which implements the above logic paired with another stratpart which will e.g., modify the portfolio to impose beta neutrality, damp volatility or some other form of reporting or risk management.  Given that this example is already complexish, we’ll just stick to the one stratpart.

A stratpart has a few key elements.  One is the metadata descriptor which allows the strategy container (stratbox) to interact with it for back-testing, optimization or forward-walking purposes, or to allow a ‘trader’ to interact with it dynamically during run-time (the so-called ‘gray box’).  Beyond that, it has an essentially event-driven design.  The method quote() is called when relevant (i.e., subscribed) market data arrives.  The method execution() is called when executions for orders that originated with this strategy are received.  And the method strategyEvent() is invoked on a variety of ‘interesting’ events that the strategy implementer might care about.  None of these methods are required, so in its simplest form, a stratpart is a very simple piece of code to write.

services in a stratpart

Once one has complied with the minimal requirements for implementing a stratpart, one is rewarded with a variety of useful services: subscription-based market data feeds as well as a fast snapshot db and an in-memory historical db for quick correlation analyses cover market data needs.  Position management is handled at both the strategy level and at the exchange level (if you’re trading across multiple exchanges).  There is more, including performance analysis (e.g., sharpe, and its various analytic friends) and hierarchical cash allocation within and across strategies, but we won’t look at these in this example.

implementing metadata

So, the first thing we need to do is define our metadata.  This is done through the use of a conventionally-named static method Descriptor().

public static Descriptor Descriptor() {

 ArrayList<Param> params = new ArrayList<Param>();

 String cd = "Unleveraged capital to apply";
 params.add(new Param(InitialBalance, cd, false, 1000000.0));

 String n = "# of instruments to hold in portfolio";
 params.add(new Param(N, n, true, 20));

 String tc = "Trailing days over which to calculate correlations";
 params.add(new Param(TrailDays, tc, true, 252));

 String mc = "the minimum correlation of pairs to consider";
 params.add(new Param(MinCorrelation, mc, true, .75));

 String zd = "trailing days over which to calc z-score";
 params.add(new Param(ZScoreDays, zd, true, 21));

 String mizs = "minimum z-score";
 params.add(new Param(MinZScore, mizs, true, 1.5));

 String mazs = "maximum z-score";
 params.add(new Param(MaxZScore, mazs, true, 3.0));

 String ew = "evenly-weight portfolio? (or weight by z-score)";
 params.add(new Param(EvenlyWeight, ew, true, true));

 Descriptor _desc = new Descriptor
   ("puppetmaster.strats.Pairs", _Desc, params);

 return _desc;
}

public Pairs(Strategy strat, Descriptor d) { super(strat, d); }

Not the most interesting chunk of code, but pretty straightforward and its inclusion opens up a lot of functionality within the environment as each of these parameters can now be systematically or manually modified in-flight.  The third parameter to the Param constructor, a boolean, indicates if we want to allow the system to optimize this parameter.  Thus, the allocation/initialBalance parameter isn’t optimizable as it wouldn’t make any sense to do so.  Even so, we can (ourselves or through an allocation algorithm) modify this value to force the strategy to lighten or extend its financial footprint.  The others are fair game for optimization.

The constructor is also required – with that exact signature.  This is pretty much it for the ‘boilerplate’ code required.

‘installing’ the stratpart

Now that we’ve met the minimum requirements for a stratpart, we can install it into stratbox.  This requires adding one line to your personal stratparts.xml config file.  Like so:

<?xml version='1.0'?>
<stratParts>
 <descriptor class='puppetmaster.model.strategy.FwdWalker'/>
 <descriptor class='puppetmaster.model.strategy.StrategyPortfolio'/>
 <descriptor class='puppetmaster.strats.Pairs'/>
 <descriptor class='puppetmaster.strats.AMBO'/>
 <descriptor class='puppetmaster.strats.meta.AdjMeanReverter'/>
 <descriptor class='puppetmaster.strats.meta.AdjTrendFollower'/>
 <descriptor class='puppetmaster.strats.meta.Binary'/>

Once we’ve informed stratbox of the new stratpart we’ve written, we can view it within our library of stratparts within the strategy wizard in stratbox:

Our Pairs strategy in the Strategy Wizard's stratparts library

And all of the parameters we defined within our descriptor show up on the next page where we can parameterize or optimize the strategy:

Configuring the Pairs stratpart within the strategy wizard

This means that the stratpart can be combined with other stratparts to create new strategies as I’ve described  and that it can partake in the rich set of functionality within the stratbox gui.  Including actually trading.  But first, we must make the strategy actually do something…

implementing event handling

To that end, we must implement the event handling methods I’d described previously.  Since we’re not going to do any detailed execution handling in this strategy, we’ll only implement the quote() method and a subset of the events sent to the strategyEvent() method. In particular, we’ll listen to activation events (so we can initialize cleanly), changes to our descriptor and changes to our positions.

/** listen for select strategyEvents */
public void strategyEvent(StrategyEvent event) {
  super.strategyEvent(event);
  switch (event.type) {
    case Activated:            _init();                 break;
    case DescriptorChanged:    _readDesc();             break;
    case PositionChanged:      _posnChange(event);      break;
  }
}
<pre>

For quotes, we’ll just listen to the Beginning of Day (BOD) event.  If we wanted to trade intraday, we’d have to listen to tick events, but we’ll leave that for another day…

From a high-level, our code is pretty simple.  We determine the universe of pairs we might trade, we select from among them, we determine what our new portfolio should look like and then we trade to transition from our current state to our desired state.  That’s it.

/** mkt data goes here, but we only listen for the BOD
 * ("Beginning of day") event */
public void quote(Quote q) {
  if (q.type() != Quote.Type.BOD) return; // daily strat...

  try {
    List<_Pair> pairs = _getPairs();
    List<_Pair> sel = _selectPairs(pairs);
    List<Position> dp = (List<Position>)_desiredPfolio(pairs,sel);
    _Log.debug("PORTFOLIO: "+dp);
    _trade(dp);
  } catch(Exception e) { _Log.error(e.getMessage(),e); }
}

the messy details

Digging into the details gets more complex, but not horrifically so, as stratbox already has basic analytics like correlation matrices, z-scores, volatility built into it.  Likewise with basic portfolio analytics and transformations. Thus, our code to trade a (pairs-based) long-short portfolio with an arbitrary number of elements is surprisingly simple:

/** given a desired portfolio, trade to make it so */
void _trade(List<? extends Position> desiredFolio) {
  PositionRecord[] currFolio = posns();
  // Given our current portfolio (posns) and our desired state (dfolio),
  //  we generate the set of orders which will transform from the former
  //  to the latter
  //
  List<Order> orders = PortfolioComposer.TransformPortfolio
    (currFolio, desiredFolio, _orderF(), _strat, _strat.account());

  // place the orders for execution
  for (Order order : orders) {
    try {
      order.setType(Order.Type.MOC); // we trade at the close only
      _subscribe(order.contract());
      _execP().placeOrder(order);
    } catch (Exception e) { _Log.error(e.getMessage(), e); }
  }
}

As promised, I provide the full source listing below for those interested, but won’t over burden the discussion with a complete description of every little piece of it.

a telling omission and some results

Early in my career, working as a software engineer in a wall st front office technology department, I was befriended by a manager who told me he had once been a trader.  I asked him why he wasn’t a trader anymore.  He quipped:

I was half of a talented trader.  I knew, in my bones, when to get into a trade.  Sadly, I was never really sure when to get out…

It seems that the author of the Palantir example suffers a similar characteristic to my old friend (or is just being understandably cagey) – she specifies an entry condition, but not an exit!  Since she doesn’t impose any costs for trading, this has no big side effects, but in stratbox we do impose (pretty onerous -> realistic!) costs for trading.

In any case, since this is just a simple example I’ve maintained the ‘no exit’ policy and simply exit a trade once its pair goes out of the acceptable ranges for either correlation or z-score.  This has a bad effect on the strategy as you end up churning a lot of trades at the high end of the z-score range.  That is, if we set our upper boundary for z-score to be 3, then we might enter and exit a position multiple times as the z-score oscillates between, say, 2.9 and 3.1.  A more careful implementation could minimize this effect.

Another difference imposed by our more realistic simulation as opposed to the original example is that I’m restricting the portfolio size explicitly and explicitly limiting myself to trading a given contract within one pair as opposed to allowing a contract to trade across multiple pairs (invoking higher costs but allowing positions to offset and thus providing a neat little optimization – assuming trading costs nothing!).

All the same, the strategy clearly has some positive characteristics and when parameterized to take advantage of its naive exit strategy (by making the z-score low-end quite low, e.g., 0.5) takes on the character of a hedged trend-following strategy.  How so?  Well, while we are getting churned around the high-end (entry) z-scores, our profitable exits only happen once the z-scores have diminished (the prices have converged) considerably.  So we take more, smaller losses while our winners are held a bit longer.

some Pairs results in '10

Adding to my portfolio size improves things generally (and also increases the strategy’s scalability) while increasing the MinCorrelation parameter seems to hurt performance (not sure why, honestly – perhaps because if they’re very highly correlated and already diverging so strongly it may mean they’re actually diverging fundamentally). Returning to the topic of costs, each of these strategies incurred costs of over $5K for the period in question. And returning to the parameter which governs the weighting strategy employed, although I haven’t illustrated it here (as I haven’t looked at it very carefully), my initial finding is that it changed very little. 

There’s a great many areas where this strategy could be improved – that’s the fun, after all! – but hopefully even in its primitive form this example brings to light both an interesting baseline portfolio pairs strategy, while answering the question of how one goes about concretely implementing it within stratbox.

The entire strategy weighs-in at ~400 LOC including comments, boilerplate and individualized imports.  The source is here: Pairs.java.

Recently I had a thought-provoking email exchange with a reader of this blog.  It was with a fellow who wanted to startup an algorithmic trading business and was seeking my advice.  Its arrival was timely for me and I hope the advice I provided the aspiring entrepreneur was helpful.

The conversation forced me to reconsider familiar terrain from a different perspective, so I share it along with some further thoughts in the hope that it might act as something of a counter-weight to articles and blogs entreating you to “learn algo-trading!” (as though it were a fun! weekend hobby) or “make a million % trading the xyz” and more nefarious advertisements selling the trading equivalents of male-enhancement pills & potions (“I turned a used bus ticket and some pocket change into $7M in 2 months trading the e-mini!”).

I’ve changed the writer’s name and all identifying characteristics for obvious reasons.  He wrote:

Hi Tito,

First of all I want to say I really enjoy your blog.  I particularly like your latest portfolio of strategies that updates itself intraday while running a host of strategies in simulation.

I saw that you left Wall Street back in 2005; I am currently at [a 'money-center' bank in] London and am contemplating a similiar move.  Was wondering if you have any advice or words of thought, particularly in this market/economic environment?  My dream has always been to work at a small hf or high freq shop or to start one of my own.  After several years, I finally think I have enough strategies and angles to go forward.

Btw, you are not at Tower Research, are you?  They have t-shirts which read “Hack Wall Street”

Regards,

[Sean]

Not knowing ‘Sean’ from Adam, I replied aiming to be helpful but with a characteristic dose of snark:

Hi [Sean],

Thanks for your kind words – they’re appreciated.

I always have advice.  Quality is the issue.  That said, even my ordinarily low standards can’t be met w/o more info about you.  (e.g., what kind of work have you been doing?, what kind of money do you have salted away (that you are willing to lose!)? etc etc).

But I can still make a few observations.  My first is general:

If you haven’t been working, directly, in a hedge fund or a hft shop, I wouldn’t recommend trying to start one on your own.  Join one, understand all aspects of the business, build a track record and contacts and only then (if at all) attempt your own gig.

Regarding: “particularly in this market/economic environment”  and “HFT”:

HFT is (wrongly, imo) currently under serious attack.  Off-market derivatives and bought politicians were used to wreck economies, but HFT has become the scapegoat.  I believe that [the bank you work at] has shut down their prop HFT biz and I know that [another bank] has.  Many others go under every day.  Whatever else this might mean, it certainly means that there will be fewer jobs and more qualified people on the street competing for them.

It also means there is a possibility (probably a small one) that some kind of regulatory action literally kills the field.  A tobin tax or the like would mean the end of HFT.  Which would make HFT an unfortunate business to be in…

Also, HFT is a *very* specialized field.  You need to have very specialized skills to really add value.  Only you can answer if you do or not.

Finally, a wise wall st old-timer once told me: “there are years to make a killing on your bonus and there are years to keep your job…”

Regarding: “a small hf”

Hedge funds are (mostly) a very different business from HFT shops.  The successful hf mgrs I know all aver that the game is all about raising money – managing it is very secondary (unless you’re seriously foolish or a fraud).   It also raises the question:

Most of all: What do you really want to do with your life?

Good luck to you.  Best,

Tito.

ps – I’m not sure that I’ve ever even heard of tower research…

This probably wasn’t what he wanted to hear and he came back a bit piqued at my necessarily ill-targeted advice:

Same clarifications if I may:1)  When I said my dream was to start my own small hf or high freq shop, I should have stated that is my long-term goal.  In the short term, I see myself working for a hedge fund first while also trading my own PA in respectable size (quant strategies, fully systematic).

2) I should state that I have run a $50M systematic book successfully at [London bank] albeit only on low frequency strategies.  I have also worked at a very large prominent electronic market making desk at [London Bank].  So while this is not the same as have direct experience at a hedge fund, I dont consider myself a novice either.  However, I agree that I am not ready to start my own hf or hf shop.  My issue at the moment is I would like to focus more on the medium-to-high frequency end at the spectrum, but it appears this is not going to be possible at [London Bank].

3)  I agree with many of the the things you stated about HF.  Just curious, do you consider yourself a HF trader?   I honestly consider myself more of a medium frequency/intraday with no overnight risk trader.  I am definitely not ultra high frequency or doing any latency arb and I think “medium frequency” is a good place to be.

4) Question for you:  do you consider a 6 figure PA account large enough to establish a track record in terms of hashing out if something indeed has an _actual_ out-of-sample edge?  Granted this is not alot of money, but my assertion is that this is enough to “go live” with several systems _in very small size_ with strict money management. [ie no more than 2% risk per trade, absolute max 10:1 leverage, etc]  And I am not starting from scratch; I have been in quant strategy for over 2 1/2 years and have several systems ready to go.

Where I am at now is that I am ready to do #4 full-time while continuing to interview with hedge funds and high freq shops.  I have the right background Electrical and Computer Engineering from [top-tier American University], graduate work in time series analysis and econometrics from [top-tier UK biz school], [more impressive sounding experience] but it is taking longer than I had hoped to make the jump to the buy side.

I realize you are probably thinking “hold on to your job on the sell side at all costs etc..” as it is really bad at there and may likely get worse, and while I dont disagree, at some point a person has to move towards what they want instead of living in fear forever.  Let’s just say the past couple of years in banking have not been enjoyable.

Sorry for ranting but hope you can understand where I am coming from…would appreciate any advice you may still have…

Regards,

Sean had provided some useful information with which we could do some simple math. He had a good background and some hundreds of thousands of dollars he was willing to risk on the venture. Not bad, right?… I responded:

It all sounds reasonable and none of it sounds like a rant.

1,2  Sound good to me.

3 Definitely not ultra high.  Many hundreds of trades per day at most for any given strategy.

4  Yes, but for what?   Your (excellent) background is certainly no handicap (though many or most places require a phd for a quant position with p&l), so the question as to whether you can land the job you desire is an empirical one.  Go for it!  Certainly I wouldn’t recommend that you quit your job and start trading as that immediately puts you at a very substantial disadvantage from the perspective of employers.  I empathize with the “past couple of years in banking have not been enjoyable,” but 6 figures of savings aren’t a ton if you’re used to living a banker’s lifestyle in an expensive financial center like nyc or the city.  Going to the buy-side sounds like a great idea; quitting your job? – not so much.

Your decision will also depend on your position in life.  If you have a family (or want to), a partner who enjoys spending money, a desire to travel, etc etc.  Quitting your job to trade your own account is certainly ballsy and if you are successful then shazam!  If not, then you may have irrevocably pushed yourself off the (successful if not enjoyable) path you’ve worked so hard to obtain; if economies really go down the tubes or bigger wars become fashionable, you could find yourself in a really funky spot.  Or you could be wildly successful and build a giant sailboat that kick’s larry ellison’s butt…

New Holder of the America's Cup!

Let me ask you a few questions.  What platform(s) do you intend to use?  Where will you get your data?  How will you store it?  Manage it?  For me, back in 2005 I thought I’d just head on out there, buy some spiffy product and be off and trading.  I was profoundly wrong about that.  Retail level platforms are crap.  Seriously unusable.  (My info on this may be dated, I admit – hence my questions to you )  I don’t think you can afford an institutional platform with 6figures in the bank unless you are *immediately* profitable and burning through half your wad in a few months will impose stress that’s altogether more serious than whatever ridiculous nonsense I imagine you have to put up with at [London bank] these days.  Building your own platform isn’t so easy and is quite time-consuming.  Worse than that, it causes you to have to focus on a lot of things that are *not* central to your primary aim.

Let’s look at numbers.  Say you have ~$500K in hand and a requirement of ~$100K to live.  20% seems plausible.  But you will actually need to do rather better than 20%.  Insurance costs are a mother (maybe not in the uk), but they’re a good deal less than usable office space, bandwidth and lots of other oddments you’ll find yourself paying (eg, accountants aren’t free, lawyers aren’t cheap, and programmers mostly suck).  So, your real expenses aren’t 100K/year, but more like say 150K or worse.  Oh yeah, there’s taxes, too.  So, in reality starting from a 6figure base, you need to make ~30% or rather better to *break*even*.  If you have a year in which you make less than your break-even point, then the numbers get really grim.  On the other hand, you need to have a serious blowout year to make a significant difference in these basic numbers.  There’s no margin for error or, worse, misfortune.  Don’t get sick, don’t break an arm, don’t have a loved-one need anything from you and don’t lose any money!  In spite of all of your experience, intelligence and hard work, a bit of bad luck can put you into an untenable position.  There’s a good reason the Greeks believed that the only thing more powerful and less controllable than their rich pantheon of powerful Gods were The Fates.

All of this said, who the heck am I?  Sounds like you have a great background and some passionate ideas about what you want to do.  I don’t want to be negative or dissuade you from pursuing a dream.  But I assume you’ve got all the up-side worked out and have thus focused on the (potential) down-side!  ;^>

Best,

I haven’t heard back from Sean and at this point don’t expect to though I gave him as direct and honest advice as I was capable of providing.

a cautionary tale

This exchange resonated with me because, almost exactly five years ago I had a ‘serious’ conversation with my then-fiancée (and now wife and mother of our beautiful son).  I had been working hard for years and had lived very frugally (would you believe a $900/month 1-bedroom apartment in manhattan two blocks east of union sq?), saving my money to one day pursue a long-held dream.  I said to her, “I can buy you a house with cash today or I can start a business and, most likely, lose everything and have to start all over again.”

(This was my own version of David Einhorn’s memorable ‘Greenlight moment’ as recounted in his “Fooling Some of the People All of the Time.”)

She assented and not too many weeks later I walked away from my steadily ascending career and Puppetmaster Trading was formed.  I was then in a similar circumstance to what my erstwhile pen-pal Sean is in today.

Around the time that I had this email exchange, I was being forced to face the fact that Puppetmaster Trading was in just such an untenable position as I’d described.

Someday I might write a post-mortem along the lines of the courageous and revealing post Roger Ehrenberg wrote describing the errors he’d made in one of his startups.  But today isn’t that day; I need some distance and perspective to adequately compose my thoughts and identify the key errors I’ve made along the way.

So, to Sean and others like him, view this as a cautionary tale.  Don’t view it as ‘proof’ that starting your company is a bad idea.  Even given constraints like Sean‘s, I’m 100% certain that success was possible but for a few costly errors of judgment I personally made.  After the first year and a half, our trading returns were always positive.  In 2008 we had a real blowout year which could have been a game-changer had I not elected to pursue an ill-fated partnership (which hinted beguilingly at an outright sale) rather than sticking to what was working.  There were other mistakes, too – some of them possibly worse.  Interestingly, all of them were about running a business and not about algorithmic trading per se.

Selecting a business model and focus, funding issues, partnership dynamics, effective team-building and the like are where the game is won or lost.

the funny business (models) of algorithmic trading

As far as business models go, algorithmic trading can be taken in quite a few directions, the simplest of which is proprietary trading which is where we ultimately settled.  If you’re building a platform you can try to be a product company, but then I don’t think you’d really qualify as an algo-trading concern.  One can also form one of many varieties of funds or advisories to bring in external funding and all of these have their strengths and drawbacks.  And all require the track record, marketing savvy and connections to bring in the commitments.  We’ve looked at almost all of these options and pursued a few.

If you’re building your own platform with the intent of trading it, it’s a bit of a funny situation, because no-one wants to bear the cost of building a system you won’t sell.  Buying systems and then integrating them such that they’re practically usable is such an expensive proposition that one is essentially forced to bring in external money unless they happen to have a few spare millions of dollars lying about that they don’t mind spending. Actually, in retrospect, this is nearly as true if you’re building instead of buying.

Perhaps the most interesting models that we explored were those in which algorithms backed structured products or ETF-like vehicles.  I think there will be many such products in the future with good reason.  But to do this, you need to partner with a bigger financial firm.  Here, again, you need connections (and probably will be better served by a firm name a bit more sober than our own!) and at least some heft of your own as established investment houses aren’t in the business of working with the proverbial two guys in a garage.

All of these are viable models, and all present unique challenges and opportunities.  The bottom-line is that to startup successfully you need to select one and then either obtain funding early on or you need to be lucky *and* commit no errors.

to build or buy…

As for the platform we built, Stratbox, I can only view it as a success – many tens of thousands of trades have passed through it and only once, very early on in (I think) december’05 (before it had even a rudimentary GUI), did we lose any money from it on account of a coding error.  Although I haven’t looked carefully at algo platforms in years, I’m told it has features that even well-established companies haven’t put into their commercial products and there’s no doubt that having full source code enables degrees of freedom that are impossible with off-the-shelf products.

The fact that Stratbox isn’t a commercial product has real drawbacks, especially in terms of the ‘fit and finish’ one expects from established software.   But it also brings considerable  benefits in that you can really drive development based on immediate needs.  In very short time frames, you can build the 90% of functionality that you need *now*.   It’s just not possible for an established vendor to implement, test, and roll-out new features with the requisite solidity and 100% finish anywhere near as fast as can an ‘agile’ team with skin (and maybe an organ or two) in the game.  Thus, if I were to start all over again, while I would look at the various vendor offerings (and might well change my mind based on that review), I’m still inclined to think that I’d continue to “roll my own”.

the evolution will be blogged

I started this blog as a means of composing my thoughts and sharing my ‘algorithmic trading experiences’  and in that light it’s been a quiet success.  Although it has been a minor time- and cash-sink, I enjoy it and have benefited from the people I’ve met through its pages.   I also always view writing as its own reward.  I send a sincere “thank you” to all who have read and all who have shared their own insights and experiences.  At some point – not too soon – I may try my hand at a constructive post-mortem.  In the meanwhile, I’ll continue as I have and write when I have something to say.  The longer-term fate of the blog remains to be seen and will likely depend on what I end up doing next.

One of the recurring technology themes in these pages has been the ongoing and dramatic move from single to multi-core systems and the need to seriously increase the parallelism in our software designs. For me, one of the seminal, large-grained design patterns was the SEDA Architecture. For years, this informed my systems’ designs and formed a conceptual backbone for development. That said, I’ve been broadly aware for some time that SEDA’s golden age has (incredibly!) already passed us by, but haven’t identified what might replace it as a reference point for my design efforts.

Before considering tools, languages or patterns that might help, we need to reflect on the problem(s) we’re trying to solve. The problems inside an EMS look to me, after years of development, a lot like network routing problems. Indeed, my current view is that this (not just concurrency as I’d suggested at the time) is why the unfortunate Aleynikov & co. at GS were using erlang.

Why network routing? Think about the load on an EMS. The main issue is that you’re getting many thousands of teeny little messages per second and only a relatively small number of them matter to only a relatively small subset of ‘agents’ within the system. Reducing latency is all about making sure the time you spend on each message is minimized, and that the agents who are interested in a particular message needn’t wait for each other to do whatever they care to do based on the message. So, really you’re trying to route each message through your system with as few ‘hops’ possible and as much parallelism as you can muster under the (radically!) new assumption that you may have hundreds or thousands of cores available to you during the lifetime of the design.

I spent some time thinking (hoping) that languages might help furnish an answer. Perhaps a move to a functional language like erlang, ocaml or scala might help furnish at least a partial answer. But erlang is slow and peculiar, ocaml doesn’t support intra-process concurrency and scala looks like a bloated language on a bloated platform (jvm+java class library). And none of them seem to have achieved anything near the critical mass which is so crucial for the development of usable libraries and the availability of skilled developers with long experience in the technology.  Naturally, reasonable people will disagree about such things, but this is my view (today). Java is ok (and certainly sells servers), but it’s not obvious how it’s going to help me offload my work onto a GPU anytime soon (and jni is both painful and slow) and I’ve never been able to get comfortable with just how damn big VMs get.  Image size isn’t free and if we’re looking to go deep into the sub-millisecond response time, while running thousands of concurrent strategies, it seems we need to disintermediate the VMs and interpreters of the world. If they’re really necessary, they can be happily used for the analysis process (as I currently use R), or they can be lit-up and bridged from some lower-level language for batch-like services.

The good people at Intel have been thinking about this problem for a while as have many other seriously over-educated people. One of the (sensible sounding) conclusions reached as people look for ways to solve problems similar to my own, is that in such systems we should keep messages waiting as little as possible – ideally, not at all(!). This can be a problem in SEDA-like architectures which are basically made-up of (non-blocking, asynchronous) i/o processes linked to (blocking) queues linking pools of workers. Blocking queues can pile up and cause all sorts of problems like priority inversion and other such enigmatically named nasties. Lock-free queues and other data structures, algos and techniques promise some ways around this and I’ve been spending time looking into how they might be employed to address my issues.

Before I’m besieged by throngs of angry erlang/ocaml/scala/java developers, allow me one last observation on the topic.  (Peeved python and ruby users may rant away – vous m’amusez  ;^)

Why might a lock free algorithm be better than an equivalent, hardware-based locking implementation?  The answer isn’t obvious.  If locking is implemented in hardware as is typical (eg, with a compare-and-swap (CAS) instruction), then its explicit cost is measurable in (few) nanoseconds.  Hardware is fast.  The issue isn’t the speed of execution of the underlying primitives so much as it’s a consequence of the side effects of these operations at a very low level.  For real performance, cache coherence is King.  See here for an accessible discussion by IBM’s Paul McKenney and here for some remarkable examples from Igor Ostrovsky.  This indicates that if you want the highest possible performance, you need to be aware of what is happening ‘in the metal.’  So we need to use a system-level language and erlang, java & friends lose their candidacy in spite of any fantastic benefits they might offer.

Given that even the DoD has mostly given up on ADA means that we’re left with C/C++.

Ok, so language doesn’t seem to resolve much for us. (Indeed, it was mostly hopeful thinking on my part – design is mostly language agnostic and hardware is hardware…)

Apart from Intel’s own Threading Building Blocks (TBB) framework, there are a variety of toolkits available for exploiting lock free parallelism. Perhaps the newest and least known is called FastFlow, which is a C++ template library that provides a variety of facilities for writing efficient lock-free network models. It also claims to be faster than TBB, Cilk and OpenMP while holding out the promise of one day becoming CUDA- (or more generally, GPU-) aware which would be an incredible win. Finally, it is very small – the current version (not including tests and examples), weighs in at ~5K lines of (mostly) C++ templates.  Thus, it seems to me particularly well-suited for some experimentation to assess the fit of these techniques in this space and the level of difficulty of doing so.

In the remainder of this post, I’ll briefly describe the FF design and then illustrate a sample C++ program which uses FastFlow to ‘architecturally prototype’ a feed handler interacting with strategies inside an EMS / strategy container.

FastFlow architecture

FastFlow’s (FF) goals seem quite ambitious, but the design is layered to allow users to work at a level that makes sense for the problems they are trying to solve, so they needn’t worry themselves (or even understand necessarily) layers below where they need to work.  At bottom is a thin layer of hardware-specific code supporting, one level up, single-producer-single-consumer (SPSC) lock-free queues and a threading model for their efficient interaction.  Above this are defined objects for generalizing from simple to arbitrary network types.  Here you see the definition of a composable Node which serves as the key abstraction for general-purpose streaming networks.  Above this are defined a variety of skeleton templates for building commonly useful graph types.  Because they are Nodes themselves, they can be nested and coupled to any depth.

Consider a simple pipeline:

a simple FastFlow pipeline

A farm is slightly more complex and acts something like a load balancing threadpool.  The Emitter pushes ‘tasks’ of arbitrary type to an arbitrary number of workers who then coalesce back into an (optional) Collector.  A simple Farm makes no promises about the ordering of tasks, but (I believe) FF provides mechanisms for ensuring their ordering at the collector.

a FastFlow Farm

Finally, because of the composable nature of Nodes, all of these graph types can be combined arbitrarily.  Here you see a Farm of pipelines with an ‘accelerator’ (which, if I understand correctly is just an integrated thread for pushing tasks into the network) and a feedback channel.

composition of FastFlow subnets

For a more complete (and probably accurate!) description of FastFlow, I encourage you to visit their site and tutorial.  The documentation remains limited, but there’s a good and growing selection of examples to examine and all of the source is available for inspection.

a simple example

Given my use-case, my first interest is in seeing how FastFlow might work in the context of asynchronous, non-blocking network handling.  To this end, I’ve built a simple Farm as depicted above in which the Emitter uses select() to read messages off the network which it parses and pushes into the farm for ‘processing’ which is currently NOP-ed.  The messages/tasks are then coalesced into a collector which does nothing more than free memory allocated in the Emitter.  Within the Farm, each of the nodes inhabits its own thread and outside of the farm I’ve implemented a very simple ‘client’ which connects to the Emitter’s socket and sends delimited Integral ‘messages.’  Ordinarily this would appear in its own process, but for the sake of only requiring 1 file, I’ve mashed them all together and the client lives in its own thread.  That’s it.

It turns out that the FastFlow part of this is, by far, the simplest.  Since the FF skeletons are implemented as templates, extending them couldn’t be much easier.  Here’s the definition of the worker and collector components.

// typical worker: does little ;^p
class Worker: public ff_node {
public:
  void * svc(void * task) {
    return task;
  }
};

// collector that just frees the malloc'ed memory
class freeing_collector: public ff_node {
public:
  void * svc(void * task) {
    int * t = (int *)task;
    if (*t == -1) return NULL;
    else free(task);
    return GO_ON;
  }
};

They are both nodes and their only requirement is to implement the svc method which is called by the FF framework when they are meant to service a task.  They can optionally implement svc_init and svc_end methods which provide an opportunity for initialization and cleanup.  The socket-reading emitter makes use of the svc_init method to initially setup the socket.  Later, as it repeatedly reads from the socket into a buffer, it cracks the incoming stream and then calls the ff_send_out() method to push those messages out to the downstream workers.  The whole example is about 350 lines of code and nearly 200 lines are consumed by the Emitter.  Almost all of this bulk is network-related, so I won’t illustrate here.  But highlighted here you can see the main FF-specific part of the Emitter:

 // handle new data coming in
 void new_data(int nbytes, const char* buf) {
   static char remainder[kMaxMsgSz];
   int *t;
   int j = 0;
   int i = 0;
   char msgbuf[kMaxMsgSz];
   for (; i < nbytes; i++) {
     if ( kDelimiter == buf[i] ) {
       int rlen = (j==0) ? strlen(remainder) : 0; // remainder?
       int len = i-j;
       if (rlen>0) strncpy(msgbuf,remainder,rlen);
       int start = (rlen==0) ? 0 : rlen;
       strncpy(&(msgbuf[start]),&(buf[j]),len);
       msgbuf[len+rlen] = '\0';
       t = (int *)malloc(sizeof(int));
       *t = atoi(msgbuf);
       j = i;
       ff_send_out(t);  // push message into farm
     }
   } // for

   int k = j+1;
   if (k < i) { // save 'remainder'
     strncpy(remainder,&(buf[k]),(i-k));
   } else remainder[0] = '\0';
 }

Finally, all of the pieces are assembled and launched in the main().

//  We construct a 'server' which is a fastflow emitter and which reads
//    integral 'msgs' from a client over a non-blocking socket.  Msgs
//    are parsed and fed into a fastflow farm for further handling.
//    The client is placed within its own thread - this is only so the
//    whole example can be placed in one file.
//
int main(int argc, char * argv[]) {
 // use: argv[0] <#msgs=1024> <#port=9999>

 int msgs = (argc > 1) ? atoi(argv[1]) : 1024;
 int port = (argc > 2) ? atoi(argv[2]) : 9999;

 int nworkers = 5;               // how many workers will recv msgs

 printf("main: sending #%d msgs to port <%d>\n",msgs,port);

 select_reader sr(port);         // we create 'server'
 freeing_collector fc;           // and a freeing collector
 ff_farm<> farm;                 // and a farm for it to live in
 farm.add_emitter(&sr);          // add both to the farm
 farm.add_collector(&fc);

 std::vector<ff_node *> workers; // build a collection of workers
 for(int i =0; i < nworkers; i++) { workers.push_back(new Worker); }

 farm.add_workers(workers);      // add all workers to the farm

 farm.run();                     // launch the farm
 printf("main: started farm.\n");

 sr.wait_til_ready();            // don't create client til srvr ready

 // create client which will send msgs via socket
 //
 printf("main: creating client...\n");
 client_thread* client = new client_thread(msgs,port);
 if (client != NULL) client->join();

 farm.wait();                    // wait for farm to be done its workload

 // emit some stats
#if defined(TRACE_FASTFLOW)
 std::cout << "DONE, time= " << farm.ffTime() << " (ms)\n";
 farm.ffStats(std::cout);
#endif

 return 0;
}

The whole example can be found here.

Writing this example was a bit of a ‘tale of two frameworks’ for me as I found the FastFlow part to be really easy and powerful.  On the other hand, I wanted to try out the boost::asio framework for non-blocking networking i/o (primarily in the hope it might cut down the code size a bit) and while implementing the trivial (blocking) client was, well, trivial, implementing a non-blocking server with boost::asio was a serious pain and I ultimately decided that Linus just might be on to something wrt to boost and scrapped that part of the effort (but kept the boost client).

Nothing about this example illuminates the potential performance advantages of a lock free approach, but this exercise has certainly convinced me that the FF authors have managed to combine very low-level performance considerations within a package which can be easily used from a high-level perspective.  Most problems will most likely be better served by higher-level programming languages, but some problems remain intractably bound to the metal and I suspect that writing ultra-low-latency trading systems will remain in the latter class for some time to come.

In the example below, we look at another ‘meta’ strategy named StrategyPortfolio which maintains a dynamic portfolio – P – of strategies which it will select from a set of strategies – S – running concurrently in simulation.  The constituents of P as well as their cash allocations and parameterizations will be rebalanced/adjusted regularly after an initial ‘out-of-sample’ period during which only the S strategies are run.

Apart education, the intention of this strategy, as I’d originally suggested here, is to ‘back-into’ a regime-switching strategy without attempting to directly quantify the regimes explicitly.

This has proved to be even more interesting than I’d expected, not so much because it performs particularly well (though it’s promising), but because of all of the things it has taught us.  In particular, the transitions are a killer and there are properties of strategies which (dis-)qualify them from being effective in such a scheme…

bad news good news

As before, I’m not going to take the time to write-up my results in any formal manner but will again rely upon a quick screencast of the software running.  The good news is that I’m figuring out how to edit these things, so it’s mercifully shorter than earlier screencasts…

Please click here to see the screencast.

I hope you find it interesting and will look forward to any comments or suggestions you might make.

photo by Simon Rogerson

Some time back, I’d written about NVidia’s CUDA noting that it looked ideal for many asset-pricing and monte-carlo type problems in finance.  At the time, I was hopeful that it would be quickly integrated into existing open source efforts like QuantLib, but adoption has proved slower than I’d hoped, most likely because implementing non-trivial problems on CUDA is, well, even less trivial than doing them without..

LMM on CUDA

Happily, I’ve just seen a promising first step in this direction as Über-quant and C++ artisan Mark Joshi recently announced an open-source project, Kooderive which looks to implement the LIBOR Market Model (LMM)  on top of CUDA.  His announcement on the QuantLib mailing lists reads:

Dear All,

various people have shown interest in the use of CUDA with QuantLib. I
have now made some progress on a CUDA implementation of the LIBOR
market model.

In particular, I now have a path generator for the LMM working which
does 16384 paths for 40 rates, 40 steps, 5 factor model, displaced
diffusion predictor-corrector that takes 0.1 seconds on my Quadro 4600.

The state of the project is code fragments that can be called from
other code. Those who are interested can get the code via
the subversion repository on kooderive.sourceforge.net .  The only
project file is currently for VC9 x64. It also uses thrust and the
CUDA SDK.

The next stage will be writing routines, that use QuantLib for the CPU
stuff and kooderive for the GPU stuff,  to actually price things.

A gentle reminder that I will be giving a course on the LMM and
QuantLib in June in London, and I will include a session on kooderive
if there
is sufficient interest.

I am happy to take code contributions for kooderive. However, I am not
looking for a redesign of the library or contributions which introduce
dependence on other libraries. I am interested in contributions of
separate routines and of optimizations of existing routines that do
not change interfaces.

regards

Mark
–
Pricing exotic interest rate derivatives – The LIBOR Market Model in
QuantLib June 2010, London,
http://www.moneyscience.com/training/index.html

Assoc Prof Mark Joshi
Centre for Actuarial Studies
University of Melbourne
My website is www.markjoshi.com

Dingbat Kabuki and other structural market hacks

When I first started puppetmaster trading, one of my dearest friends, a Yale-educated economist and professor of same, asked me an important question.  He asked:

In the markets, there are always ‘insiders’ who have the ability to trade on knowledge that you can’t know or with an advantage that you can’t have.  How are you going to compete with these players?

I provided a variety of answers, but at the time my conception of the universe of people with both inside knowledge and the ability to trade on it was limited to cases like that of Mr Rajaratnam.  I believed that cases like these were constrained by clear laws that were duly surveilled and prosecuted by the appropriate authorities.  The problem seemed like a very real one, but constrained in size and not essential to my enterprise.  I still hope that my belief of the time was true, but since then I’ve certainly understood that there’s more than one way to hack the market.

For some, a market hack might consist of some kind of simple (or complex) algorithm(s) applied to some set of markets.  But this really isn’t a hack so much as it’s a trading strategy – like many that have long existed – only that it’s now implemented in software where originally it would have been implemented in wetware.  While implementing trading strategies in software does open up new vistas in terms of the kinds of strategies that you can look to implement – computers are faster than people by a noteworthy amount in many tasks – but, for the most part, you’re really still just trading and when you take on positions, you are still bearing risk.  You might be ‘hacking’ but it’s really not a market hack as I’ve come to appreciate.

hack the market structure

Another area where my perspective has changed substantively since those halcyon days of ’05 is my appreciation for market (micro) structure.  In futures, market structure is pretty plain as most contracts are effectively monopolies run by their listing exchanges.  There are a few cases of instruments which are tradable across markets, but a rich market microstructure is just not a core identifying characteristic for futures like it is in the fragmented and incredibly dynamic and quickly developing world of equities.  So, while there are some dozens of futures exchanges scattered about the globe, as a futures trader, routing algorithms just don’t enter the picture except in limited and relatively simple cases.

If I have a view on or a hedging need for interest rates that I see effectively expressed through futures, then I go to e.g. the cme and I don’t need to worry my pretty head about getting a great execution because the exchange is in one place and it’s all lit up and filled to the brim with liquidity.  Easy peasy.  But if I have a view on, say, General Electric that I want to express with an equity position and I’m trading in sufficient size or sufficiently close to the market that I really want to ensure best execution, then I might find the need to look around some dozens of lit exchanges and maybe even ping about in a dark pool or twelve.  This is decidedly not easy peasy and is one of the reasons wall st has an insatiable hunger for propeller heads with advanced degrees in seemingly unrelated fields.

Last summer’s brouhaha about ‘flash’ orders first illuminated for many a rich ground for genuine market hacks: the incredibly dynamic terrain of equity market micro structure which changes almost daily with the emergence of new exchanges, ATSes, order types, rebates and pricing structures and all of the many other critical minutia that differentiate the many venues.  Some of these undoubtedly provide meaningful and important services; contrary to populist inclination, dark pools are largely a defense against frothy HF trading strategies.

But the concern (if perhaps not all of the attention) is merited.   Equity microstructural ‘rules’ change so quickly that even independent and upright regulators could hardly be expected to keep up.  This is clearly fertile ground for genuine – if perhaps fleeting – market hacks.

steganography and hacking the market

What characteristics make equity market structure a fertile ground for these kinds of genuine market hacks which I’m describing?  I think the main answer is to be found in the ancient art of steganography – hiding in plain site.  Everything about equity market microstructure is public.  All of the rules for each of the venues are available to the people who might make use of them.  Unraveling what they imply about where you should be executing for any given order is the tricky bit which is hardly revealed through a simple reading of all of the various rules.

Where else do we see this phenomenon?  The US Tax code comes to mind (sort of like the iPhone: “there’s a loophole for that”).  The 2,000 page health care bill comes to mind.  Everybody seems to care about health care, but who’s actually read that bill?  Who could?

Totally public yet it may as well be encrypted.  Steganography.

Elizabeth Warren has also made this observation in the context of contracts made between banks and their retail clientele.  Noted scholar Scott Adams dubbed it a ‘confusopoly.’

a group of companies with similar products who intentionally confuse customers instead of competing on price

So, could people hack the tax code?  Hank and I think so.  So do my friends in the lobbying business.  How about all those recently converted bank holding companies – hack much?  In my last post we read one insider’s view that Madoff had effectively hacked the SEC… perhaps regulatory organizations themselves are also hackable instruments!  How about that health care bill?  …

misdirecting the hack

A lot of attention has focused recently on exchange traded markets and people are up in arms about HF traders and proprietary trading.  I’ve argued before that this appears to be a great ploy to take attention away from the real issues at hand; in the worst case, HFT improprieties might account for no more than 1% of the money disappeared in the last few years as part of the so-called ‘credit crisis’.  It’s easy for non-finance-professionals not to understand that the big bad wolves of the credit crisis essentially all happened off market in essentially unregulated multi-trillion dollar otc markets of ingeniously engineered structured products. These are the real market hackers.  Exchange traded instruments had effectively nothing to do with our current circumstances but remain a convenient scapegoat.

Some argue persuasively that there are fundamental, structural market hacks at the very foundations of our financial system.  I wouldn’t know.  But it is the kind of thing one might think about while watching the highly stylized performance of our leaders last night.

dingbat kabuki and the transparent market hack

To my knowledge, Cal’s (go bears!) Professor Brad DeLong coined the term ‘dingbat kabuki’ back in 2005 in shrill response to a Washington Post article.  He reprised the term this week in response to the latest bit of macroeconomic genius out of washington.  What an inspired phrase.  Yesterday at 2:30 and last night at 9pm the markets voiced their applause and cheered on the performance.

Indeed, it is masterful.

–

A technical note about this post.  While writing it, I seem to have accidentally published it at some point and then realized the error and ‘unpublished’ it.  I apologize if this had any untoward effects on you or your RSS reader.

But some are revealing and one in particular is 100% required reading for anyone interested in electronic markets.

The writer introduces himself thusly:

I am Steve Wunsch, the principal inventor of two SEC‐regulated stock exchanges, the Arizona Stock Exchange “AZX” (originally called Wunsch Auction Systems, Inc. “WASI”) and the ISE Stock Exchange, both of which include dark pools. In fact, both of them, like all modern stock exchanges, have both lit and dark components and, thus, have provided me with potentially useful perspective on the dark pool question and on transparency in general. I will focus heavily on the latter, for it is impossible to understand the dark pool issues raised without understanding the value of transparency or, if improperly applied, the lack thereof. The AZX experience was, I believe, particularly instructive in this regard. Its highly transparent call market structure, combined with its unique regulatory status as a “low volume exempt”exchange, enabled me to see transparency and the role of regulation in promoting it from a perspective that I don’t believe anyone else has.

He deftly mixes snark and a historical perspective on regulation with an opinionated and informed view on the forces driving current equity markets’ microstructure arguing that the worst issues are due to regulatory failures.  He concludes, logically enough, that the SEC should be disbanded.  Perhaps his most inflammatory bit is his claim that the “SEC made Madoff.”  For effect, the section is entitled “An American Oligarchy”:

AN AMERICAN OLIGARCHY

It is not in the Commission’s interest to admit failures of policy, such as the ones I have described in this letter, and I have never seen it done. It was not in the Commission’s interest to admit that Bernie Madoff was the SEC’s most trusted and intimate confidante in formulating and selling transparency, electronic trading and
the whole NMS concept to Wall Street, the public and Congress. His legitimate business was the epitome of the kind of transparent electronic competition that NMS’s leveling policies were trying to create, and he occupied the most favored place of all industry advisors on policy and rules as NMS was being created. In a very real and literal sense, Madoff’s legitimate business and NMS were made for each other. NMS cleared a path for the application of continuous transparency by new electronic competitors, very visibly led by Madoff, enabling him to become at one time the third largest market in the United States, even though he wasn’t officially registered as anything but a broker‐dealer.

Had the SEC not emasculated the rules by which the NYSE controlled its members, Madoff would never have happened. In the time before NMS, when the exchange had Rule 390 or the stronger Rule 394 before it, diverting orders away from the floor or selling them to Madoff would have been banned. But on antitrust principles, the SEC wanted to foster NYSE‐busting competition in NMS, and Madoff became its PosterBoy for such competition. In order to make way for him, the SEC opened up a variety of loopholes that allowed orders to be diverted from NYSE to Madoff and printed on regionals like Cincinnati. Rules 19c‐1, 19c‐2 and 19c‐3 were in this vein. There were perennial attempts by the NYSE to plug the loopholes and rein in the membership, but the SEC batted them all away, enabling Madoff to continually grow his business. Eventually, the NMS environment forced the NYSE to abandon Rule 394, then Rule 390 and ultimately its membership organization altogether when it demutualized. This was all very good for Madoff. And Madoff was very good for NMS, giving it industry cred far in excess of what this poorly articulated socialist leveling theory could have had without his support.

In spite of a 457‐page SEC investigation into Madoff and how his Ponzi scheme was missed, the most obvious reasons were not considered, namely, that Madoff played a central role in helping the Commission design and sell NMS, and that NMS made him rich long before the Ponzi scheme. Most importantly, the credibility that theCommission’s collaboration with Madoff on NMS conferred on him was the principal factor enabling him to bring in money for the Ponzi scheme. Although the investigation’s report notes his credibility in the industry, it is mentioned as if itwere just a fact of life and was already there. Not mentioned is that his superior access to the SEC and apparent influence over the Commission, both of which were implicitly proved by his ability to get rich on NMS, are the most important reasons that he had such extraordinary credibility in the industry. The truth is that the SEC made Madoff. He could not have existed as a threat to investors without the Commission’s active and dedicated support over several decades.

Although, in typical blogger fashion, I’ve highlighted his spiciest claim, the rest of the letter is more technical and informative while just as entertaining.  I encourage you to read it and then engage in a thought experiment in which You are the designer of an electronic exchange and must balance the needs of a very heterogeneous set of users and stakeholders while ensuring transparency, liquidity, profitability, “fairness”, performance (he references an exchange targeting 100M executions per second) and utterly fail-safe transactional integrity…

I have embedded the full letter below the break…

FIX interface?

Cloud computing looks to have turned yet another interesting corner.   This time the turn leads towards the development of a liquid, fully electronic new marketplace in “spot instances”.

‘Spot‘ means what you would expect it to in the context of trading: the current pricing for immediate delivery of a commodity.  ‘Instance‘ is the atomic element within Amazon’s cloud environment; an instance is the smallest chunk of computing capability which can be provisioned within the cloud.

Amazon is making markets in cores and they’re exposing functionality just as a regular exchange would: both through user interface ‘screens’ as well as programmable APIs.

From their announcement:

Spot Instances enable you to bid for unused Amazon EC2 capacity. Instances are charged the Spot Price set by Amazon EC2, which fluctuates periodically depending on the supply of and demand for Spot Instance capacity. To use Spot Instances, you place a Spot Instance request, specifying the instance type, the region desired, the number of Spot Instances you want to run, and the maximum price you are willing to pay per instance hour. To determine how that maximum price compares to past Spot Prices, the Spot Price history is available via the Amazon EC2 API and the AWS Management Console. If your maximum price bid exceeds the current Spot Price, your request is fulfilled and your instances will run until either you choose to terminate them or the Spot Price increases above your maximum price (whichever is sooner).

embedded optionality

While the inclusion of, effectively, a market data service is neat, probably the most interesting aspect of the initial protocol they’ve designed is that it contains embedded optionality and behaves a bit like barrier options.  That is, when I setup an ‘order’, I need specify a maximum price I’m willing to pay.  When the spot price drops below my max, I get “knocked-into” a contract and instances are allocated to me.  If the spot price rises above my max while I’m running, I get “knocked-out” of the contract and my jobs get terminated.

The intent is to allow for low-priority jobs to be dynamically run whenever pricing drops below a user’s threshold, but the (intended?) consequence is that it adds the delicious and malleable tang of path dependency to these instruments…

secondary markets, FIX, arbitrage..?

Amazon currently controls the market entirely, but it’s not hard to imagine a secondary market evolving.  Given that others are beginning to copy Amazon’s APIs, one can also imagine markets which operate across providers …  perhaps accessed via FIX?…

Who knows?  In the not-too-distant future, we may well be able to implement ‘core arb‘ strategies…or make markets in cores… or find that we can effectively hedge with disciplined exposure to the ‘core market’ or …