Hack the market

poor Jorge wasn't ready...

In this post I’m going to revisit some of the topics discussed in the recent ‘containing a strategy‘ and ‘multi-strategy trading with regimes‘ posts, focusing on the process of assembling a strategy and its context in preparation for its launch into any of a variety of modes.

I recently realized that – from the perspective of a strategy container – the process of walk-forward testing is remarkably similar to the regime-switching model we’d discussed previously.  Up until now, I’ve employed walk-forward testing in an ad-hoc manner by taking an existing strategy and then writing a little driver very much like a unit-test scaffolding which would walk the strategy forward, permuting parameters based on previous performance.  Not a general solution, but straight-forward as I employ the strategy parameter optimizer from stratbox in this kind of a toolkit use-case.

I sat down to write one of these walk-forward scaffolds yesterday and started to think about how I could generalize the solution and roll it into stratbox’s GUI and it occurred to me that I could likely kill two birds with one stone…

walk-forward testing

–

I imagine there are different ideas/implementations of it, but for me walk-forward testing is the case where you are using repeated historical parameter ‘optimizations’ to dynamically modify the parameters on a back-tested strategy.  For example, say we have strategy X with parameters p1 and p2.  Let’s say we want to walk-forward test it from the beginning of the year until now.  We’ll need a look-back period for optimizations – say 20 trading days – and a frequency with which to apply the results of our optimizations – say every 5 trading days.  So, on a weekly basis we’re going to look back at the prior (rolling) month performance and adjust our parameters to match the best performer.  How you define ‘best’ is your business but I’ll use something simple like sharpe here.

Continuing with our example, we’ll say that p1 will be permuted 3 ways with values from { 1, 2, 3 } and p2 will be permuted across 4 values { “a”, “b”, “c”, “abcd” }.  So we have a basis of 12 strategies for optimization and we’ll call these O1…O12.

To perform a walk-forward test on X from 2009/01/01-now, we will begin a parallel back-test on O1..O12 starting 20 trading days before Jan1.  On Jan2 we will look at the prior ~month’s results and pick the strategy who had the highest sharpe ratio and clone him and start testing him “walking forward.”  Each week, we’ll repeat the process except that instead of cloning the winner as we did the first time, we’ll just make sure his params are the same as the new winner’s params.

Repeat this process until ‘now’ and you’re done – you’ve performed walk-forward testing on X.

–

Let’s think about what’s happening here.  There are 12 strategies running from ~2008/12/1 and another strategy running from 2009/1/2; all strategies run until now.  There’s also the activity that “we” have done during this process.  What we are doing is the role of a meta-strategy just as we’d seen in the regime-switching case.  The walk-forward meta-strategy needs to watch a set of ‘optimization’ strategies and permute another target (“X marks the spot”) strategy on a regular basis to match parameters with the prior period’s winner.

differences vs. allocating to strategies with regimes

Let’s imagine that we’re going to take the 13 lovingly-named strategies O1…O12 and X, along with the newly revealed meta-strategy we’ll dub “M”, and put them onto a live environment instead of a historical back-test.  Suddenly we’re pretty close to what we’d described previously.  The biggest difference is just the temporality of the strategy context: historical or real-time.  Another difference is that O1…On (let’s generalize just a bit) are no longer necessarily all the same strategy – they may be completely different. X is also no longer a strategy, but a portfolio of strategies:  a replicated subset of O1…On we’ll call X1..Xm.  “Best” is no longer a unary choice but must be rich enough to allow M to support some kind of an allocation algorithm across the X family.

OK, so there are some significant differences.  But from the perspective of the object model on which all of this can be implemented, the differences may be limited enough to warrant tackling together.

ready to launch

Before looking at the workflow of a strategy launch, let’s consider all of the various modes we support with our design:

• Historical: back-test single strategy
• Historical: parameter-based optimization of strategy (decomposes to many back-tests)
• Historical: walk-forward testing of strategy
• Real-time: SIM – run single strategy with real-time market data and an exchange simulator
• Real-time: LIVE – run single strategy with real-time market data and a live connection to OMS/broker/exchange
• Real-time: LIVE+SIM – run two cloned instances using the same market data, but with one routing orders to live venue(s) and the other to a simulator.  This is mostly useful for the development of a simulator or to evaluate the quality of simulation for a given strategy.
• Real-time: optimization – run many strategies, permuted differently across live data but routing to a simulator.  I don’t know what this is useful for, but I find it entertaining… The screenshots from the regimes post were illustrating this mode.
• Real-time: meta-strategy – our mythic regime-switching allocator will be a nice example of this mode.

-

Consider the following sorta-flowchart illustrating the preparation for launch of a strategy.

click for larger version...

The process has 4 or 5 stages depending on whether we’re being “meta” or not – that is, if we’re using optimization of any sort.  The highlighted parallelograms are the concrete end-result(s) for each of their respective steps.

• Assemble Strategy Context – The basic channels of I/O for a trading strategy are its sources for market data and the exchanges where trades can be executed.  The MktDataSrc and ExecutionPlatform abstractions need to be specified.
• Assemble Strategy – Whatever mode we’re looking to employ will require a strategy either to itself go and trade for us or else to serve as the seed or prototype for our activities.  In this step we define that strategy.  For me, this can be a composite process as a strategy is made up of an arbitrary number of stratparts.  The assembly of a strategy can be done manually with a gui or by loading a strategy from a file or other store.  Once the parts are identified, they can be bound together to form a strategy instance.
• Configure Strategy – Once assembled, the strategy needs to be configured.  This may just mean we need to parameterize it appropriately for our use.  If we’re being meta and require optimization then we’ll also need to describe the parameters to modify and the ranges for each.  This gets boiled-down into an OptimizationDescriptor which is composed of a prototype strategy and the parameter/range data required to perform the requisite back-tests.
• Configure Optimization or Meta-strategy – If we’re not running an optimization or utilizing a meta-strategy, this step is skipped.  Otherwise, this is where the optimization and/or meta-strategy should be built and configured.
• Review & Launch – Anytime you might loose a piece of software on an account with real money in it, it behooves you to pause and reflect for a moment.  Now is that moment.  If, after that moment of reflection, both you and whatever risk measures you employ agree that it’s OK, then it’s time to launch.  Happily, this is easy because we now have everything we need.  Again, the meta cases are a bit more involved as the strategies to be optimized or managed must be first generated based on the optimization descriptor we’ve inherited from earlier steps.  Then it’s a matter of assembling the pieces previously selected and configured and away we go.

-

Anytime you might loose a piece of software on an account with real money in it, it behooves you to pause and reflect for a moment.  Now is that moment.

I was taught that writing software is easy if you can express the problem you’re looking to solve and its solution in clear and precise English.  I’m not sure about this being grammatical, clear or precise, but hopefully it’s enough to illustrate a means of uniformly modeling and implementing these beasties across a variety of use-cases.

For me, at least, writing this description is helpful as it forces me to walk through the various twists and turns in some detail before digging into the code.  When I get the integrated walk-forward testing working I’ll try to provide some illustration of its use and will look-ahead to developing a regime-switching example using the models/mechanisms we’ve been discussing.