simulating an exchange
As we’ve seen, a trading strategy has two special relationships – one with an OMS and one with a market data source. Interesting things can be done when we swap out one or both of these collaborators. First let’s consider the OMS.
From the perspective of the strategy, the OMS is really a proxy for an exchange (or perhaps multiple exchanges) or more generally liquidity sources. The relationship is characterized by messages which are passed back and forth between the participants. A strategy might send an order to which an OMS might respond with an acknowledgment, a fill, a reject etc. If instead of partnering a strategy with an actual OMS we partner it with a piece of software that behaves like one but doesn’t actually touch our accounts, that is – an exchange simulator, then we can take our real-time trading activity and convert it to paper-trading activity. Suddenly, we are free to experiment with even foolish risks because no money is being placed in harm’s way.
Now let’s consider the market data source. Imagine a market data source which instead of being subscribed to real-time market data simply played back historical data. If our strategy is partnered with such a historical market data source and an exchange simulator, we introduce the ability to back-test our trading strategies. That is, we can see how our strategy might have performed over some historical market data conditions. This is not without complexities, but all the same, it’s easy to see that in many cases this can be a powerful capability.
Let’s consider some of the difficulties with this approach. For one thing, it’s difficult to model the impact of one’s own trading activity on the market when trading in size or on insufficiently liquid markets. The quality of the market data is also an issue. Consider what our exchange simulator needs to do when it gets, say, an order to buy something at the market. It needs to find the inside ask for that particular instrument. It then needs to decrement the size of the ask with the size of the order and continue in this fashion, “walking” up the order book until the order has been filled. If the order is very large or the market data is limited (e.g., it doesn’t have the full order book or, worse, it’s only OHLCV data) then the simulator has to make some educated guesses about how the market will receive the order and how the order will be filled. In the case of market orders, this is potentially an issue, but it is certainly not insuperable and can be handled with judiciously pessimistic assumptions.
For orders which are waiting for some condition to obtain before being released into the market, the issues get worse and the quality of the simulation suffers. Consider an order to buy with a limit price. If we have full order book data, then we can make note of the size of the queue at our limit price and decrement it as orders fill at that price level. This can be done and if we have good quality market data can even be done well, but it’s a necessarily imperfect solution because we can’t actually have a place in the exchange’s order book without a live order. This means that even in the best case – when we have premium market data and a sophisticated exchange simulator – we might get incorrect results which means that our simulated trading experience might diverge from the experience we’d have on a live market. If we use lower quality market data or a poorly implemented exchange simulator, it’s easy to imagine such simulations being worthless or worse – misleading.
This doesn’t mean that we should abandon back-testing our strategies when they employ conditional order types – it just means that we need to be aware of these limitations. We can gain hands-on experience by launching strategies simultaneously on our simulator and the real market place to get an idea of what kinds of divergence we might see given the particulars of our strategy, market data, and market conditions. Remembering our important result that strategies are options, we can also attempt to use statistical techniques gleaned from option pricing to ensure that we’re getting a realistic view of performance in our simulations by looking at many such simulations.
We’ll look at two such approaches in subsequent posts.
