Thank you for your kindly words and especially for spending the time to share your observations and ideas – they’re greatly appreciated.

Although it seems standard and may well be the correct way of reasoning about such things, I generally resist representing a strategy as a state variable like that you describe (buy|sell|hold). The ‘hard’ reason is that it doesn’t capture the important sets of multi-contract strategies or strategies that are simultaneously looking to buy and sell with limits (like the one in this example); and the ’soft’ reason is that it doesn’t jibe with my intuition. The former might be resolved by introducing richer states, but the latter might well require reverse brainwashing at this point!

That said, your suggestions are creative and welcome. Honestly, it will take a good bit of research for me to fully appreciate them as I tend to learn empirically and I’m not familiar with all the techniques you highlight.

Your comments about applying some form of policy evaluation, and in particular your mention of using agent-based reasoning are also great – if non-trivial! – suggestions. There are always so many different and individually interesting approaches to explore.

As for the taq/tick db, I had heard of that product you mention, but by now am pretty firmly committed to maintaining and further developing my solution as having an end-to-end stack for analysis/development/execution remains a desire near+dear to my heart. I agree re: the PSF, but am currently thinking that the way to acquire it is by sticking my terrabytes into a cloud and using hadoop to
implement a seriously scalable simulation environment… yet another compelling avenue to explore. In the meanwhile, what we have is good enough for many purposes.

Anyway, thanks for your excellent remarks.

Congratulations on your blog, its of a great quality, it truly shares insights I hardly see in the street these days ( or any for that matter). I wish there were more quorum! any way after reading your posting history, here is my feedback.

To potentially enhance your back testing methodology

Use importance sampling

This methodology is typically presented as a method for reducing the variance of the estimate of an expectation by carefully choosing a sampling distribution. In this case, each of your individual strategies can be considered as an stochastic policy that have a non-zero probability of taking any action (buy, sell, do nothing) at any time ( upon the state or regime of the market in your FWD-WALKING)

Policy evaluation is the task of estimating the expected return of a fixed policy.

Every policy induces a probability distribution over histories ( here you could add an ABM simulated price or many).
Then the probabilities associated with the policy combined with the probabilities of the envi-ronment will produce a complete distribution over histories. The returns are a deterministic function of the history. Therefore, we desire to calculate E [R(h)[X] where the expectation is taken with respect to the history probability induced by the policy X.

Because each X is potentially different, each h is drawn according to a different distribution and so while the data are drawn independently, they are not identically distributed.

As an added filter and once you have run your fwd-walking, Try incorporating an heuristic to find the trends of each strategy’s P&L in the history of the last year, with the intention to allocate funds into the losing ones and retreat from the profitable (yes I know it`s backwards), since the classifier will pick up the patterns at which some of the strategies will stops being profitable and or start being losers. Once you get the time laps and IDs of these profitable strategies, you collect them and re run a second FWD WALKING. allegedly the ones that are able to find the abstracted inefficiency will show up as the more robust strategies.

Regarding the database you were looking for managing high frequency terabytes. have you considered

RMD Server™ from modulus? I haven’t tried it but some guys over a prop group mention it.

Regardless of which solution you end up looking for, I would argue that you would want to include a PSF (parallel file system) as it truly increases the reading and writing of thousands of files (symbols) in tandem.