One Big Table (and chair)
Last time I described the trajectory of my research into using hdf5 for large amounts of tick data. This time I describe the basic design of the prototype I implemented and some of its performance characteristics.
EMS Internals, market data, open-source software, technology
One of the nicest things about the holiday season (Happy New Year, btw) is that it provides a lovely opportunity to spend some quality time with a project that’s a bit more exploratory than might be meaningfully undertaken while trading in lively markets.
A number of months ago, I mentioned using HDF5 to manage tick data as RDBMSes just aren’t up to the task and specialized Tick DBs are absurdly expensive. While I’d spent some time exploring this idea through the fall, I never had a discrete chunk of time to really explore the technology beyond determing that its Java interfaces weren’t production-worthy. This meant that we’d have to drop into C to access the functionality we’re interested in and that we’d have to come up with our own bridge out into Java for access by StratBox while StratCloud could access it directly.
Below, I describe what I’ve learned through my holiday geek-spelunking-trek including some timings on various configurable characteristics of HDF5 (e.g., compression and “chunking”).
EMS Internals, market data, open-source software, post-trade analysis, technology
I’ve written here about exchange simulation in service of back-testing trading algorithms and briefly mentioned the difficulties of simulating the behavior of an order book. I just came across “A stochastic model for order book dynamics” by Cont, Stoikov and Talreja of Columbia and Cornell financial engineering groups. (I’ve also saved a local copy of the paper here.) While their focus isn’t on simulation for the purpose of back-testing but on probabilistic reasoning in real-time for high-frequency strategies, they illustrate a variety of models/methods for such reasoning. The equation depicted above is part of their description for reasoning about the likelihood of being able to make the spread in a stat arb strategy which places orders simultaneously at the bid and ask. It’s very technical, but interesting even if only to illustrate the kinds of tools being wielded in the service of algorithmic trading!
Our algorithmic trading platform, StratBox, features a unique strategy component model that supports the modular development and re-use of “pieces” - we call them StratParts - of a quantitative trading strategy. StratParts expose metadata which can be manipulated by a human or software agent (e.g., a trader, an optimizer, a regime-switching protocol). A StratPart might be an entire strategy, a risk management component, a graphical or reporting component or really whatever a trader might envision. StratParts can be composed within the StratBox GUI to create a strategy which can be tested, analyzed and executed. Naturally, users can create their own StratParts which integrate seamlessly with the environment. Read more…
Among the more challenging questions we face when describing the Puppetmaster environment are those like “how do you create new proprietary trading strategies within the environment?” It’s a difficult question because of expectations - people want to hear about some super simple scripting language that any non-technical person can immediately learn and be up and algorithmically trading in no time. A few platforms intended for retail users offer such things - one is even appropriately named easy language. When researching approaches for our system, we spent some time learning easy language and found that it in fact did make easy things easy!
The problem was that it also made sophisticated things impossible.
This led us to pursue another, more powerful, approach for which we are currently seeking a patent.
EMS Internals, events, open-source software, strategy development, technology
It turns out that one can actually sculpt or engineer randomness. Further, this ability can shed remarkable light on otherwise mysterious phenomena - like the value of an option or the performance of a complex trading strategy. The applet above is governed by the equation (for Geometric Brownian Motion) below. It provides you with the ability to produce a random stream with three modifiable characteristics: an initial value (in this case fixed at 100), volatility (or ‘diffusion’) and expected return (or ‘drift’ or slope). Every five seconds it will generate a new path based on the current settings. 
Hopefully, this applet provides a little insight into the machinery of a powerful algorithmic trading tool: monte-carlo methods. Read more…
EMS Internals, monte-carlo methods, open-source software, performance analysis
There are many conceptions of our world and its underlying nature, each seemingly more creative than the last. One of my favorites - adopted by all serious fans of Star Trek as well as some physicists who’ve run rather far afoul of friar Occam’s good rule - posits the existence of a multiverse. There are various such stories, but the basic idea is that at every quanta in time the universe partitions itself off into an infinitude of parallel instances of itself, each representing a different branch of realized possibilities. In this one, I decided to become a surfer while in that one I went into finance and in that one I led the US to its first-ever world cup victory. While we only experience the particular universe we happen to be situated in, the multiverse represents the simultaneous execution of all possible universes. Under this view, we really do live in the best of all possible worlds.
This idea has surprising utility. While its cosmological implications might represent the extreme psychedelic fringe of the continuum, at the other ends lies actuarial science, the foundations of risk management and, yes, a variety of option pricing techniques.
As we’ve seen, a trading strategy has two special relationships - one with an OMS and one with a market data source. Interesting things can be done when we swap out one or both of these collaborators. First let’s consider the OMS.
From the perspective of the strategy, the OMS is really a proxy for an exchange (or perhaps multiple exchanges) or more generally liquidity sources. The relationship is characterized by messages which are passed back and forth between the participants. A strategy might send an order to which an OMS might respond with an acknowledgment, a fill, a reject etc. If instead of partnering a strategy with an actual OMS we partner it with a piece of software that behaves like one but doesn’t actually touch our accounts, that is - an exchange simulator, then we can take our real-time trading activity and convert it to paper-trading activity. Suddenly, we are free to experiment with even foolish risks because no money is being placed in harm’s way.
I’ve zipped through a whole bunch of algorithmic trading stuff over the last several posts. Judging from the expressions on the faces of some very smart and experienced wall st professionals when I’ve covered the same material, it’s not obvious stuff. So, I’m going to back-track a bit by describing what each of the pieces are and how they fit together.
We’ll start with a trading strategy. What is it in an algorithmic trading context? Conceptually, it’s a simple enough thing. It’s something that can study market data and can manage positions. That’s really it. Its “eyes” monitor one or many sources of market data and its “hands” are poised over an Order Management System. There’s obvious potential for complexity with this simple definition and we’re even leaving out potentially important elements, but for now we’ll take this to be pretty much it.
