FIX interface?

Cloud computing looks to have turned yet another interesting corner.   This time the turn leads towards the development of a liquid, fully electronic new marketplace in “spot instances”.

‘Spot‘ means what you would expect it to in the context of trading: the current pricing for immediate delivery of a commodity.  ‘Instance‘ is the atomic element within Amazon’s cloud environment; an instance is the smallest chunk of computing capability which can be provisioned within the cloud.

Amazon is making markets in cores and they’re exposing functionality just as a regular exchange would: both through user interface ’screens’ as well as programmable APIs.

From their announcement:

Spot Instances enable you to bid for unused Amazon EC2 capacity. Instances are charged the Spot Price set by Amazon EC2, which fluctuates periodically depending on the supply of and demand for Spot Instance capacity. To use Spot Instances, you place a Spot Instance request, specifying the instance type, the region desired, the number of Spot Instances you want to run, and the maximum price you are willing to pay per instance hour. To determine how that maximum price compares to past Spot Prices, the Spot Price history is available via the Amazon EC2 API and the AWS Management Console. If your maximum price bid exceeds the current Spot Price, your request is fulfilled and your instances will run until either you choose to terminate them or the Spot Price increases above your maximum price (whichever is sooner).

embedded optionality

While the inclusion of, effectively, a market data service is neat, probably the most interesting aspect of the initial protocol they’ve designed is that it contains embedded optionality and behaves a bit like barrier options.  That is, when I setup an ‘order’, I need specify a maximum price I’m willing to pay.  When the spot price drops below my max, I get “knocked-into” a contract and instances are allocated to me.  If the spot price rises above my max while I’m running, I get “knocked-out” of the contract and my jobs get terminated.

The intent is to allow for low-priority jobs to be dynamically run whenever pricing drops below a user’s threshold, but the (intended?) consequence is that it adds the delicious and malleable tang of path dependency to these instruments…

secondary markets, FIX, arbitrage..?

Amazon currently controls the market entirely, but it’s not hard to imagine a secondary market evolving.  Given that others are beginning to copy Amazon’s APIs, one can also imagine markets which operate across providers …  perhaps accessed via FIX?…

Who knows?  In the not-too-distant future, we may well be able to implement ‘core arb‘ strategies…or make markets in cores… or find that we can effectively hedge with disciplined exposure to the ‘core market’ or …

This isn’t surprising as I wanted the system to support all of the activities associated with our algorithmic trading.  It turns out that that’s a lot to ask of a system.  It also turns out that you learn as you go and so the system continues to evolve.  A few years ago I’d posted about the basics of a strategy container and in this post I’m going to come back to this topic and describe some of the layers of code and thought developed since then.

First, let’s consider the role of a strategy container.  Its job is to intermediate between trading strategies and the external environments with which they interact.  It must also provide services that strategies can use (e.g., position management) and that it wouldn’t make sense for each strategy to re-implement.  In the past I’ve focused on the former responsibility of adapting strategies to external environments.  Why is this necessary and interesting?  Because it allows us to take the same exact strategy and run it live, or in simulation or in backtest, etc.  Interesting and necessary, but not what I want to focus on this time.  Instead, I want to look at the services provided to strategies; the ‘ecosystem’ a strategy container provides in the hope that strategies might flourish within it.

We’ll go from the bottom up.  At the bottom you have one or more pipes of market data coming in.  You might also have news feeds, weather feeds or other such things coming in and we’re just going to conflate them all and say “market data.”  Still at the bottom, you also have one or more two-way pipes with your oms or broker or directly with exchanges or with an exchange simulator.  Within Stratbox we call this abstraction an “ExecutionPlatform” because it’s conceptually where trades get executed.  The lingua franca for execution platforms is FIX, so the baseline object model of a strategy container is likely going to look a lot like messages described by the FIX spec.  Here we have orders, executions and the like.  Within Stratbox we’ve implemented an exchange simulator, a QuickFIX-based FIX interface and a couple broker-specific APIs and they are all of type ExecutionPlatform.  To a strategy, they all look the same.  Likewise with market data.  Within the strategy container, we provide a publish-subscribe model wherein any kind of market data can be subscribed to regardless of the ultimate source of the data.  The intention is always to intermediate between the strategy and its external environment.

the baseline object model

Great.  But now I want to write a trading strategy.  What’s that going to look like?   And this is where things get increasingly interesting as we get to decide what kind of facilities we’re going to provide.  We also need to manage concurrency in some fashion as trading is an intrinsically asynchronous activity.  For this lowest level of strategy, we’ll say that each market feed is handled by a thread which timestamps and enqueues a “Quote” for later consumption and redistribution to relevant subscribers by a MarketDataService within the container.  Likewise, each execution platform will be serviced by a thread who similarly enqueus “Execution” objects from exchanges real and simulated.  The threads handling executions should likely have a higher priority than the threads handling market data.  So, the strategy is naturally exposed to the markets’ asynchronicity right off the bat.  What baseline facilities will our strategy have at its disposal?  Well, it needs to be able to manage orders and positions, so it needs some kind of a “blotter” facility.  With this in place, a strategy can safely assume that it doesn’t need to listen to each execution coming in just to have a correct picture of its book.  It needs access to a MarketDataSvc and it likely also needs access to historical data.  But, staying simple, that’s more or less all we have to provide and we’re going to provide all of these things to the strategy through a single handle: a strategy context.  By swapping out a strategy’s context, we can move the strategy among environments (eg, from simulation to live execution).

With these facilities in place, basic algorithms can be implemented within a strategy and you should be able to test and trade them.

But you’re dealing with very low-level stuff.  You’ll also find that you’re writing the same kind of code over and over.  This inspired the introduction of a layer on top of the baseline object model which I’ve described before: StratParts – a strategy component model.

StratParts – a component model for strategies

With StratParts, we introduced metadata and composition (in the “composite pattern” sense) to our Strategy object model.  A strategy is itself a stratpart which contains other stratparts.  Thus, stratparts introduce a hierarchical structure to a strategy.  Each stratpart publishes a metadata descriptor which is aggregated at the strategy level.  The descriptor contains all of the modifiable characteristics of the strategy and can be changed while a strategy is executing thus giving a sort of grey box capability wherein a trader (or another strategy etc) can modify the behavior of a strategy as it’s running.  Stratparts also create an effective means of providing scoping within your strategy environment.  Stratparts can “see” the activity of other stratparts within the same strategy – they’re all peers in this sense.  This scoping can also be used for allocation of resources across different elements of a strategy.  For example, one stratpart might be allocated 80% of the cash available to the strategy while another stratpart manages the remaining 20%.

Stratparts have proved to be very powerful and we’ve written many of them though we’ll sometimes write strategies as one monolithic stratpart where it makes sense.

Stratparts have a lot of uses but one thing they don’t do is help manage the low-level complexity inherent in trading activities.  This was best described by a trader with whom I’d collaborated.  He’s a reservist in the IDF and uses military metaphors like I use sports metaphors.  He bemoaned the low-level handling of orders required for a very close-to-the-market strategy we were working on and said that:

we need an order like a ’smart’ missile: fire it and forget it

He was right and that inspired the next level of abstraction/support that we built into the system.

’smart’ orders and the tradeflow stratpart

The strategy we were developing was meant to look at a universe of futures spreads and generate all of the ‘cycles’ that might result in an arbitrage opportunity.  Since these are rare at best, we were really looking for circumstances where it looked like we had an advantage based on depth-of-market and various heuristics we’d apply.  Among the functionality that he wanted was an ‘order’ which would act as a limit unless some conditions obtained in which case it might pay the spread or otherwise stop being a fixed limit order.  He didn’t want to handle this inside the strategy but instead wanted to fire these things off and forget about them unless they required attention (eg, if they’re rejected by the exchange or his firm’s risk management checks).  Although we ended up calling these smart orders, they’re really a sort of very localized execution strategy themselves.

To support them, we utilized the same workflow (= state machine) framework we’d implemented for our ExchangeSimulator as we wanted to ensure that these smart orders had a very well-defined life-cycle with a clear set of states and guarantees about transitions among them.  We embedded this workflow engine into a stratpart which supported smart orders operating within this “tradeflow”.  Thus, the extra functionality (and weight/expense) of this functionality is only achieved/incurred when actually required or desired.  This has proven to be a powerful addition to the system and raises the level of services enjoyed by prospective strategies substantially, particularly given the variety of orders we’ve implemented which stand ready for use by any new strategy that might want to employ them.

At this point, it should be clear that we’re blurring the lines between the strategies themselves and the container.  Another way of thinking of it is that one container can have a set of pre-made building blocks that can be applied for families of strategies.  Each one provides a particular grammar with which a strategy’s aims can be expressed.  For very close-to-the-market strategies where very low-level handling of orders is required, the tradeflow stratpart provides a set of really useful facilities, but if I’m just trading with limits then it might not be so interesting.

portfolio-oriented strategies and beyond

I’ve mentioned before that I only care to think of strategies that operate on a portfolio.  As such, we’ve written a good deal of facilities for creating, analyzing and manipulating portfolios within strategies.  By using these facilities within a tradeflow stratpart, I can express quite complex strategies relatively simply by using portfolio analytics to determine my current ‘model’ portfolio and then using smart orders to most effectively transition me from my current state to my model state.  For the kind of trading I do, this provides a rich ‘vocabulary’ for the development of strategies.  All the same, my brain is having a tip-of-the-tongue moment as I feel that there’s an entirely different model possible for portfolio-oriented strategies just at the periphery of my imagination…  oh well, there’s always something new to explore.

Different trading styles, perspectives or trading problems are undoubtedly better served by different sorts of facilities.  I’m finding that just as language constrains and shapes thought, the strategy container that you employ shapes and constrains the kinds of strategies that you can implement.

If you have any ideas about such facilities for any kind of trading you do, I’d love to hear them.

–

a note about Stratbox

Although I’m talking about our system in this blog, we’re not marketing the system and happily “just” use it for our own trading activities.  We had looked into marketing the system previously but ultimately feel the same about selling stratbox as selling strategies…

One of these days I’m going to give an overview of all the excellent open-source software I use on a daily basis. Until that day comes, I’ll observe that finance remains one of the big areas where open-source software has made relatively limited inroads.

Two production-quality packages fight that unhappy state: QuantLib – a comprehensive framework for quantitative finance – and QuickFix – a full-featured FIX engine. Both are C++ libraries and both provide very nice interfaces to facilitate integration with other languages, including Java. QuantLib is a big and complicated library and integrating it with Java is not totally obvious. Below, I’ll describe how to build and use QuantLib from Java.

These instructions are based on a unix installation. I’m not really a windows developer and don’t have all the shiny tools that windows developers use, so it’s not an area of focus for me. That said, I have managed to build QuantLib under windows by using MinGW+MSYS but it wasn’t terribly easy and I don’t currently have a working installation, so I won’t cover that here. If this is your aim, don’t be dismayed as it is possible and it had all the functionality I enjoy under linux.

Using QuantLib from Java (on linux)

• Build QuantLib
  • Requires a working version of Boost. This may prove to be the hardest step of all and you’ll need to use the ample documentation provided by the Boost team.
  • Once you have a working copy of Boost, building QuantLib should require little more than

  sh autogen.sh
./configure

  make
sudo make install

• Build QuantLib-SWIG
  • Requires a working copy of SWIG. Again, look to the SWIG instructions, but it should be easy.
  • Once SWIG is available, building the QuantLib/SWIG interfaces should only require:

  sh autogen.sh
./configure \

  –with-jdk-include=${JAVA_HOME}/include \
  –with-jdk-system-include=${JAVA_HOME}/include/linux
  make -C Java
sudo make install

• Now you’ll have a Jar file with all of the SWIG/JNI stubs in it available in /usr/local/lib/QuantLib.jar. Add this to your classpath.
• Programs which call QuantLib functionality will need to have the LD_LIBRARY_PATH set. This can be done by invoking the vm with something like:

-Djava.library.path=/usr/local/lib

• Programs which call QuantLib functionality will also need to explicitly load the QuantLib libraries. This can be done with something like the following static block appearing before your main method:

static { // Load QuantLib
try { System.loadLibrary("QuantLibJNI"); }
catch (RuntimeException e) { e.printStackTrace(); }
}

• That’s it. Now test your configuration by running the examples in Quantlib-SWIG/Java/examples.

It’s worth understanding how Quantlib is being used from java. SWIG is creating a JNI interface into those methods within Quantlib which have been exposed through their declaration in the swig *.i files. These files are found in Quantlib-SWIG/SWIG and they determine what functionality from Quantlib will be available to you. You’ll likely need to get familiar with a subset of those files that you care about. If you find that some functionality you care about isn’t exposed in those files, you may need to expose it yourself.

There’s a learning curve, but it’s worth traversing so you can get at all the rich functionality so many smart people have put together.

The booths we checked out included a few of the big sell-side firms shilling their creatively-named execution quality algos and their white-labeled oms/ems offerings. They had, by far, the best shwag (I should thank Merrill for their nice umbrella and commend citi for the very knowledgeable lady at their booth) but their algo offerings weren’t very interesting to me as they aren’t of the alpha-seeking variety.

My favorite exhibit from an innovation perspective was actually a hardware offering from a firm named tervela which makes, essentially, a complex-event-processing (CEP) engine implemented in hardware. Very cool. I can’t imagine why any big shop would use any other sort of CEP solution. I had heard of them before as a guy I knew had joined them before their product had been delivered, so they were already on my radar, but seeing the hardware running was a big next step. Given the nature of their offering, it actually seems to me to have a bigger market in military than trading applications, but in both spaces it looks pretty compelling.

We looked at a few of the ems/algorithmic trading vendors and they didn’t offer any surprises but still had points of interest for us. The three systems we looked at were portware, tethys, and flextrade, and they can all credit themselves with knowledgeable, engaging and enthusiastic presenters in their respective booths. The platforms each had their own personalities and strengths. Portware had a very pleasant looking user interface and was, to me, most intuitively laid-out. The tethys platform seemed to have a real affinity for options and had sophisticated analytics to support that inclination. And flextrade just seemed like the most solid and mature of the platforms, but it’s hard to know such things from a trade show drive-by.

I was very interested to see that each platform offered user-interface level programmability via scripting of the sort I’d spoken of here but with a big difference. Instead of providing a scripting language, portware and flextrade offer users the ability to bring up an editor and add columns to tables via the platform’s native programming language – java and c++ respectively. Whew – can you imagine your average end-user whipping out some c++ to add a column. Mutant super users, maybe!

Overall we were very impressed by these trading systems. In particular we were happy to note that their strengths aren’t ours. If you want to use, say, lehman’s super-ginsu execution strategy (I’m making up that name), all of the platforms have the order available as a custom FIX order type which you can easily select from a pull-down menu. So their integration with a multitude of sell-side execution-quality algorithms is very complete and seems well-done. But their offerings are very limited compared to our own when it comes to the development and analysis of alpha-seeking strategies. They’re really not even close and it seems clear that their emphasis on the resource-intensive efforts of connectivity and broker integration has allowed us to create the class-leading platform for alpha-seeking strategy development.

So it was good to see the state of the industry and it was especially good to see our niche persisting at trade tech.

I’ve zipped through a whole bunch of algorithmic trading stuff over the last several posts. Judging from the expressions on the faces of some very smart and experienced wall st professionals when I’ve covered the same material, it’s not obvious stuff. So, I’m going to back-track a bit by describing what each of the pieces are and how they fit together.

We’ll start with a trading strategy. What is it in an algorithmic trading context? Conceptually, it’s a simple enough thing. It’s something that can study market data and can manage positions. That’s really it. Its “eyes” monitor one or many sources of market data and its “hands” are poised over an Order Management System. There’s obvious potential for complexity with this simple definition and we’re even leaving out potentially important elements, but for now we’ll take this to be pretty much it.

The market data might be a simple stream of “bars” or “candles” representing a summarized Open-High-Low-Close-Volume (OHLCV) data. It might be a stream of time and sales tick data. It might include information on the market depth or order book. It might be many such streams of compressed or direct data. Maybe it comprises news as well and maybe some kind of layer – like a Complex Event Processing (CEP) engine – helps make sense of the streams as they arrive. Whatever. Conceptually, it’s one or more streams of data that the strategy receives. How it uses this data – whether to annotate a complex pre-existing financial model of the universe or to simply seed a random number generator which flips a conceptual coin on whether to buy, sell, hold or fold – is not important at this point.

When the strategy decides that it actually wants to do something, it interacts with its OMS. OMSes predate algorithmic trading and come in a variety of flavors. On the sell side, an OMS might be a server system which “listens” to clients’ orders and routes them to various exchanges or counter-parties and then echoes back executions, cancels, etc when appropriate. I’ve built and worked around such beasts, and they can be complex and interesting but this isn’t really what we’re talking about here.

We’re more concerned with a buy side OMS which will typically sit on a trader’s workstation and provide access to one or more markets, a trading blotter, and possibly various ancillary functions including portfolio management and analytics, risk management, reporting, compliance and regulatory functionality and potentially much more. But the basic function of the OMS is to provide a means of managing positions: entering, cancelling and modifying orders and creating and updating positions on receipt of execution reports. Such client-side OMSes typically will have a Graphical User Interface (GUI) which is frequently quite sophisticated and might include all sorts of bells and whistles. Increasingly, OMSes are opening themselves up to programmatic interaction through proprietary Application Programming Interfaces (APIs) or through common standards like the FIX Protocol.

This is how the algorithmic trading strategy can interact with the OMS – through some kind of an API. This, incidentally, is also how it will interact with the market data – it will typically subscribe to a set of data and then passively receive notifications of changes – through an API. So, a trading strategy will have at least a subscription-based relationship with a market data service and a close working relationship with an OMS. It may have many other such relationships, but these two are the minimum necessary.

It’s worth noting that this basic algorithmic trading setup is also what’s required for a wetware trader – in all cases you need to know where markets are trading and you need to be able to place orders and receive responses. As OMSes have opened up to programmatic interaction, a new kind of software has emerged – the Execution Management System (EMS). EMSes come in a variety of flavors but they’re important for our purposes because an algorithmic trading strategy needs a place or container in which to do its thing. This place is the EMS. The line between the OMS and EMS is blurry, but the OMS provides the baseline functionality (and maybe a whole lot more) while the EMS will typically rely upon an OMS and provide some special, specific functionality. Thus, most algorithmic trading products are EMSes in this sense.

Now that we’ve covered a basic trading strategy and its primary collaborators, we’re positioned to look inside a strategy container and try to make sense of what it’s doing and how it does it. Which is what we’ll do next time.