Pattern Recognition, Money, Algorithms, and Web 2.0

By Kevin Farnham (107 posts) on February 7, 2008 at 3:14 pm

Day one of the Money:Tech Conference covered many different aspects of modern investing and technology. While it's clear that computation algorithms and automation have become a significant component of institutional investing, many in the audience, along with some speakers, identified themselves as either being on the "money side" or on the "tech side."

Tim O'Reilly opened the sessions by talking about the genesis of the conference. A key phrase (for me) in his discussion was "pattern recognition." Back in 1987 I had developed my first mathematical models for investing in the stock market, applying my experience in developing scientific modeling and simulation software to a new data set (stock market prices since the late 1960s). Back then, the market was very different from what it is today in several important ways. In the late 1980s:

Back in the late 1980s it was therefore possible for me to invent methods where I could trade successfully using only historical end-of-day prices as my database.

In one of my favorite sessions today, Larry Tabb's "Search, Dark Pools, and Disappearing Traders: A Financial Technology Roadmap", I found out that floor traders are nearly an extinct species. The short description of Larry's talk is:

Dark pools, algorithms, disappearing traders--the markets are being transformed by technology. Capital is increasingly disappearing into markets where immense computational power is required to find trading parties; and algorithms are replacing traders at many funds.

I learned from Larry's presentation, the presentation by David Leinweber ("If You Had Everything Computationally, Where Would You Put It, Financially?"), and several others, that a few trends I had noticed in my own studies (which ended in the late 1990s) have continued inexorably:

The result of all this shrinkage is a marketplace where profit margins for professional investors are increasingly thin. In 1988 I was often able to take a 3/8 spread ($0.375) between bid and ask prices, apply my models, enter a position on one edge of the spread, then complete a profitable trade within a day or two on the other edge of the spread.

So, what the methods are institutional investors applying to try to outperform their competitors (or try to at least stay even with them)? Algorithms are an enormous part of the equation. Algorithms that often run on very high powered systems, crunching large amounts of data (including near-real-time financial information).

And Threading Building Blocks fits into this how?

Multicore is bringing vastly increased amounts of computing power to desktop computers. Instead of running models on high performance computers, it will soon be possible to run fairly complex algorithms on many core computers, with the algorithm code multithreaded using technologies like Threading Building Blocks to maximize performance and minimize computation time.

Some of the conversation today at Money:Tech was about bringing some of the intelligence that Wall Street investors have to "Main Street" (that is, to everyday, non-professional investors). In some cases, this would happen through harnessing the collective intelligence of the crowd (i.e., through Web 2.0 technologies).

But the upcoming availability of powerful multicore systems at an economical price, the emergence of new multithreading techniques, and the growth and development of open source algorithmic libraries of many types -- this combination raises the prospect of Main Street investors applying (and potentially creating) fairly sophisticated models of their own.

If I could succeed at mathematical investing using models I ran on a single inexpensive PC in the late 1980s and 1990s, surely today's hardware, open source software, and programming techniques like TBB, can be applied to enable "Main Street investors" (and perhaps also professional investors) to find an edge in today's market.

Kevin Farnham, O'Reilly Media, TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB

Categories: Open Source, Parallel Programming, Threading Building Blocks

Comments (0)

Trackbacks (0)


Leave a comment  

To obtain technical support, please go to Software Support.
Name (required)*

Email (required; will not be displayed on this page)*

Your URL (optional)


Comment*