Skip to main content

Historical data: fish or wine?

Some goods can improve over the years and benefit from storing and safekeeping; others quickly spoil and can have adverse side effects when used past their due date. The difference between a good Bordeaux or Barolo and freshly caught fish can be illustrative in looking at historical data.

Good wine ages well and develops new depth, flavors and fragrances over time. There is gradually more to discover and appreciate as the moment of actual production recedes in the past. Fresh fish on the other hand, quite quickly turns into, well, nothing but rotten fish.

Is there something of value in historical data other than simply fulfilling requirements from tax authorities, clients and regulators to keep records? Quite often, historical data is all we have for projecting into the future and the alternative is a direct model of reality calibrated, again, by historical data. New trading book regulation expands the length of history to be used to 10 years, which conveniently puts the last common-sense defying market moves in the crisis of 2008-2009 out of reach.

But historical data can also put you on the wrong foot. It can provide false clues that can derail your processes and infect your judgment. Of course, it all depends on what conclusions you draw from history: recent historical data can give you a false sense of low volatility. Markets undergo structural changes and the portfolio of businesses that make up a large corporate is not necessarily the same as that of 10 or even 3 years ago. Consequently, correlations from the last year may not hold next year, even in the absence of significant market stress. A trivial example is a currency peg that is under attack. The exchange rate versus the dollar will show zero volatility until the dam breaks. A fundamental fact is that risk and scenario management is always calibrated to (recent) history. And history, as we all know, is subject to change

Market patterns of the past do not observe natural laws. I’ve on occasion executed a Rorschach-like test, providing people with an anonymized mix of time series return charts depicting natural phenomena (temperature, wind speed) and stock and corporate bond prices.  An assumption in using historical data is that price dynamics remain the same: that business stay in the same business, that volumes and asset allocation remains the same, that the price of risk remains the same and that the geopolitical landscape remains the same. Ironically, history often teaches us that the reality quite often trumps the imagination. Few people entertained scenarios in which correlations changed as quickly, where volatility spiked as rapidly and where liquidity dried up as completely as in the crisis nearly 10 years ago. History should therefore not so much guide us as inspire us in thinking about the future.

In looking at transaction histories and instrument returns, it is good to realize that the specifics of trading venues, products and execution strategies may change. What will be less subject to change are the underlying drivers: the search for liquidity, the need to construct hedges, trying to extract more information than giving it away when executing strategies, the lookout for trading and investment opportunities whether driven by hidden economics in an investment, regulatory or fiscal arbitrage. Looking instead for those secondary patterns and side effects of fundamentals like this will be more valuable than relying on historical volatility and correlations. In this sense, the wine metaphor may be more apt than the fish one.

Toolsets and techniques that help banks and active asset managers to separate the signal from the noise will be increasingly valuable. History gets bigger and bigger in the sense that the amount of recorded information is growing dramatically. The use of tools and techniques that not only distill patterns out of these mountains of data but that also infer underlying drivers, coupled with human heuristics, will separate the leaders from the laggards in financial services.

We will explore this theme further in future blogs