EDM & Data Sourcing (Part II)
Martijn 2 minutes
In our recent article, we looked at how recent changes in business and regulatory reporting requirements now mean that firms have to be able to both respond to and manage real-time data requests for a wider range of sources. These requirements for increased frequency, granularity and availability of data, combined with changes to the enabling technologies, are fundamentally changing the sourcing step in the traditional EDM process.
In turn, this increasing breadth of data sources has major repercussions on the mastering element of EDM.
Typical data mastering processes are predicated on processing differently structured data sources with common identifiers such as ISIN, CUSIP or SEDOL codes and work against set domains of e.g. currency and country codes and cross-referencing different taxonomies of financial instruments and industries. The mastering process adds value by providing users with different sourcing options and by spotting discrepancies between different sources. This typically helps in efficiently sourcing and in building composite ‘master’ records with – in some cases – data enrichment through proxies and other derivations.
Adding unstructured data sources – from, for example, web crawling sources to mine news and spot corporate events, sentiment analysis, satellite and geospatial information, traffic and travel patterns and property listings – pivots the mastering process from an operational ‘error-detecting’ process to a process oriented towards pattern discovery.
All this data will come with new indices and summary statistics which, when properly accessible, can be analyzed and monitored for investment signals and risk management. However, contrary to processing structured data sources, in this case you don’t know a priori where the relations between sources are. This means finding patterns is not only oriented towards improving operations or reducing risk but also oriented towards improved pricing and new revenue opportunities. Matching of data items will not only take place through common keys but also through spotting the same behavior in hitherto unrelated data or otherwise finding repeating patterns in time, space and across different data sets. Especially for active investment management, the use of non-traditional data sources can help competing and differentiating against passive investment strategies. However, in compliance and risk management too, accessing a broader range of sources can help trigger early warnings on suspect transactions, relationships or price movements.
New financial data management models (or EDMs as we have historically known them) must be able to allow for broad sourcing, efficient exploration (ie. Mastering including new ways of associating different data sets) and exploitation (eg. easy access and distribution) of these alternative sources, both structured and unstructured, to enable organisations to respond to ever shorter reporting cycle times and get new insights from spotting patterns across the spectrum of sources, unlocking data value in the process.