Big Data is the new Big Bang. It is a buzzword that has exploded into every discipline that process expansive data sets. Computing, medical sciences, biology, and advertising are adjusting their methodologies to harness the ever expanding processing power that is available. In finance, the data is omnipresent: exchanges generate tick data, news services deliver real-time machine readable newsfeeds, and Internet traffic analysts produce sentiment feeds.
Even the United States Federal Reserve has seemingly jumped on the big data wagon and is producing more data points than ever before. All of this is a great thing.
Big data generates transparency, it allows people to make timely informed decisions, and helps predict the future based on current trends. The challenge, of course, lies in what to do with all this data. Transmission of the data, data storage, and data analysis are just three key areas where many traditional approaches do not apply. And big data finance is becoming a big business. Financial companies of all sizes and stripes are investing into big data technology, acquiring newsfeeds, data analytics, databases, network switches, leasing space in data centers, and hiring consultants to help make sense from the data that surrounds us. The investments further fuel research in the field. And as a new conference devoted to the subject (http://www.bigdatafinanceconference.com) shows, the innovation in big data finance is evolving at a mind-boggling speed, surpassed perhaps only by the speed of growth of the data itself. As financial industry gradually recovers from the crisis, the adoption and demand for further innovation is only going to continue to grow.
Big data finance resides at the nexus of three academic disciplines: Finance, Computer Science, and Mathematics. In addition, Electrical Engineering plays greater and greater role, as big data requires specialized hardware to process and interpret the data efficiently. Traditional finance guides data analysis by focusing the analysis using common principles. The latest innovations in finance, such as high-frequency trading, algorithmic trading, and market microstructure initiatives help transform the data into actionable revenue-generating results. Computer science brings sophisticated new tools to process the data. Mathematics, of course, helps distill meaningful conclusions from reams and reams of unstructured incoherent data.
Taken in isolation, each of the sciences may not produce quality results. Mathematical conclusions, divorced from financial reality, engender spurious, or random, inferences. Computer science, while capable of streaming and storing the data, alone cannot function beyond most basic predictions. Traditional finance was not developed to process fast and furious onslaught of data. Only together, the three disciplines of mathematics, computer science, and finance can deliver solid foundation for big data applications, under the umbrella of a new science, Big Data Finance.
By now, big data finance is used in almost every corner of financial services. Mortgage desks archive and process gazillions of data points describing their loan customers. Foreign exchange traders seek to interpret a variety of fundamental news coming from every corner of the world. Derivatives traders create forward-looking models that accept an ever increasing number of factors and generate, as a result, more precise and realistic scenarios than ever before. Retail bankers parse through the minute data of their customers to develop better products and more targeted and profitable offerings. All of these initiatives are supported by thousands of specialists in information technology, data analysis, and mathematics. Big data finance is here, it cannot be dismissed, shaken off, or forgotten. It is knocking on everyone's door. When are you doing about it? Leave a comment below.