Stock market prediction using natural language processing

Abstract

A method of using natural language processing (NLP) techniques to extract information from online news feeds and then using the information so extracted to predict changes in stock prices or volatilities. These predictions can be used to make profitable trading strategies. Company names can be recognized and simple templates describing company actions can be automatically filled using parsing or pattern matching on words in or near the sentence containing the company name. These templates can be clustered into groups which are statistically correlated with changes in the stock prices. The system is composed of two parts: message understanding component that automatically fills in simple templates and a statistical correlation component that tests the correlation of these patterns to increases or decreases in the stock price. The methods can be applied to a broad range of text, including articles in online newspapers such as the Wall Street Journal, financial newsletters, radio & TV transcripts and annual reports. In an enhanced embodiment of the system statistical patterns in Internet usage data and Internet data such as newly released textual information on Web pages are further leveraged.

What is claimed: 
     
         1 . A method of predicting investment behavior using trading software having a data analysis tool implementing natural language processing and an investment predictor implementing an investment prediction model, said trading software being executed by a processor so as to cause said processor to implement the steps of:
 said processor extracting information from news media relating to a particular investment using said natural language processing to parse or pattern match on words in said news media to identify natural language text describing activities or announcements of said particular investment that is in or near sentences containing a name of said particular investment and to automatically fill templates with said natural language text;   said processor using a clustering algorithm to cluster at least some of said templates into groups that are statistically correlated with changes in investment price of said particular investment;   determining a statistical significance of said changes in investment price of said particular investment based on information in said clustered templates; and   predicting changes in price of said particular investment based on new information about said particular investment if information of the type included in the new information has in the past caused a statistically significant change in the investment price in said particular investment.