Visualizing the Stock Market

I've finally gotten the stock market analysis program up to a point where I can start plotting data again. There has been some issues with robustness of the program vs the data I am using so I've had to lots of debugging and internal checks to get the code base where it is now. I am confident for the time being that I should be able to start ramping up on writing code to analyze general stock market data to understand the stock market better.

This project has been nearly 2~3 years since starting as a pet project while I was in university. The code base was originally developed in Matlab, but had many limitations when it came to running parallel processes and speed issues. After nearly 3~4 rewrites of the code base, I have finally settled down on the data structures and how to interact with the data. There is a saying when it comes to programming in that you should be prepared to throw out the first few versions of your software.

The program I am developing at the moment is mostly educational, to help me look at the stock market in different ways compared to what is presented at most financial webpages. Looking at the stock market through the S&P 500 and the Dow Jones Industrial index gives a very limited look at the entire stock market and there is a lot of information lost when everything is averaged together. Why look at only 100 to 500 stocks when you can look at nearly 3400 all at once to have an idea of what is going on?

Though I have been participating in the stock market in the last 4~5 years, I never did feel that I understood the big picture of the stock market. The most basic question that I desire to understand is to find out the "right price" for a share in some company. I have read a few investment books and most of them give the general idea that companies with good growing net incomes, low debt and risk factors are good companies to invest in. I generally would have to agree, but what price would you pay for a share in that company? If the share was $20 would you buy it? What about if it was $50 instead? Would it still be a good buy?

The problem with the stock market is that there is no canonical analysis technique to determine the price of a stock. Canonical, being a procedural method when applied by any person result in the same answer. Thus there is no way to say "because condition A therefore the result is B." At best, we may only make predictions, in other words "because condition A result is probably B." Because of this mode of thought, I believe that it is helpful to take a statistical approach to looking at the stock market (though I am against technical trading). In this endeavor, it is necessary to look collect and analyze larger amounts of data an apply decision making strategies and determine how often you are "right" (where being "right is still ill defined at the moment...)

The first few phases of this project has finally been completed, where I have been finally able to get stock market data and plot them for the majority of the listed companies. I've made one plot, which is a Share Price vs EPS plot. One would generally expect that the share price is somewhat correlated to the EPS, which is something we see here with a very wide band going upwards and towards the right in this plot.

Share Price vs EPS

What we also notice from this plot is that there is a large scattering of points to the left side of the graph and with few points scattered to the right. Points to the far left represent stocks that are probably over valued and points to the right are stocks that are possibly undervalued. From what we see in the plot here is that there are many companies out there with a very negative EPS but still high stock price. What has yet to be determined is if the market is expecting those stocks to recover or if those stocks are truly over priced.