Saturday, January 16, 2010

Update on the Stock Analysis Project

From text editor to IDE

I started learning a language called Python about 2 years ago after coming back from some traveling. The reason for that was because I fell in love with a dynamically typed, compiled on the fly interpretive language called Matlab which I used for a variety of scientific calculations.

The language had some limitations in the sense that it was slow and I hadn't figured out a way to run multi-threaded programs when it came to doing downloading and parsing html data. The bottle neck when dealing with data in this case is not the PC, but the speed at which the online servers can pump data into my computer. To alleviate this problem, I moved onto a different language called python.

In following my pursuit of a language with dynamically typed variables, easy methods to handle arrays of data and a large breath of built in packages, learning to use this language took some time at first but it would only take me a week to make my first real program: a program to download and compare price vs measured performance of a CPU with published CPU data online.

All of this was written without the luxury of an integrated development environment, which meant a text editor, compiler and debugging environment. I spent the better first part of 2 years writing code in the equivalent of notepad. Maintaining code in a plain text editor is a pain but I stuck with it since it was simple and I didn't want to have learning how to use a IDE getting in the way of learning a new programming language.

As of last month, I've finally installed a free IDE called Eclipse and the Python plug-in from pydev. Eclipse is an open software IDE written in Java designed to run on both Windows and Linux. I still remember when Microsoft Visual Studio was the the big software suite costing a few hundred dollars. Now all of this for nearly any language is available for free. I am impressed with how far things have come in the open source community.

It took a few weeks to learn the basics of learning how to program with the IDE and I am getting more comfortable with it; and the benefits are great. Code organization has improved, automatic look up of functions is awesome, a variable spell checker with a real time code parser in the background to highlight bad programming syntax cuts down on development time. The debugger is useful as I can trace how variables are accessed and if the right data is going to the right place while the program is running. I still have many things to learn.

For the past 4 weeks I have been migrating the stock market analysis code into a format that makes it easier to manage with the IDE. I used to keep all my code in a single directory in different importable files, this made things messy and hard to find. I've finally reorganized and divvied out the code into modules which made things easier to manage and track problems. While moving the code over I've also been rewriting significant chunks of code to make things cleaner as I've found some really ugly bits of code.

What are my objectives with this program?

There is a lot about the stock market that I don't know. I blew a lot of cash after the crash last year, I had good predictions about where the economy was going but was terrible on the execution. Being able to make economic predictions doesn't necessarily mean that you are going to make good trades. My biggest mistakes were leveraging myself too much and was negligent in determining a good price and contingencies. In order to correct these mistakes and fill my informational gap, I am developing a stock analysis software to help me in evaluating stocks and help make better decisions.

Comparing against having my own analysis program compared against online stock screeners, the online screeners are limited to the search parameters and conditions provided. The problem with looking at search terms like PE, EPS and asset/debt ratio independently is that you can't combine the information into a more comprehensive analysis function.

For example, what is the true/implied value of a company's share being sold on the market. This is important when determining when a share is expensive or cheap. When a hot tech stock is jumping in price, at what point do you say this share is over-priced or still have more room to grow? The importance of answering these questions is directly related to the success and failure of one's trading strategy. Because if you don't know when to get in and out then you're asking for a lot of trouble.

This is the biggest challenge when it comes to the stock market; knowing how to valuate the price of a stock. When a person says that the stock is expensive or cheap, I want to know in relation to what data and the direct calculations instead of hand wavy analysis you might see on mad money. For those that might be interested in taking a look at success rates of prominent stock gurus, take a look at the continually updated list at CXO. Most of these gurus are close to the 50% range with a few in the 60% and 40% range. I might as well flip a coin and pray, but I'd rather do better.

What I want to be able to do with the software I am developing is come up with a technique to calculate the intrinsic value of a share based on financial data provided from a company to determine a fair value of the share and market expectations of where the company is going in the next quarter or few years. If the intrinsic value of a stock is at price 'x' and the market is trading at 'y' then the market is expecting that the worth of the company to rise in the future to justify price 'y'.

I have found no screener that calculates the intrinsic value of a stock and I have taken it upon myself to develop a program to test different valuation models to find something that works. Of course, no valuation model is perfectly correct and it would be nice to back test against historical data to find something that works most of the time. As a human being, I am limited by the amount of data I look at and perform calculations, I've decided to enlist the help of a computer and hopefully be able to make well informed decisions in the future.

Programming objectives

Most of my programming experience comes from numerical analysis of experimental data sets. In most of these projects, I would just need to write a very simple program to take measurement data and another program to perform analysis. Unfortunately, I've never needed skills to manage complicated data structures, program modules and software architecture. Making good software design decisions is important to easily add new features without requiring a complete rewrite of code. There are still many things that I still need to learn.

In list form here are a few of those programming objectives:
  1. Develop skills in software architecture
  2. Become familiar with dealing with large and complicated data sets
  3. Learn what kind of data structure to use and when
  4. Develop robust code that can automatically handle errors
  5. Learn module unit testing techniques
  6. Improve programming speed
  7. Become more familiar with using IDEs
  8. Learn how to make a distributable software package
  9. Learn how to make a user interface (meaning: also learning event driven programming)
Knowing good software design is a blessing as it means taking a complicated programming problem and making it simpler. I've just managed to decide on the implementation of a simple analysis engine design. Instead of writing a hard coded loop for every analysis run, I can just import functions and requirements to the engine and let it do the work instead. Now I don't have to keep track of what code when where and I am starting to see the beauty of functional programming, which was a topic that used to go over my head.

I still have much coding work to do but hopefully I'll be able to produce some interesting results in the future.

No comments: