Insufficient data vs a bad model

I nearly blew a gasket trying to explain to a co-worker the difference between a calculation model being incomplete and having insufficient data. I was trying to do something a little tricky which these guys have never done, calculate the fundamental parameters of a diode using measured current-voltage data.

The problem I was trying to solve was backwards from the problems that most text books give you. The text books give you the diode parameters and maybe a resistor in the circuit and you had to go and calcualate the current flowing through the circuit at some voltage.

I managed to setup the basic diode resistor equation and then startted back solving for the unknown coefficients. For those that are interested the equation is:

V = I*R + n*Vth*Ln(I/I_s + 1)

with the unknown factors being n, Is and R and you are given a set of IV data with some noise.

This problem is impossible to solve algebraically so one does need to use a numerical solver to find the most optimal values of n, Is and R to fit some modeled data to the curve. Obviously there are likely other effects that aren't in the model, such as temperature effects which may cause the resistance to change. I made the simplyifying assumption that the temperature change was insignificant enough to ignore and proceeded to do my calculations through a program I wrote.

The results I got was interesting, that I realized that we were taking insufficient data to for my simulator to make an accurate fit. For those that know diodes, diodes are non-linear devices, where the amount of current that goes through them is exponential in voltage or I = Is*e(V/Vth), resistors on the other hand, are linear devices and the current that goes through resistors is proportional to the voltage or I = I/R.

The interesting thing about having a diode and resistor in series is that the diode starts off as being the current limiting device at really low voltages (because the exponential is really small at first) then the resistor becomes the current limiting component at higher voltages. The problem was that our measurements had the current and voltage too low for us me to calculate a reasonable value for resistance, since resistance measurements becomes far more accurate because the resistor is influencing the circuit the most. For example, I modeled out a 1300 ohm resistance with a limited data set and then a 200 ohm resistance with an expanded data set.

Immediately after, everyone thought that the model was wrong since a different resistance was coming out after using a data say from 0 to 600 uA versus 0 to 2 mA. The biggest argument was that the model was flawed because I was getting different current values using different ranges of data. My head wanted to explode because I was telling them and had verified that we were getting in sufficient data.

The simplest example to illustrate this would be like the following example. Suppose that you had the following function: y = -a*x + b*x^2 + c*x^4 where a, b and c are unknown. You can measure both (x,y) and your objective is to find (a,b,c). There is some measurement noise when you get y and y is > 0

Suppose that you took data with x between 0 and 1, -a*x term is going to be the strongest and you'll probably be able to rig the equation to be always positive for some value of (a,b,c) and look about right for x between 0 and 1. But what if you took the same parameters and measured beyond [0,1] and up to [0,100]? How good would your fit be then? if your c*x^4 term had a lot of error, then the resulting graph is going to be out of whack. The (a,b,c) are going to have to be rejiggled so that the fit works well for both [0,1] and [0,100], the whole range. The values might change drastically!

And that is the problem when people try to fit limited data with higher order polynomic or non-linear functions. You could have many different functions that could describe a line segment very well, but everything falls apart when the line segment is extended, because the higher order terms were not fitted well.

I tried to explain that with all my might in Japanese. But they just didn't get it and they kept going with that the model was probably missing a few things. I even back calculated and showed reversed approximations giving a confidence interval down to 12% on the data, but alas no dice.

This is quite a problem when most people I work with are experimentalists, they just do stuff to see if things work or not. When it comes to simulation, programming or theoretical stuff, they know the basics but the real fruits are hidden in the details. Since we make some of these devices, it is important that they are characterized properly so we might have a good idea on what factors might impact the performance of these devices. Just saying that voltage is up or down isn't sufficient because you want to go after the biggest impact factors when improving things first.

Update on the Stock Analysis Project

From text editor to IDE

I started learning a language called Python about 2 years ago after coming back from some traveling. The reason for that was because I fell in love with a dynamically typed, compiled on the fly interpretive language called Matlab which I used for a variety of scientific calculations.

The language had some limitations in the sense that it was slow and I hadn't figured out a way to run multi-threaded programs when it came to doing downloading and parsing html data. The bottle neck when dealing with data in this case is not the PC, but the speed at which the online servers can pump data into my computer. To alleviate this problem, I moved onto a different language called python.

In following my pursuit of a language with dynamically typed variables, easy methods to handle arrays of data and a large breath of built in packages, learning to use this language took some time at first but it would only take me a week to make my first real program: a program to download and compare price vs measured performance of a CPU with published CPU data online.

All of this was written without the luxury of an integrated development environment, which meant a text editor, compiler and debugging environment. I spent the better first part of 2 years writing code in the equivalent of notepad. Maintaining code in a plain text editor is a pain but I stuck with it since it was simple and I didn't want to have learning how to use a IDE getting in the way of learning a new programming language.

As of last month, I've finally installed a free IDE called Eclipse and the Python plug-in from pydev. Eclipse is an open software IDE written in Java designed to run on both Windows and Linux. I still remember when Microsoft Visual Studio was the the big software suite costing a few hundred dollars. Now all of this for nearly any language is available for free. I am impressed with how far things have come in the open source community.

It took a few weeks to learn the basics of learning how to program with the IDE and I am getting more comfortable with it; and the benefits are great. Code organization has improved, automatic look up of functions is awesome, a variable spell checker with a real time code parser in the background to highlight bad programming syntax cuts down on development time. The debugger is useful as I can trace how variables are accessed and if the right data is going to the right place while the program is running. I still have many things to learn.

For the past 4 weeks I have been migrating the stock market analysis code into a format that makes it easier to manage with the IDE. I used to keep all my code in a single directory in different importable files, this made things messy and hard to find. I've finally reorganized and divvied out the code into modules which made things easier to manage and track problems. While moving the code over I've also been rewriting significant chunks of code to make things cleaner as I've found some really ugly bits of code.

What are my objectives with this program?

There is a lot about the stock market that I don't know. I blew a lot of cash after the crash last year, I had good predictions about where the economy was going but was terrible on the execution. Being able to make economic predictions doesn't necessarily mean that you are going to make good trades. My biggest mistakes were leveraging myself too much and was negligent in determining a good price and contingencies. In order to correct these mistakes and fill my informational gap, I am developing a stock analysis software to help me in evaluating stocks and help make better decisions.

Comparing against having my own analysis program compared against online stock screeners, the online screeners are limited to the search parameters and conditions provided. The problem with looking at search terms like PE, EPS and asset/debt ratio independently is that you can't combine the information into a more comprehensive analysis function.

For example, what is the true/implied value of a company's share being sold on the market. This is important when determining when a share is expensive or cheap. When a hot tech stock is jumping in price, at what point do you say this share is over-priced or still have more room to grow? The importance of answering these questions is directly related to the success and failure of one's trading strategy. Because if you don't know when to get in and out then you're asking for a lot of trouble.

This is the biggest challenge when it comes to the stock market; knowing how to valuate the price of a stock. When a person says that the stock is expensive or cheap, I want to know in relation to what data and the direct calculations instead of hand wavy analysis you might see on mad money. For those that might be interested in taking a look at success rates of prominent stock gurus, take a look at the continually updated list at CXO. Most of these gurus are close to the 50% range with a few in the 60% and 40% range. I might as well flip a coin and pray, but I'd rather do better.

What I want to be able to do with the software I am developing is come up with a technique to calculate the intrinsic value of a share based on financial data provided from a company to determine a fair value of the share and market expectations of where the company is going in the next quarter or few years. If the intrinsic value of a stock is at price 'x' and the market is trading at 'y' then the market is expecting that the worth of the company to rise in the future to justify price 'y'.

I have found no screener that calculates the intrinsic value of a stock and I have taken it upon myself to develop a program to test different valuation models to find something that works. Of course, no valuation model is perfectly correct and it would be nice to back test against historical data to find something that works most of the time. As a human being, I am limited by the amount of data I look at and perform calculations, I've decided to enlist the help of a computer and hopefully be able to make well informed decisions in the future.

Programming objectives

Most of my programming experience comes from numerical analysis of experimental data sets. In most of these projects, I would just need to write a very simple program to take measurement data and another program to perform analysis. Unfortunately, I've never needed skills to manage complicated data structures, program modules and software architecture. Making good software design decisions is important to easily add new features without requiring a complete rewrite of code. There are still many things that I still need to learn.

In list form here are a few of those programming objectives:

1. Develop skills in software architecture
2. Become familiar with dealing with large and complicated data sets
3. Learn what kind of data structure to use and when
   
4. Develop robust code that can automatically handle errors
5. Learn module unit testing techniques
6. Improve programming speed
7. Become more familiar with using IDEs
8. Learn how to make a distributable software package
9. Learn how to make a user interface (meaning: also learning event driven programming)
   

Knowing good software design is a blessing as it means taking a complicated programming problem and making it simpler. I've just managed to decide on the implementation of a simple analysis engine design. Instead of writing a hard coded loop for every analysis run, I can just import functions and requirements to the engine and let it do the work instead. Now I don't have to keep track of what code when where and I am starting to see the beauty of functional programming, which was a topic that used to go over my head.

I still have much coding work to do but hopefully I'll be able to produce some interesting results in the future.

2009 Yearly Expenditures

It's that time of year again, the annual review of my living expenses. The 2009 expenditure review will be the 4th consecutive year I've been keeping track of all my expenses. For now, I will be posting my 2009 expenditures and be comparing against my 2008 expenditures. For those that are interested my 2008 and 2007 expenditure charts are here.

Firstly let's take a look at 2009's monthly expenditure bar chart:

2009 Monthly Expenditures (click for a larger image)

As usual, my expenses have been broken down into Food, Goods, Transport, Service, Room, Entertainment, Travel and (Salary) Deduction categories.

Food:

Annual Total: 443,727 Yen ($5,014.12 CAD)
Monthly Average: 36,977 Yen/month ($417.84 CAD/month)

At first glance we see that the my food expenditures have been some what varied through out the year. The cheapest being in January for reasons I am not sure about. The monthly average I spent on food was 36,977 Yen and for January where I spent 21,272 yen for food. I highly doubt that I ate 35% less food in January, but it might have been a result of being treated to a few lunches, being on a ski trip and attending plenty of post new year's parties. The fluctuations in monthly food prices is likely a result of me not having time to cook and eating out due to a busy work schedule.

Goods:

Annual Total: 189,725 Yen ($2,143 CAD)
Monthly Average: 15,810 Yen/month ($178.66 CAD/month)

As seen in the graph, there I've made only a few large purchases in August and September, those purchases being a TV capture card + 2x 1 TB hard drives, a few memory cards and the new Asus EEE PC netbook that I now use while on small trips. There was one other item I bought in February which was a $100 project bench for my room. Other then that, most physical good purchases were just small items.

Transport:

Annual Total: 127,714 Yen ($1,443.17 CAD)
Monthly Average: 10,642 Yen/Month ($120.26 CAD/Month)

Transportation costs for this year was quite low. I generally travel by scooter to pick up groceries. I use the trains on the weekends when I have a few events to attend in Tokyo, otherwise I would usually spend the weekend at home doing some reading or relaxing.

Service:

Annual Total: 130,190 Yen ($1,471.15 CAD)
Monthly Average: 10,849 Yen/Month ($122.60 CAD/Month)

In the service category, there were only 2 main expenditures my cell phone monthly fees, my gym subscription and internet service. I find it rather interesting that I spent more money in the services category compared to the total amount of goods I bought.

Room:

Annual Total: 262,162 Yen ($2,962.43 CAD)
Monthly Average: 21,846 Yen/Month ($246.87 CAD/Month)

Nothing too much to say here other than that my room related costs were pretty fixed. I am saving a bundle by living at the company dorm. These monthly costs include water, electricity and rent.

Travel:

Annual Total: 302,437 Yen ($3,417.54 CAD)
Monthly Average: 25,203 Yen/Month ($284.80 CAD/Month)

As noticeable by the bar graph, I go traveling somewhere at a rate of about once every 2 months. This year, most of my adventures has been through domestic travel to the country side. My travels included a trip to Okinawa, 2 trips to Nagano, 1 trip to Sendai and a trip to Yamanashi. For the mount of traveling I've done this year, the numbers look quite reasonable.

Deductions:

Annual Total: 2,852,197 Yen ($11,241.69 CAD)
Monthly Average: 237,683 Yen ($936.81 CAD)

Taxes this year were kicked up a notch for me as salary deductions from municipal income taxes occur 1 year after the start of work. The few blips seen on the bar charts represent taxes paid due to summer and winter bonuses in Japan. There is an additional blip in October as a result of paying a lump sum into a pension plan which I didn't have to pay at first when I came to Japan, but something I had to pay after I started working.

Now for a quick comparison between my 2008 and 2009 expenditures.

2008 vs 2009 Annual Total Expenditure Table

I've removed the pay cheque deductions to take a look at my living expenses directly. According to the data, it seems that purchases are down all across the board except for the food category. I've just been eating out a little more which has driven my food expenses up a little more. For 2009, there were less large purchases which has driven the amount of money spent on goods down. The largest purchase for me last year was a ~$1000 computer and a few misc things after moving which accounts for the $1,500 difference. The service group remains largely unchanged. The rooming costs have decreased. 2008 included 4 months in Sendai where I paid higher rent than I do currently in the company dormitory. Entertainment costs remain largely unchanged. Travel expenses this year are down by about $2,000 which is directly related to a return trip to Canada for a wedding in Toronto plus a stop in Vancouver.

The total difference between 2008 and 2009 expenditures is a total less of $3,797.49, or a 15.32% reduction in spending. This is a number that would probably have most economists in tears. On average, I spent about $1,749 in living expenses, I think that this figure is a quite reasonable number for living in Japan. If I were not living in the company dorm, monthly expenses would probably be up by about $500/month due to rent to about $2,250 for living about 1 hour outside of Tokyo. Living around the Tokyo area isn't all that expensive that most people might make it to be, but if you want to live somewhere pretty posh, then that might be a different story, but that goes the same way for anywhere else.

So, there's 2009 for you and it was a relatively cheap year.

4 Days and 3 nights in Okinawa

This trip was planned as sort of a get away from the Tokyo and to finally do some vacationing after the long hours I've been doing at work as a chance to go somewhere and relax for a few days. At first, I was looking into getting out of the country so somewhere warm like Bali, Indonesia or somewhere but unfortunately to my dismay, most of the tickets were booked up. By a stroke of luck, I managed to find a travel package to Okinawa with a return ticket, 1 days of accommodation and 4 days worth of car rental included for about $320 and so I was hooked. This is even cheaper compared to just directly booking a round ticket from Okinawa as the tickets themselves would have ran me $360.

Okinawa is probably best described as the Hawaii of Japan, though the Okinawan islands aren't as developed as Hawaii. The islands are fairly south of Japan, putting them in the tropics or sub-tropics region of the equator with temperatures for December ranging in the low 20 Celsius range.

A Map of Okinawa ripped of some guy's webpage

The Okinawan islands have an interesting history given that they are the most southern islands in Japan, having been used in trade between Korea, China, Malaysia and other countries in South East Asia. As a result, the Okinawan culture has many Chinese influences, especially from what I saw in the historical buildings and castles I saw.

Shurijo Castle, Center of Government and Religious activities in Okinawa during the 14th Century (source: Wikipedia)

The throne room

The color red is obviously prominent here, red representing good luck and prosperity in Chinese culture. All other castles in Japan generally do not use the red but rather black and white instead.

As a result of close relations with countries in South East Asia, it is often said by the Japanese living in Tokyo and else where that Okinawa is sort of like a different country. From what I have heard, the culture and language can be quite different from standard Japanese as a result from trade with other countries. Older people in Okinawa apparently speak the Okinawan dialect of Japanese, for which most ordinary Japanese people wouldn't be able to understand. However, during my travels in Okinawa, I didn't find that to be the case as most people spoke standard Japanese.

The weather I experienced for the first 2 days was excellent with plenty of sun. I had a chance to explore the outdoors. The beaches in Okinawa were what you would expect from an island in the tropics. Very clear/transparent blue seas.

Left: The first white sandy beach I found, Right: Looking down at the water from the top of a cliff, below are corals

The ocean was great to look at but unfortunately the weather didn't hold out for me to find and book a diving trip. When it comes to weather dependent activities, I don't usually like to book in advance in case the weather goes sour. But it would have been a great experience. From what I am told, Okinawa is in the world's top 5 best diving spots. I will have to make a note to come back again to do some diving.

The fauna found in around the island was very tropical looking with palm, banana and papaya trees accentuating the tropical environment. The island it self also appeared to be formed through a volcanic process as some parts of the island had incredibly jagged rock formations near the coast though I wasn't aware of any dormant volcanoes on the island which might have given the impression that you were on another planet.

Left: Palm tree boulevards, Right: Jagged rock formations by the coast

Onwards from the landscape and onwards to food, Okinawa is well known for both their pork and seafood. Though I still am not sure what is generally considered traditional Okinawan cooking, I did have some Okinawan esque cooking while driving around.

Left: Okinawan Soba, Right: A buffet at a cafe

One of the things that was listed in the tour guides I've read was to try out Okinawan Soba, which after eating, was like a cross between soba and udon. The noodles in the soup were thicker and chewier compared to ordinary soba reminding me of udon noodles. The noodles are topped with stewed pork which were quite good. The image on the upper right was a buffet I had at a rather nice cafe, consisting of an assortment of rice, fried chicken, cooked seaweed and fried bitter melon. Bitter melon is apparently, one of those things that tourists ought to try when coming out to Okinawa. I personally thought that bitter melon was something that you couldn't find in Tokyo, but apparently after going to the grocery store just yesterday, I managed to find it in the produce section. If you are wondering what bitter melon is, the image below should give you a good idea, it is also often used in Chinese soups.

Chinese bitter melon

A place where I had a good time looking around on Okinawa was at one of the large markets on "International Street" in Naha, the largest city in Okinawa. The market sells a variety of goods like clothes, bags and food. The food market stood out the most to me.

Being the city slicker that I am, I've generally bought my produce in shrink wrapped packages and brought them home, as most ordinary people would do. The island being well known for their pork and seafood, sells their meats in large chunks compared to what most ordinary grocery stores do. It was a very fascinating sight to see.

Meat for sale at the meat market

It's probably been a while since I've had the chance to see stacks of pork bellies like these in a long while as I probably bet that there are a few places back home which sells meats like these, just by looking at them, they already look much more delicious compared to the usual store packaged meats.

While walking around the markets, I also managed to run unto a "salt shop," selling a variety of different salts from all over the island and around the world. If there was one thing I noticed about salt used in Japan compared to salt in North America is that Japanese salt is much "saltier." In other words, if you're cooking in Japan and you use the same amount of salt that you would usually use in Canada, you're going to be in for one salty meal. I never did dig into that much more since coming to Japan but the clerk at the store was nice enough to give me a nice explanation of all the different tasting salts.

Left: Snow salt found in Okinawa, Right: A section of different salts found in Japan

Of the salts they had, the the salts were ranked on a "smooth to sharp," sharp being "very salty" and smooth being mild. After sampling a few different salts, they recommended to me that sharp salts are generally good for seasoning seafood, which is predominantly used in Japan. The smoother salts are generally used in western cooking. After tasting a few of the smoother kinds of salts, I'd have to say that you want to use those in foods like pop corn, but would want to steer clear of using the sharper salts in a similar situation.

It was a very educational experience through it is the only place that I have ever gone to that has rated the differences in the salts. In all, they had about 50 different basic salts at the store and also a wide variety of mixed salts with herbs.

The last 2 days of the trip was spent visiting some of the old war museums as Okinawa was a very major battle front before the surrender of Japan. Casualties of this battle was very lopsided with the Japanese losing 110,000 compared to the US losing 12,000 soldiers, nearly a 10:1 ratio. One of the reasons for this lopsided victory was that much of the natively living populations on the islands were conscripted into the war effort, even the Japanese school girls being sent into service by attending to the wounded. I did some see some old relics such as old torpedoes and very rusted war equipment.

In general, Okinawa was a nice place to visit, but not as resort-like as I imagined it to be. Many of the buildings looked a bit past their time and traffic over there is a pain as the train system is not as developed compared to the central parts of Japan. Actually, I find traveling by train to be something pleasant compared to going around by car because I know exactly when I'll be able to make it to some place, though it isn't as liberating as being to take a car anywhere.

As a place to live, though the culture is a little more laid back and rural compared to most parts of Japan, it isn't a place I would want to stay for long periods of time. Other than the touring, there isn't a whole lot to do. It was a good trip and if there is a good travel deal, I may fly out to visit again for some diving.

The power of IB students?

It's new year's eve and I decided to watch a Japanese sing off show which they have every year (this would be their 60th time), in addition they are running a bunch of boring documentaries which for which I am bored out of mine watching.

The world's fattest female model? Putting a guy in a box of ice for 1.5 hours for a new Guinness record? Top 5 most "astounding" UFO videos?

My brain is exploding from boredom. Interestingly I managed to find a site online where we have a bunch of 16 (17?) year old guys in an IB program somewhere doing a marathon game of nearly all the Zelda games to raise money for charity.

It's apparently their second day and they've raised $2,600 already at the time of this writing. They've made a site with streaming video of them playing live with commentary and a IRC like open forum where everyone can post messages while they play. The dialogue is immature and hilarious at the same time, but I am quite amazed at these guys for setting up the entire website.

Goes to show that intellectual prowess does not necessarily have to go hand in hand with maturity.

If you've got time check them out here, they'll be on for only a few more days.

The nice thing about eating at home

Winter holidays started Friday for me and it feels great to be taking a break from work. This Saturday was a hotpot party with some friends, Sunday I had a chance to check out a Capoeira (a type of Brazilian martial / dance art) where there were some very excellent performances and then spent the day doing some errands, then to come home and cook some pork chops.

I got a great deal on these pork chops, 4 slabs for about $5 so not all meat in Japan is that expensive. The only thing that can get pretty pricey over here is the beef. If you want chicken, pork or fish, then you're in for a treat.

It's been a while since I did any real cooking. The limitation being time and I don't have a kitchen in my room, not that I let the lack of a kitchen stop me. A good hot plate, which costs about $50 here does a very good job of cooking meat or even for a hot pot. The other reason is that I'll be going on a trip for the next 4 days and it wouldn't be a good thing if the food in the fridge went bad.

I bought some spices a while back and spiced up the port as such in the image below and let it rest for about 10~15 mins. The dry rub consisted of coarse pepper, salt, chili powder, basil, rosemary and some 5-spice mix.

The seasoned pork chops looked a little something like this

After that, I had the hot pot at about 230C ish and started cooking. The one nice thing about using a hot plate for doing cooking is the temperature control -- I have a good idea of how hot the plate is by the setting. The biggest problem with cooking dry spiced chops is that if you cook them at too high of a temperature, the spices end up getting burnt and you lose flavor.

After cooking for about 10 mins

The one unfortunate thing about the hot plate is that the one I have doesn't go to a high enough temperature to do a good searing on the meat, just for aesthetic purposes but for consumption purposes, the pork was great.

I served the pork with rice boiled with an assortment of beans and a side of smoked salmon salad, which looks like something below.

Pork steak served with rice and smoked salmon salad

The dinner was great and I ended up having 2 pork steaks for dinner which was absolutely stuffing. I am sure I managed to put some weight on from this dinner. I still got 2 steaks left and will be having them for lunch and dinner tomorrow before I head off to Okinawa. More pictures to follow after I get back!

3 more days till holidays

The holidays just can't come fast enough. I'll be starting my holidays after this Friday and won't be in the office until next year. Yay. It's going to be a much needed break. Will be heading out to Okinawa for a few days and will be back to chill out in Japan for the winter. Getting some relaxing in is going to be great!

If it says "happy new year" then it's going to cost you extra

I looked up on the Japanese postal service website this weekend that postcards cost about 80 cents to send abroad. In Japan, it is rather customary for people to send out post cards for seasonal greetings. I bought a bunch of then wrote them up and sent out 15 of them just today.

I then go back to the site to take a look around and realize that "greeting cards" cost $1.20 to send abroad. If it says "happy birthday," "congratulations" or "merry Christmas," then instead of the usual post card price the price to send out my cards jumps up to the $1.20 price mark. Instead of "net neutrality," I think we need "postal neutrality!"

Long story short, it means that the cards that I spent a few days writing are apparently going to get trashed. If you get your card late, you'll know why... time to do some re writing....

bah humbug!

My outlook on the US economy

Obviously from the last post, I have been busy so I haven't had much time to much in depth research as of late into the economy but my forecasts are still going to be bleak for the US economy. I haven't been doing much research as of late so I'll be talking off the hip today, it's late and I should be in bed.

What I will be watching for in the US is their debt and interest levels for the foreseeable future because that will be effecting the value of the US dollar, tax rates and bank interest rates. Interest rates in the US was at about 5% before the whole housing crash, which brought down the interest rate at 0.25%.

Using a future value of money model with the internal rate of return set at the FED interest rate we can calculate for the present value of a security that provides a perpetual return. The present value is multiplied by the annuity by a constant, dependent on the interest rate. I have no time to write the formula here but for those in the know, you know where to look or how to do the calculation.

At 0.25% the factor is 401 and at the FED rate of 5% at 2007 the factor drops down to 21. This model obviously breaks down for very small values of interest rates because the multiplier starts to become huge. As interest rates drop we should expect stock prices to rise. This was exactly what happened as the FED started cutting interest rates. What they were really doing was masking the real drop in equity prices by lowering the FED rates, thus pushing up stock prices. With the FED rates at 0.25%, they have nowhere to go when it comes to rate cuts and the only direction that the interest rates is up. Unless the US decided to pull a Sweden and go for negative interest rates. This scenario is unlikely.

The multiplier curve for the present value of cash as a function of interest rate is very steep at interest rates rise from a very small interest rate. The US economy is still very weak as it is right now and raising interest rates will cause equity and commodity prices to fall considerably. While interest rates are low, however, money will be pumped into the financial system leading to inflation. The drop in the USD is already is a direct result of this. As long as the FED rates continue to be this low and the US continues to pump out debt, expect further devaluation of the USD.

The only thing that will stop value erosion of the USD is if the US economy becomes productive, producing a financial surplus to pay off their debt and with significant enough profits such that the FED can raise rated without killing the economy. I don't think this will be happening for a while.

We've already seeing speculation on a interest rate hike because the USD is getting hammered. If a rate hike does happen, it will probably be done to stave off fears of inflation in the sort term by temporarily driving down prices of say gold and oil, but I believe that the economy is still going down; I think it's just rumor mongering to temporarily stop the drop in USD and rises in gold. The economy will continue to drop until the full extent of bad investments have been realized. In other words, until when the parties that made the bad investments have taken their losses. Unfortunately, these losses are being mitigated from the private sector to the public by bailouts to banks and other large institutions.

Recovery will happen if these institutions become profitable using public money and paying off their debts. If that doesn't happen then these institutions will likely to continue on a "life support system," until someone pulls the plug on them. It's a matter of accepting the losses or trying to pretend that nothing happened and climbing out of a hole with a weight around your neck.

For US economy to improve, we need to see debt numbers to stabilize and see good investments in the economy. Right now, I don't see it. People don't have money to invest and they have too much debt. The people that can probably make the biggest difference are the ones that don't have debt, either people that are too young to have debt or the very rich with no debt. People without debt are the ones with power.

Got sent home early from work; did too much over time

Looks like there are even limits to the Japanese when it comes to doing crazy overtime hours. I've been busy building a new generation of prototype samples. The schedule is kind of tight so I've been working some long hours to get things done. In addition, I managed to develop a simple simulation model to start evaluating our devices.

The guys I work with are hard core experimental types and there seem to be a lack of theorists in the group. From the measurement data we've been collecting, I've found that I can back solve for some device properties like resistance and semiconductor constants. We've never been able to get hard numbers before and this is a first for the group. I ended up staying in the office till 2 am on a Friday night summarizing data and writing up a presentation for our Monday meetings because I knew I'd be busy with experimental work.

Apparently I hit an internal 16 hour work day limit. Meaning that it is usually mandatory for people to take the next day off. Unfortunately on my experimental schedule, I don't have the luxury of taking the next day off. Since I have booked machine time and the experiments must go on. Instead, I got sent home early, at 5 pm, because HR called in to say that I shouldn't be doing any over time. The call was at 4 pm and I got into the office by 7 am and I was already on overtime.

I spent the weekend chilling out, doing nothing but after getting home and eating dinner, I konked out and slept for 4 hours. It's 1:30 am now and I'm need to get into the office by 7 am again to start taking data. The problem is that I'm not sleepy. Tomorrow is going to be rough.

Looking forward to a vacation and I'll be taking one later this month. But more on that later.

Do they want a blood sample too?

What does making an online transaction have to be such an intrusive procedure. If I was buying a physical item, I would understand why I would need to provide my address and contact information, but when it comes to buying something like a music subscription to some streaming audio, do they need my address and telephone number?


The payment is done through paypal and this is the amount of detail they want out of people doing transactions through them. When I go in person to make a physical transaction with my card, I am not asked my age, where I live or for any contact information. What next, do they need a blood sample too? I shake my head at this.

Friday night, at the office till 2 am

I don't know why I did it. I managed to create a crude model of a resistor and a diode and it looks like I can get plenty of data out of some measurement data. I still can't convey complicated thoughts in written Japanese but after finishing experimental work at 11 pm, I was compelled to stay late at the office to finish writing a presentation for a regular Monday meeting.

I gave up writing in Japanese for the first time. I had detailed ideas that I wanted to communicate, that unfortunately are beyond my skills to convey in written Japanese. I find it awkward that I can't think as deeply in a second language and sadly a lot of information does go over my head. Learning to master a second language and being able to communicate in the subtle nuances is tough stuff. There's more to communication than just the words. It's like a good joke; it's about the delivery.

Language is more than just words.

A day off to do nothing

I decided to have the day off today and it's great. Turns out that if I worked today, I would have over run my over time limit and since I maxed out my over time hours already, I might as well take the day off and expect a nice pay cheque for this month's work.

I woke up at 10 am for once, which was nice as I've been starting work at either 8 or 7 am recently. Just because it is hard to get machine time other wise. Everyone likes to work late so I've decided to in the other direction to start early, use all of the busy machines and switch over to the less busy equipment later in the day. This method of working does wonders because I don't have to waste the day waiting for equipment to become available.

Things have been so busy as of late that I haven't had the time to catch up on doing a lot of small things like cleaning the room, going out and buying things like new socks or what not since I just don't feel like doing too much after coming home from work; it might be some exercise in my room while watching a few episodes off West Wing on the computer.

One of the nice things about having a weekday off is that no one else is available, meaning that I have plenty of time to concentrate to do things like program instead of getting dragged out (which isn't a bad thing of course). It's so much easier to concentrate on a project when you have an entire day to dedicate to it, instead of doing something after work. I've started to understand why people mention that it's really hard to work on something like a startup while working for an ordinary company. It takes free time and concentration to make something, which is something that I don't have a whole lot of after work.

Though I usually have a few hours of relaxation time before I go to bed after work, if I can get focused enough, I might be able to use those hours for something productive. So I'm going to try and see how that goes.

There ought to be good money in reporting software

The thing about working in a large company is that there are lots of meetings. It's unimaginable how many meetings there are. We have meetings for general office announcements, a meeting for one sub group, then there are other meetings for other sub-groups, other meetings for managers and etc.

In my situation, I am working on a project that has implications to a few other teams and thus it becomes necessary to attend several other meetings to keep everyone abreast of what is going on. The problem is that once you start attending many meetings, the time to make presentation material starts eating into time that could have been used for me to get real work done.

Things for me are starting to move out of the research to the development phase, which means that the work we do is starting to be come more standardized and more focused more on making incremental improvements instead of radical changes. For me, this means generally carrying out the same experiments with slightly different conditions to try and find the perfect parameters to make what we're building. There is a crap load of data that needs to be taken, plotted and then thrown into a power point presentation and that does take up a considerable amount of time.

I've already managed to cut down some of that time to automate the analysis but, it seems to me that more is possible to speed things up. Specifically, if I could find a way to automate to take data and automatically make reports and presentations out of them then life would be a hell of a lot easier for me and I can imagine how software like this could be useful in similar kinds of corporate situations. There are likely companies out there that do specifically this when it comes to finances, sales, inventory tracking and more.

There are companies that do exactly this and most likely make good money by tailoring their software to their client's needs and having something like this right now, would really be a kick ass tool.

The West Wing

Just started watching The West Wing a few weeks ago and it's a series that I've found myself rather addicted to. I wouldn't call myself a political junkie and usually I am quite happy to stay clear of politics, but for some reason, because of the exceptional writing and interesting characters, I'm hooked.

Hard to imagine that this was a pretty old series, but the story is pretty good and it's fascinating watching a series about busy people running the Whitehouse. I even thought that the series at first was about pilots... but I had it confused with another series called Wings.

Just goes to say how far out of the loop I've been with TV.

Prioritization

Prioritizing things to do is important. I've just started to have that concept sink into my head recently. Time is becoming scarce. During the weekdays, I have work and come home usually brain fried; meaning that I don't want to do any other work after work. Weekends are dedicated to going out, meeting people or other recreational things. This leaves me with the feeling that I don't have much time to do anything else and I see that as a problem.

I equate the the flow of a person's life to a position-velocity-acceleration kind of physics model. Basically, you are at some position, moving in some direction at some speed at some time in your life. Work needs to be applied to change one's momentum to change the vector of your direction and speed to get to desirable position.

Excuse the looseness of the language but to affect change in one's life occurs when one produces that "work" which I define as work = effort*time. In other words, no effort or time means no change.

I view work (as in working for a company) as "dead time" which is where a person trades time for money. Relaxation time is a by product of recovering from work which also falls into this category. Whatever is left can be invested into one's self into productive means... if one so chooses to do so and it is this kind of time that I find precious.

This is where prioritizing comes in.

When I used to be a student, my priorities used to be relatively simple. I had a set of courses that I needed to pass/obtain good marks in and I would allocate time to do what was necessary. Being human, it was usually the allocation of the "minimum" amount of time to do what was necessary and the rest to do whatever else I wanted to do, which was goofing off. Life back then was simple because my priorities were simple.

Now that I have graduated from University and moved into the work force, those priorities are gone. Now what?

What are your priorities?

I've been trying to prioritize what I've been wanting to do for the last while but couldn't come up with anything then it dawned on me for a second that I wasn't sure of what my priorities were. It wasn't one of those black and white moments that whacked me in the face, but one of those "maybe I don't know what my priorities are" kind of thoughts that made me want to write about this.

In no particular order of importance, my general priorities are :

1. Enjoy life
2. Maximize income for minimal work
3. Do work/projects that I enjoy
4. Travel
5. Have an awesome girlfriend
6. Have a circle of friends that is talented, reliable, active and fun
7. Family

Short term priorities:

1. Establish a reputation at current company
2. Perfect my Japanese
3. Learn Chinese
4. Head into automation technologies (both informational and physical)
5. Improve memory
6. Increase muscle mass, get a mild 6 pack
7. Passive income stream
8. Develop a network of talented people
9. Reduce time wasted goofing off

The list is by no means definitive and will likely chance in as my priorities change, but I'll leave it as an alpha version for now.

It's obvious that there are many things that I want to do and there isn't going to be enough time for me to do everything. Being ruthless at making decisions is not one of those things that I am particularly good at but when it comes to having a tight time budget, one does not have much of a choice when it comes to making a plan. Playing an active role in making a time table for one's self personal life seems as something unnatural, but it is going to be something that I need to do.

The other thing is that I also have a bad habit of getting sucked into events and work that aren't particularly aligned to my priorities. I just have a terrible habit of saying "yes" to everything and I am going to have to put an end to that because I've been stretching myself too thin in the time department recently.

Well, here is a start and we'll se how things go.

Developing a Crawler and Parser library

The internet has a huge plethora of readily accessible information, the problem is that the average human being isn't able to process large quantities of information in a short period of time. So I have taken to the task of developing a small library of functions to allow me to automate downloading of webpages and extract data out of them for analysis.

This project of mine has been going on and off for the past year since doing work in matlab and then moving on to python which has the advantage of running multi-threaded programs. This is a must since you are usually limited to the response time and the speed of a website instead of the processing capabilities of your CPU. Processing textual data can be ridiculously fast, even for an interpretive language like python.

Mashing up data from different sources is a very fascinating experience and I do see some interesting opportunities in the future. The first application I designed was downloading all the prices of different CPUs from an online store and then combining that data with published benchmarking data from another site. Using both pieces of information, I could easily calculate price/performance ratios for every listed CPU to select a CPU which was best suited to my computing requirements and price range. I wouldn't have to thumb through long benchmarking articles anymore to figure out what might be the best CPU for me.

It even gets better as the process can also be automated to the point where it is possible to create a program that will inform me when certain products have gone on sale and notify me if I would like to purchase something. This brings a new paradigm to shopping as it becomes possible to set up an automated notification system when something you want comes into your price range. This is a good thing since you don't have to waste time reading advertisements since you have a program that does that for you.

I am now ramping up the scale of this program to allow me to download larger data sets to mash up financial data from publicly stocks with their stock price to see if I can develop a valuation model to determine the fair price of a stock by using fundamental data. Instead of limiting myself to tracking small groups of stocks, I might be able to look at the stock market as a whole. Using a large data set to look at pricing distributions I can determine with greater confidence as to what makes one stock expensive or cheap compared to another set of stocks or perhaps understand what is going on with the stock market as a whole instead of the filtered (and usually useless) information provided through most financial sites.

The power of this program is immense as I have been working on downloading financial reports from every company listed company for the past 10 years. To get to this data, one must transverse several webpages to finally get to the real data. In the past week, I have already downloaded and parsed 180,000 webpages so far and will be going be downloading another 150,000 tonight while I sleep. This program has read and processed more than I will ever read in my entire lifetime and I think that is amazing.

I have no idea what I am going to learn yet, but the prospect of looking at large data sets and interpreting them has me quite excited. I am looking forward to working on this little project.

Data Backup Action: Backing up a life time of data

If you have one HD crash, you're probably more likely to have another one in the future with the same hard drive.

Just yesterday, I had a small HD crash, leaving some data corrupted. I managed to backup most of my important data before the crash happened (what a stroke of luck) and restored stuff like my e-mail address book. I just realized that I have close to a life time of data sitting on this computer, starting from my university years.

Probably the most important data are my digital pictures and old university assignments. Looking back, I think I managed to solve some rather interesting problems in my spare time and I would hate for the work I did a long time back to get suddenly lost. Who knows, I might find some interesting use for them in the future. Probably the most important data to me are my pictures. I have digital photos dating all the way back to 2002 on this computer and it would be a terrible shame to lose them all. It's also quite interesting how all the data relating to our lives have become so digitized of the span of the last decade. Children born nearly 10 years ago, may be able to store their entire lifetime of photos in a digital format and by the time they grow old, they'll have a digital photo album of their entire life. The sound of a project like that now sounds really fascinating to me and I think there are lots of interesting opportunities for this kind of data. But I digress.

After the small crash I had yesterday, I just realized how vulnerable my data was and decided it was time to start baking up data that I'd hate to lose, mainly all my application data, pictures, university work, and music collection.

The interesting part of this is how little data I have, about 130 GB of data. I have about 3 GB hard drive space on the computer! Maybe over the course of a life time, I may only have about a 2~3 TB in total. That is amazing little compared to the size of a standard HD today.

Anyways, I just finished backing up all my data today on a separate HD and found a great program in Linux that just checks files and folders for changes only and will automatically update the archive copy. I've also setup the process to be automated so I'll have a fresh backup everyday. There is no way, I want to lose a lifetime of data now.

Perhaps it is also time to consider an offsite backup location too.

OS Upgrade: Ubuntu 9.10

This is the fastest OS upgrade I've ever done.

I downloaded the newest version of Ubuntu last night and made a bootable USB stick. I nuked the old OS partition (note: I keep my profiles directory and data on a different partition) and then fired up the installer and told the installer where my profile directory was.

The computer got rebooted and all of my software settings (and most of the preinstalled software I use) just came back online without a hitch. Simply amazing.

I used to spend a half-day doing OS upgrades, where I pulled out CD after CD to install software and then configure everything and reload settings. I just opened firefox and thunderbird and all my bookmarks and e-mail settings were just there. I probably had the whole OS upgraded in about 20 minutes and installed a few extra programs (which can be done in a batch btw) and that was it.

The computer response is far better than I had with Ubuntu 9.4, so far, I haven't had any of the sound problems (yet), open office and flash seem to be running much smoother now.

I haven't timed the how long the computer boots up, but I am up and running from a cold start in about 30 seconds. All in all, I am a pretty happy camper now.

90% of your computer cycles are wasted

I usually work with the task manager open as sort of a habit so I might have a general idea when my computer might start lagging on me, and it does from time to time. But for most of the time, the task manager usually has your CPU idling at about 3~5% of it's max usage for most of the time.

For all the impressive CPU speed we've got with growing clock speed and then now with the growing number of CPU cores, we just aren't using the computer to their maximum potential and it is going to stay that way for years to come-- it's just that most programs are very dependent on human input.

But what if we could reduce the need for a computer to try on human input and perform useful work while we are not around? I believe that there is an immense opportunity for software like these. Seti@home did something like this but it's more for charitable work. What if background processes could be created for a PC to your work while you're not using it?

To me, I think that would be awesome.

The things that you should do, but don't

There are whole bunch of things that I want to do but don't get around to. I come home, read the news (most of it garbage), play some computer games, goof off some more then all of a sudden it's 2 am and it's off to bed else I'll be screwed for the following morning.

Most of the time I spend after work goes into killing time, unfortunately and nearly everyday it feels like a terrible waste, but yet... it... is... so addictive. It's terrible because the time could be used for so much better purposes.

Will power is one of those strange things about human nature. I do wonder why can it be so hard to do the things that you want/should do? I used to think like a mathematician, since the solution was possible and it could be solved/done then no further work is necessary. Unfortunately, things don't usually work out like that and a lot of will power is required to do the hard work to accomplish something.

So there are a ton of things that I want to do but never get around to doing. How does one becomes excellent with executions with plans? I don't know, but for now, it just comes and goes with me. There are times when I really want to get out and do things and other times I'm must "meh." The real trick is finding the recipe to getting the things that you know you should do. The right motivation? It's gotta be something along that direction.

On Ubuntu: Pretty good, still some raw edges

I switched out of the Windows about 2 years ago. I started of with an EEE PC, the first one that ran the Xandros linux distribution while I was backpacking through asia and didn't think it was bad at all. I got internet, could watch videos, listen to music and make skype calls through the little thing. My impressions of the Linux OS system was that it wasn't bad.

So last year, I bought myself a desktop PC because I had enough troubles with my laptops dying on me and the rediculous repair fees and the turn around time to get my computer back. If the laptop dies, then it is going to be chucked. Why spend $400~$500 for repairs if for about double the price you can usually get double the performance (which is about after the warrenty expires!). But spending $1000 for a new laptop is pricey anyways so I have moved onto using netbooks which cost in the $400 range. Completely disposable if something goes wrong. But I am going off on a tangent.

Anyways, about Ubuntu.

Ubuntu is a packaged version of Linux by a commercial group called Canonical. These guys basically put an interface on top of the thing, streamed lined the installation interface and have been working on the thing to make Linux user friendly. They've done a pretty good job of making the system user friendly as the installation process was quite smooth and testing the OS before you install it through the live CD is a nice feature. You can even install it through a USB disk if you like even.

I will note that the distribution is generally based on the same software base but with a different set of selected software installed. When some something doesn't work, it usually a configuration error or incompatibility of some software working with other software. Ubuntu it self isn't entirely to blame if the base software is buggy (the solution, however is a better selection with more stable software). Anyways, let's get started.

A little on hardware

Hardware support is fairly good, for generic pieces of hardware. You won't have problems with the video card which is usually the only piece of hardware that people install on their own. But support for other pieces of hardware is sometimes sketchy (everything else is OK though)

For example, I installed a new TV capture card designed to work on the PCI-e bus but found out after in installing the thing that there weren't any drivers that supported it. And no, it isn't as simple as looking up the part on the internet to see if the thing is supported. I was basically using commands to ID the chips the capture card used to see if there was any support for the thing. It turns out that there isn't for the time being and I would have been better off buying a card that ran on the standard PCI bus.

For "exotic" hardware sometimes things don't work, but it is a work in progress.

On Open Office, the MS office replacement

I am a heavy excel user. I do lots of calculations and usually deal with lots of data. One thing I've noticed about Calc for Open Office is that it is slow. I hate slow software. Interacting with the spreadsheet can start getting jerky when a data set is large and you are flying through the data. It is disorientating, but still usable. The spreadsheet experience on MS office is more pleasant. The Office Office word processor isn't bad, but I don't generally use word processors that much now a these days -- it's mainly simple text editors I use for programming

Sound has some bitchy quirks

If sound doesn't work properly, then it can be a show stopper. Sound didn't work perfectly me right out of the gate when I installed Ubuntu. Sure I could play music but from time to time the music would skip. I used to have skipping mp3s on a 90 MHz computer about 10 years ago, mp3s were just catching on at that time and it was acceptable. This is 10 years later and I am on a 1.6 GHz dual CPU computer having skipping sound out of the box isn't going to be acceptable for people that didn't have problems on a windows platform.

I eventually got the problem fixed after searching on the internet to install some patches and then do some tweaking. The problems seem to be different from computer to computer and not all of them always worked. The whole sound problem is through a sound server system (which in theory is nice, but still need more work in practice) called PulseAudio. For many people pulseaudio is the bane of their sound experience because sometimes it works properly and other times it doesn't.

Pulseaudio is a software layer that handles sound processing and mixing, which some of the older Linux sound drivers were not able to handle. Before you could only have one piece of software using the sound card at a time then software started to get developed to enable mixing.

There is a long story about this here. Read it if you have time, but it explains the development of sound for linux... and sort of why we have problems with sound from time to time.

Slow hard drive transfers (intermittent, but recently fixed)

There are times when I had to move large amounts of data around on the computer that would drive me nuts. Considering that I had bought new hard drives and wanted to redistribute my data, sometimes the transfer rates would be in the order of just a few 4~8 MB/sec. Slow compared to what you should be getting, which is in the 70~80 MB/sec range which you should be getting. The problem turned out to be a problem with the x64 version of ubuntu for AMD CPUs. I eventually downgraded to a 32bit version to get more relyable transfer rates. The 32 bit version, is also more stable than the 64 bit version (flash works properly and doesn't crash as much).

There were people out there that said that the 64 bit version of Ubuntu was stable. It wasn't good enough for me, and the 32 bit version is more mature. Had to learn it the hard way.

So you have a few problems, here's a shovel and dig

When it comes to dealing with software problems with Ubuntu your best resource is the Ubuntu forums, where many people ask questions and post about their problems. From my personal experience, searching through the forum is a pain in the ass. The search isn't good enough and sometimes the solutions I've found were for older versions which may not apply to the version I am using. This needs fixing.

There are people out there that are nice enought to post a link to a main article posted somewhere which my search didn't hit with solution for some strange reason. But still, I consider myself a competent searcher and I shouldn't have to go through a maze of comments to find what I am looking for.

Problems that many people face should be put somewhere more proiminent where people can find them easier. I've killed countless hours searching for a solution to some problem. It shouldn't be a needle in a haystack problem.

Free software is decent

Ubuntu is based on the Debian distribution, which uses a software repository system. Basically people that make software in their free time upload it to the repository and you can search the repository for something that you might need. There is a whole slew of stuff uploaded there, from video, image, sound editing, office applications, games (most amature, but some polished ones out there) and more. I am somewhat fond of the software repository because many of the programs that I would usually have to go out and buy, is there and available for free, legally.

Compared with Xandros and Windows?

I used Xandros for my first EEE-PC and I had a very enjoyable experience with that platform. The software on the computer was stable and I didn't have many bugs to deal with. I never really even have to trouble shoot the software on the system even once which I was pretty happy about.

Software should just work and perform well. I understand that making software and getting all the bugs out is a pain but polish is important. I don't have time to deal with problems. In my experience so far, there are distributions that exist out there where I have never had to debug and I may want to switch over in the future.

In general, the experience in Ubuntu is pretty decent and there are things in the Linux world that would put Windows to shame. The boot time for one is pretty quick. I can have the system up and running in about 30 seconds and I mean responsive. None of this, "let's get the log in screen quickly, but I am still loading" which we see in Windows... after logging in, I still can't use the computer for about 30 more seconds!

Conclusion

I took a leap into the Linux world, it isn't bad and there is some pretty nice cutting edge stuff out here. And if you like desktop eye candy (Compiz Fusion), some of the Linux features can blow the top of windows. See video here. This video is more than 2 years old and there are newer videos out there but I didn't care for the music to them.

Is it useful? Well, not really but it's cool. MS eventually copied some of it's features to Vista. They didn't do too good of a job of it and you got the hog which is known as Vista. Compiz fusion is to desktops as are boobs to women -- they're just sexy. But anyways...

So there you have it. I'm content with Ubuntu for now, but with some of the annoying experiences, I may trade up to something better something comes my way. The 9.10 version will be released at the end of this month and I'll be upgrading. If I get too annoyed, I'll go find another distro to try. But for now, I'm not going back to the world of windows.

Back from the country side

Back from the country side in Japan. it was a long weekend the last week and took the chance to head out to Nagano for 5 days. It was great. Photos will be up later on!

In other news, the company dorm is having the shared connection shut down and everyone will be transitioning over to and ISP of their choice. Looks like the building is wired up with a shared optical connection. Going to be spending about $48/month for internet with gigabit speed. We'll see how this thing stacks up, going to be looking forward to it though, the old shared connection that I used to be on was not the fastest thing in the world, but sufficient enough for me to do the basic internet-y stuff, downloads, online videos, music and webpages.

Stopping and Thinking

In a rare coincidence, it seems that a bunch of Japanese holidays managed to line up this month resulting in having 3 week days off. I opted to take an additional 2 days off to take the week off for an entire 9 days.

I've been away from work for about 8 days so far and it's been great to get away from everything for a while.

I'll have pictures later on, but I opted to head out to the Japanese country side in Nagano to spent 5 days to attend a fire festival and go hiking through a mountain range known as the Japanese Alps. It was also nice to get away from the internet and my computer too, as it some how ends up being a significant time sink, which allowed me to do a whole lot of activities and meet people that I wouldn't usually have the opportunity to.

I'v also realized that being away from the computer for a while has been a good thing, which has allowed me to use some of the down time to just do some thinking that I usually wouldn't do, which has left me a little rusty in the cognative and reflection department. Also, I haven't been able to put perspective into what has been going on in life recently as it's rather easy to get stuck into a repetitive grind. And when that happens, sometimes a lot of things going on around loses a lot of meaning... and it also can get quite disorientating, especially when it comes to making decisions.

This has been kind of tough these last month with the long hours and other things going on with the personal life, for which for the time being I am going to leave the details out. But I have some tough but necessary things to do for which I haven't been seeing in a positive light, which also makes it tougher. It's important to find the positive things and focus on them... and keep going, because if you don't, you'll end up in a situation/place that you don't want to be.

I'll admit that I am being a little vague right now, because it doesn't seem right to say everything at the moment, though I would much rather be entirely open. I need to put all the bad things that happened this year and a few recent things behind me and move on.

How long till your first million?

Work is starting to tire me out. The long hours and running multiple projects at the same time does bad things to one's social life, stamina, stress level and concentration. For July and August, I've done 80 hours of over time and this month I'll be trying to scale back to 50 hours. A 8~12 hour work day is not exactly the way I want to be spending the rest of my life.

The trick to living a good life is working smarter, not harder. So, I've started doing some calculations to get a general idea of how long would it take for a person to make their first $1 million dependent on the amount they are able to save per year, both dependent on whether if they just stuck that cash in a bank or earned some interest.

There are probably similar types of graphs like these in financial planning books but I have created one for myself as shown in the image below (click for an enlarged and much clearer version).



The results don't look encouraging, but let's do the math. The above graph plots the amount of money you will have saved dependent on your annual savings rate and depending if you reinvest the money to earn 7% or not. The dotted lines represents a linear savings rate.

I have been collecting expenditure data for my life style and it comes out that I spend approximately $2000/month in living expenses, including taxes. Given this, I will have expenditures at $24,000 year. Suppose now that I want to save $20,000 a year, requiring earnings at approximately $44,000/year.

Without investing, it will take me 50 years to make my first $1 million, or 23 years if I reinvest my entire savings at 7%. I'm going to be pretty old by the time I get that far, considering inflation and everything, $1 million is not going to be the worth the same thing in 23 or 50 years. This is not an acceptable solution, so we are going to have to do something to increase income.

Suppose that we bumped things up a notch and aimed for a $40,000/year savings rate. Making the bad assumption that the tax rate doesn't change, I would have have to earn at least $64,000/year to make this goal. Obviously, the time to make the first $1 million is cut in 1/2 to 25 years if you just stashed in the bank or 15 years if the funds are reinvested at 7%. The numbers are starting to look a little more reasonable.

Now, suppose for a young person that your income is $44,000 and you need to bump it up to $64,000 now to hit that $1 million target in 15 years using the 7% investment plan.

You have 2 choices:

1. work approximately 50% longer or
2. find more productive ways of generating income.

Suppose that I worked an 8 hour work day and took option 1 and extended the work day to 12 hours. Including a 1.5 hours of commute time and about 8 hours of sleep for a total of about 21.5 hours/day being used up. This leaves 3.5 hours for personal time during the weekdays.

That obviously does not seem like a reasonable lifestyle to me. Thus working longer for more money is out, so option 1 is dead, so we consider option 2.

How does one find more productive ways of generating income? How do you go to your boss and say that you want 50% more pay? Under what conditions would they provide you with that cash? The most obvious answer to me would be to improve productivity either by doing more of the same in less time, or doing something completely different to produce more value in less time.

Negotiating with HR or management for salary increases is obviously a thorny issue, but you are going to have to be equipped with the right kind of information to make your point. From my point of view, talking directly in dollars and cents or productivity is very important to winning that salary increase. But also, there are obviously limits on the amount of money your employer is willing to pay an employee. If you really want to make the big bucks, then you are going to have to go at it alone.

I now have some real world experience in me to finally appreciate these numbers, I only wish that people would have found ways of understanding the magnitude of long term planning in their early 20s. Imagine if you were 35 and you made your first million.

The impact is huge.

So you want to gun for your first $1 million? I'll give you a chart for how much and how long you need to save to achieve it.


I will admit that the calculations done here are crude and requires refinement, but the basic analysis of the $1 million problem should give everyone a good idea of what exactly you're up against.

Attaining this goal is a very tough challenge, hard enough that it isn't easy to do alone. For all you young people out there, if this is a worthy goal for you to attain, look to your peers and find ways to help, work together and educate yourselves. Because it's going to be now or never, and the sooner the dialogue starts, the better.

What I have learned while working

What I've learned while working:

I work at my own pace
I don't like repitious work, and will usually dream at finding a way
to automate it
I am good at automating data analysis
I enjoy putting machines together that reduces my work load
I attend about 10 hours of meetings every week
I also don't care for most of the meetings
I don't generally like discussions with too many "what if's"
I also don't like discussions when everyone is shooting off the hip
I am generally slow when it comes to e-mail
I'll get back to people faster if it is important
My work related e-mails (when written in Japanese) tend to be short
I am far more motivated to work when I see an immediate goal (and
somewhat tangible reward)
I make mistakes (lots of them)
I am really forgetful
I tend to think about other things while doing boring work
I have a terrible habit of forgetting everything I've thought about
after getting home
I have not seen many young people contribute or have the opportunity
to contribute radically new ideas / projects

Observations:

Mistakes are a waste of time
If you are credible, people will generally take what you say at face value
Giving good presentations at meetings is 50% of what is required to succeed
The other 50% is doing a good job
The best managers have excellent memories and are very detailed
The best managers are good at identifying the important issues quickly
The worst ones (even though they are nice) hold too many meetings and
progress tends to be slow
Managers generally like definitave answers
If you are good at your job, managers tend to be friendlier
Being 2x more efficient than everyone else will not result in 2x more pay
Given the option to take a chance vs going the safe route, more often
the safe route is chosen
Getting something new approved requires a lot of testing... even for
small changes
Problems can occur even for small changes
Young people are often not in a position of high impact
When mistakes occur, counter measures are usually often discussed
Most counter measures usually slow work down to prevent mistakes
Having too many counter measures is a pain in the ass

A few things about Japan:

Business level customer service in Japan is amazingly good
If you really need something done, they can usually work something out

Resting the hands for now

It is a lull in writing on this blog for reasons I don't know at the moment. I probably am just a little tired and finding that I have less time to self reflect and write. There have been a lot of things I have been thinking about but usually end up forgetting about them, that or I've just been too lazy to write.

I've found that I've become much less motivated to write. The reasons why are probably numerous. For a reason, I have started keeping more to myself as of late, meaning that I do much less communication and at the same time, I realize, I have also have started to lose my communication skills in tandem.

There aren't a whole lot of people I can communicate at the same level that I think at, with some of the language barrier here. There are still many cultural "in" things (like old TV shows or want not) that I don't get either. From time to time, I find myself the odd man out and have gotten used to it, so I usually don't mind finding my own ways of going about and enjoying myself. Usually, just walks or heading out to new places I've never been to on my own.

Anyways, when it seems I am ready to come back out to the world again, I will make my return. For a person that has usually seen myself as an extrovert, I am also finding plenty of peace spending time on my own.

Upgrades

I bought my computer last year and it's amazing how prices for hardware has dropped.

Last year, I spent about $260 for a 19" monitor. One of these monitors now costs about $130 in Japan. A 500 GB hard drive ran me about $80 last year and this year, I can get 1 TB for about $70.

It's amazing how quickly prices fall. It's ironic how quickly the price of computer hardware drops but yet there are incredible opportunities if this hardware could be put to good use. The average person out there doesn't even come close to making 50% continual utilization of their PC.

I've decided to buy 2, 1 TB hard drives and a TV tuner for my PC. I'll be using it to record Japanese TV shows for when I want to watch them. Going to be trying to improve my Japanese by watching some TV and finally getting myself immersed in the culture. I haven't owned or watched TV regularly for the past 5 years. I've gotten used to it... but it comes at a cost: I tend to be out of touch with the average joe over here.

I'm going to have to understand these people so I'm breaking down and going to try and watch some TV and see what happens.

Dealing with People

One thing about being a foreigner dealing with locals is dealing with disagreements. Sometimes it seems like the cards are stacked against me sometimes, there is somewhat of a cultural difference and at the same time, I appear asian enough to understand the Japanese cultural norms. Then there is also sort of a language barrier, though I wouldn't call it a "barrier" in the normal sense, but my linguistic skills seem to be lacking for me to come off as someone with tact.

This is a source of a lot of recent frustration, leading me to be either annoyed or somewhat depressed when things don't go right.

One of those unfortunate things about language, is it is not nearly so important as what you say, but how you say it to garner the support of other people. There has been many a time when working with other people where i've been right but not persuasive enough to convince people that I am right in a diplomatic kind of way-- in the kind of way that they feel good about it.

It's damn trickly and somehow, I have lost patience when dealing with people. I used to be much better, but when things need to be done, I've hit the point where I don't want to waste time arguing with people and go off and do my own thing.

I think it's something that's developed after moving out and becoming more independent-- I've gotten used to having the freedom to doing my own thing... can especially be a problem if you've become good a becoming resourceful in the way that you don't need to rely on anyone else.

What results is that sometimes you can make the objectively right decisions, but still be politically wrong. Life can be a bitch that way, especially when a person like me operates on the belief that making the right logical decisions should be independent of the person. I've been right in many cases, but not politically correct and dealing with politics in a different language can be a pain... especially if you want to be good a diplomacy.

Nope, it's not one of those things I am good at and I've already stepped on a few toes. Working in a company is kind of different from doing work in a lab... the difference is that I've already proven my self once and its easier to earn the respect of peers in the same age range. Working in a company with much older people tends to be kind of tricky.

Sometimes, I think that this is possibly one of the reasons why not a whole lot of new, dissruptive technogies/business models are created with a group of older people... but I digress.

I need to get around this problem... I need to be better at expressing myself in Japanese. I still have a hell of a long way to go.