Away from Home

Some web hosts aren't all that good

2012-02-01T01:07:00.000+09:00

A while back, I tried out iPage for some web hosting to see what it was like to run a site from a remote server. I was pretty dissatisfied with the result since the sight was really slow all the time. I didn't do much with it other than just use the site to keep online notes. Sort of a like a Dropbox on the go. Expiration for the service ends this month for me and with the cost to keep them as a host, I decided that it was better off ending my contract and moving on.

I have since started playing around with running my own little box as a host. It's rather cheap and fast on the local net. Still working with pointing domain names around and learning how to access the site from outside my LAN as I am playing around with nginx instead of the standard Apache.

Learning Linux administration stuff is kind of fun too.. couple that with some web programming it could possibly be an interesting adventure. Perhaps I could launch a service in the future? We shall see.

Optimize for investments or salary

2011-12-26T23:30:00.001+09:00

My time is limited and in my spare hours, I've been trying to write code to better understand the stock market. My current work involves understanding stock options, which are quite the beast. I've so far gotten as far as writing parsers to get information form the internet and have gone as far as running monte-carlo simulations based on a variety of models but the problem is that I really don't know if these these models are correct and the more I code to create models, the more I realize the less I know about modelling the stock market.

There is just so much on my own that I will be able to figure out on my own... then I thought about buying some advanced books to read. Then it hit me, I've got a lot of learning to do and I started to wonder if my efforts in this field would be worthwhile.

It is now also the end of 2011, meaning that my expenses spreadsheet for the year is nearly complete and I have a good idea of how much I've spent and saved for the year. Since moving out of the company dorms and into my own apartment, housing costs for me have skyrocketed and the amount saved dropped by about 33% compared to last year. This is even accounting for increases in salary. But the thing is that doing more research in stock picking may increase my investment returns by a few more percentage points. This begs the question, would I be better off investing that time in developing better skills to raise my salary?

Thus, I've started doing more research into engineering salaries and pay for jobs that I could possibly do and for jobs around the world. Thanks to the exchange rate between the yen and the dollar, the numbers look pretty good, but this is after a lot of overtime pay and travel pay that I got this year. On a dollars/hour basis, I don't think that the hourly salary was all that good.

Been Busy

2011-12-20T09:24:00.001+09:00

Been really busy the last month and a bit to do any real writing, but a lot has been going on and I haven't had much of a chance to do much writing as of late. I've been flying between Tokyo and Kagoshima nearly weekly to help out with setting up an inspection machine and writing code to speed up the processes. Also, getting paid travel expenses and a small stipend is marginally nice.

On another note, I got a better than usual work review...after all was said and done, however, the better review led to only about a $600 increase in my winter bonus. The software I wrote cut down processing time by about 5 days and gave us more time to analyze data to improve the success rate of the process. If ouput and pay were plotted, the graph would be very sub-linear for the place where I work.

More notes to follow at a later time.

Defining the problem actually takes up a lot of time

2011-11-15T21:04:00.001+09:00

I've been flying between Tokyo and Kagoshima for the last month and a half after doing my first 2 month stint in west Japan. I've already manage to take a 1 week process down to 3 days and while allowing 2x more samples to be processed simultaneously, meaning 4x throughput.

By the time I got to Kagoshima, some of the engineers have been working on figuring out how to get the process to work and determine an algorithm for sample inspection. Armed with the basic information from these guys, I started writing software through the night shift and my work spawned into a project of it's own.

I've already finished writing an application and trained a technician to take over the job. I've started working on the next version of the program to try and have the entire process automated since my program doesn't work in some special cases. Classic example of the 20-80 rule, where 20% of the effort gets 80% of the output.

I am treading into territory where other people haven't gone into with the data analysis and I've noticed that I am spending less and less time writing code and more time trying to better define the problem and the exceptions that my code needs to deal with before writing any code. It's somewhat frustrating to not be writing code to get things done but I guess it's part of the process.

It's odd how the motion of "doing" something is so closely related to progress, though planning and preparation is also a very important part of solving any sort of problem. Being busy for the sake of being busy is detrimental. I generally like disappearing from the office when working on a tough problem that I haven't managed to wrap my head around and go for a coffee. I find that I understand a problem well, I work infinitely faster because everything is in my head and laid out.

There is something called the "drunken sailor's walk" which is a classic problem that nearly all probability courses covers at some point. After n random steps, the furthest you've traveled away from the origin is related to the root on n. If you walk with direction, the distance traveled is n.

In other words, if you plan things out properly and minimize mistakes, you'd be better off by the factor of a square. If you wonder why there can be such a gap in skill or level in the world sometimes, I think that it is the expression of this phenomenon.

For those that have the time, I suggest reading Richard Hamming's book The Art of Doing Science and Engineering: Learning to Lear n, which is a fairly ok read. There are some interesting notes from a well known researcher, from which I gleaned this idea from.

A disappointing exerience at a computer store

2011-11-14T03:07:00.001+09:00

I did some more thinking about putting together a server recently and came to the conclusion that even though I would be paying a little more money to run my own server at home, I could get close to 4x more performance and unlimited hard drive space instead of renting one. I also though it would be a good experience to learn to set one up should I need to set one up online using one of the services like Amazon's AWS.

I put down the cash for server parts last Friday and so far, the CPU, motherboard, case and memory arrived. Since I was going for a server system, I didn't need a monitor or a fancy video card for the system which brought the price of the computer down considerably. When going for the slim computer case, I made the mistake of purchasing one without a built in power supply. Thinking that getting a power supply wouldn't be hard enough, I dropped by the closest physical computer store I could find online only to be incredibly disappointed-- all the power supplies were really expensive and the cheap ones that they were selling were "sold out".

Obviously, I wasn't happy with spending the 20 minute (one way) commute on my scooter to get to the store, spend time looking around to find out that they didn't have what I wanted at a more expensive price.

After getting home, the first thing I did was go online to the Japanese equivalent of "pricegrabber.com" to source a part and I was done with making the order in about 10 minutes. The only difference is that the part will probably arrive in 2 days instead of me having it immediately, but when it comes to being busy, waiting isn't a terrible thing.

Over the past few years, I have been doing more and more online shopping instead of going to stores to buy things instead. It's simply more convenient. I really wonder how much has internet shopping had impacted the retail business and the number of old school stores that have gone out of business as a result of it.

I also move around every 2~3 years making finding new places to buy things fairly important to me somewhat time consuming. Ordering online never ceases to fail me since I get what I want delivered to me quick enough. But back to the computer stores, I really don't understand how they can keep up a business where people have to spend time to go to a place to buy over priced product. The only one possible answer that I can think of is for people that aren't as computer literate to talk to people to get what they need. If that information, too, can be accessed in a manner easy enough online, I think we can kiss the big box computer retail stores goodbye.

If this then that

2011-11-10T00:41:00.001+09:00

An interesting automation site, I thought that the idea was interesting when they went into open beta after Sept. I checked after 2 months to see how they were doing and found that they hadn't updated the blog in 2 months. I think they're dead in the water right now and the people that worked on the site are off working on other things.

From the stock market, business and life. Things take time to happen and you should make it a habit of checking old news. So many thing that were at the front of our mind one day become totally forgotten later. I don't know why the time span for ideas is so short, but it would be wise to periodically check old news from time to time instead of just watching the latest headlines; because just tracking the new and not tracking how things go is meaningless.

The Autobiography of Steve Jobs

2011-11-08T23:21:00.003+09:00

I haven't read a book in a real long time. Importing books is expensive and I generally abhor having more things to clutter up my room. Then came my desire to tinker with the kindle for the PC and I found reading incredibly refreshing.

After about 3.5 days of reading on both the computer and mainly on my iPhone, I've finished the book. I read of the life of a great man. For once history will have something positive to put among heroes because there just haven't all that many in recent history.

As a person to emulate, I doubt that many people would want to be like him; but to every great man, there are key ingredients. I've turned 30 this year and work life practically ends at about 65. If you really want to make a bang at something, have kids and do everything else, there really isn't a whole lot of time. Every moment is precious and I am reminded of that after reading this book.

Never have I been so consumed with putting my mind to doing something in a long while. After reading the autobiography, you can tell it was put together with care, with countless interviews and decent writing. After finding myself so consumed with reading for the past few days, I just came to the realization at how much the internet sucks; as in "holy shit, it really sucks," because there is so much garbage out there.

Maybe I haven't been going to the right places for content but, I've gotten tired of fishing through the needles in the hay stacks of social sites looking for something good; just simply having a good read was so amazingly refreshing. Yes, I will admit there are some good reads on the internet. Some great essays out there even, but nothing as cohesive as a good book. I missed that, I really did.

The other great thing about reading about Steve Jobs is that he was incredibly intolerant of things that bothered him. I found myself to be quite the opposite recently, being as accommodating and accepting as possible that I noticed that I lost a bit of an edge to life. When you get to the point of accommodating to the sub optimal it just turns into a slippery slope of accepting worse and worse. Sometimes you just need to draw the line even if it means you might have to be an ass.

There is a lot about the book that I really enjoyed, but I feel that I would do no justice to even talk about the story because what really is the point about just saying if something was interesting or not? The act of simply talking about something just feels like it has no value to me and in it's own way is what was the story of Steve Jobs about-- making things happen.

To get good at the stock market, spend time away from it

2011-11-01T23:15:00.001+09:00

The stock market right now is one big drama and generally it is usually the people that spend the most time watching dramas are the ones that get the most emotionally involved and start losing money. I made that mistake once before in 2008, just waiting for the perfect time to catch the bottom and doubled up on it because every motion up and down seemed huge to me and when the stock market dropped once in October, I thought "this is it" and it really was "it" for me with a gaping hole in my trading account.

Gone are those days and I've gotten better. To be honest, I think the market in general is on tilt, to use a poker term and it is probably best to leave the table as it is and take a drink while the players sort themselves out. Because really, when everyone is playing weird, you just simply can't control risk (even if you are making logical plays).

I'll admit that I had a few of the equities that I held before the crash over the summer drop by 50% percent. There were people out there talking about being glad that they got out and them going on about how much better it was for them to get out early. Some of those tragically massacred equities I held came back to nearly their original prices and I somewhat regretted not putting some more cash into the market. I can't expect to trade with 20/20 foresight and I'll take the bad with the good.

Sure I am a bit down from the market haywire but I am happy to bide my time, earn money from work and wait. More importantly, instead of waiting, have other things to do :)

Went Jogging for the Halloween Weekend

2011-10-30T09:16:00.001+09:00

I've been out of Tokyo for the last few month as a result of work. I've probably fallen off the face of most of my social networks during that time and at the same time, I kind of enjoy the peace and quiet from people. I used to be really busy every weekend meeting up with people somewhere, attending events. But it is nice to spend a weekend home.

I woke up, spent the morning doing some research, went out for a jog, came back, had a steak dinner and had the best sleep ever. I naturally woke up at 8 am today feeling particularly great. This is something that I need to repeat more often.

Spend time, doing productive things, go to bed tired from hard work and wake up great. This is how life ought to be lived.

Deploying to AWS doesn't make sense (not just yet)

2011-10-29T11:49:00.001+09:00

I've been mulling the thought of deploying a webserver to Amazon's AWS service as of recently and I've been running the idea through my head multiple times and something just didn't sit right. I've been playing around with a webhosting service for the past year to see what it is like to work with a webhosting service and came to the conclusion that the lack of control is a killer data portability; it is easy to create a blog and write on it, but it is a real pain in the rear to back it up and be able to move it elsewhere should I want to. The other problem is the limited control of software installation I can do, meaning that I would be completely limited to whatever the service provider has installed. This is where Amazon's AWS service comes in.

Amazon's AWS is basically a server instance that you can use, it is very much like having your own computer to play with, install an operating system and then do whatever you want with. I've been doing calculations on price for such a service and the numbers don't look favorable for a single "Small PC" instance which runs for about $744/year. At that price, I could buy a stripped down computer for far less and turn that into a usable server for at least a few years. Transitioning to AWS only would make sense if I needed signifcant computing power beyone what I would provide from home.

I'll be looking into buying a new computer to play with at home.

Rusty Debating Skills

2011-10-15T15:43:00.000+09:00

I lived in Sendai for the first 3 years of my life in Japan and one thing about living in Sendai is that people don't generally debate and disagreements are often avoided. Moving to Tokyo was rather different as being good at making arguments is important, especially when working in a company.

I, unfortunately, have not developed my Japanese debating skills and for the lack time spend among English speaking peers in Japan, I have found that my debating skills gone rusty and the "thick skin" that one needs to survive a debate has also gone thin. In effect, I believe that I've lost some mental agility and stopped keeping a pool of readily accessible information in my head to make arguments because I've had no need to before. Information retention is terrible at the moment, though some might attribute this to the aging effect, I refuse to accept that as an excuse.

Especially when it comes to dealing with upper management, when making persuasive arguments is especially important for these people to sway executives and push their career upwards. If you want to do well in a career, having good debating/persuasive skills (balanced with it not being aggressive) is absolutely important.

To be honest, I miss my old philosophy class in high school where we used to debate on a variety of topics at 7 am. Sure we would be sleepy as hell, but it was a lot of fun back then.

Playing with some new styles

2011-10-15T15:36:00.000+09:00

I've been using the old blog style for ages and figured that it was time for an update. I have no clue when it comes to website design so I just decided to play around with whatever they had on blogger. Not too excited about the new look because I think the templates they have here look too goofy. I'll just suck it up since I can't go back to the old design now and I think that most people that read blogs anyways do so through google reader (and if you don't, you should!)

Kick ass and then become a consultant?

2011-10-12T17:05:00.001+09:00

Into the second week of October and already cranked out more than 20+ hours of over time. If this was Canadian style flex time, I'd have 3 vacation days stocked up. But this is Japan, so I'll be taking overtime pay instead.

Been working on putting together some software for a project for wafer level analysis. I use a newer language compared to the older guy I work with, Visual Basic vs Python. Programming lightening fast while I have most of the base engine of my detection engine up.

The problem here is that by the end of this year, my stint in Kagoshima will probably be over and my work will need to be passed off to the guys here to continue. The problem is that no one can program in the language that I use. They'll have one of the guys eventually translate the algorithms into their native language.

Interestingly, I really do wonder how often do cases such as these occur, where one guy is able to do a good job that the other guys that follow after can't keep up. I best that this is how a lot of people start their contracting gigs.

Steve Jobs has Passed

2011-10-06T12:15:00.000+09:00

In an instant, this is the only news that is making the rounds on the internet. The iconic Steve Jobs is gone and there is a void in the world of technology.

Some links to postings about Steve Jobs from notable people:

Outpouring from the internet is simply massive right now.

Is a crash on the way?

2011-10-06T00:21:00.001+09:00

I know for sure that a huge rally is probably not on the schedule, but there ought to be either a crash or stagnation in the stock market. So long as you have the timing right, selling calls during times of high volatility would be smart to me. I don't hold any short positions yet (I wished I did) just because I haven't had time to do sufficient time to get myself into the right positions.

Of an interesting note, the venture business in the US could be looking to take a hit in the slowing economy and this is something that I'd like to keep an eye on.

As for now, the yen is sky rocketing like crazy. What that means to me is that I'll be looking in ways of putting in more over time to make more cash and sell yen for other currencies because I think the yen is over values.

From Kagoshima to Tokyo and back again

2011-10-03T23:51:00.001+09:00

Finally got back to the home after 2 months in Kagoshima. Getting off the plane and getting into the city, I realized that I was facing some culture shock from being back. All the people and tall buildings... the other thing I missed while being back is good country food.

I eventually made it back to my apartment and found that I had left a light bulb on for the entire 2 months. An interesting experience but when I did power calculation, having that light bulb on for 24/7 over a month it cost about $24. It's pretty amazing how much power a light bulb draws over the course of a month, more than the fridge I have.

But to be honest, most people don't have their lights running at home for 24/7. Interesting lesson.

The mail box at home was stuffed to the brim with fliers for all sorts of things, especially girly stuff like manicures, hair salons and frilly clothing- nothing that a man would need. Sexist marketing at it's finest? Quite possibly so.

I also dropped by the office today and saw one of my old managers and his greeting to me was:

Manager: "Welcome back. You look thinner. You okay? Maybe you should see a doctor"

Apparently I have a rather interesting work place.

I got back and picked up a $800 LCD monitor and had it shipped off to Kagoshima for me to use for programming. I even had it shipped by air. One of those perks about working in a large corporation.

Applied Stock Market Stochastics 2 (applications to options)

2011-09-30T23:41:00.000+09:00

I wrote earlier in Applied Stock Market Stochastics about learning using the mean and the standard deviation of stock market data to calculate the probability distributions of where a stock might go in the future. I've decided to take what I learned from the previous post and take it a step further in application to Option derivatives.

Understanding options is a little math heavy; option pricing comes from the black-scholes equation pricing equation by assuming that the price of an equity moves in a Geometric Brownian motion like way with a constant drift and volatility. We can model this using the mean and standard deviation by throwing the parameters into a Gaussian function. The black-scholes equation is basically a closed form solution to the differential equation representing this condition.

I got tired of simply just looking at the equation so I decided that it would be more interesting to create a statistical model to determine the motion of the price of a stock. In my last post in Applied Stock Market Stochastics, I analyzed the historical prices of the Dow Jones index as a distribution of percentage change on a month by month basis. The result showed that the motion of the stock market moves around in a Gaussian like manner.

I will assume that you have a basic understanding of put and call options. If you don't know about them, the short story is that the purchase of one of these contracts is a bet that the price of some equity will be above or below some price. If it is above or below that price, your profit is proportional to that difference (and if you need to learn more, hit the links or Google for more details).

With that in mind, the question that I want to solve is the likelyhood of making a profit or loss and by how much. If we understand the probabilities of making a profit or loss we can then calculate for the "expected value" of the trade-- which is the average loss or profit that we expect to get if we made the same bet over and over again. If the expected value is positive, we expect to make money and vice versa.

Now, I am not aiming for the level of sophisticated analysis hedge funds do out there and I won't pretend that I know that much either, but I do have a great fascination of learning about things though a first principles approach; and I think this is a great first principles approach to tackling this problem!

So let's first model the motion of an equity instead of a stock market index. In this case, I've downloaded Google's historical prices for the last 3 months (between June and September) and calculated the mean and standard deviation of it's % chance in price on a daily basis. The numbers I got from this calculation were 0.0373% and 2.409 for the mean and standard deviation in percents. For actual use in the models, I divided these numbers by 100 and ran a simulation using 100,000 simulations to simulate the resulting price of the Google stock by October 20, 2011 which is the expiry date for a set of call and put options. The resulting plot looks like the one below:

The change factor is the multiplier which you would multiply the stock price to determine it's final value. The plot here, is also normalized (meaning calculating the area of this graph will equal 1 or 100%).

Now for illustrative purposes, I took an option at random, which was the $555 strike call option dated for Oct 20th 2011, which was priced at about $12.00 when I checked at Google Finance and plotted out the returns as a function of price:


$555 strike, call option Profit/Loss profile @ a price of $12.00

So now we have now have 2 pieces of information: the probability distribution of Google's stock changing by some factor by the option expiry date and the Profit/Loss profile of the stock option. Now the most important question to answer is, given the current price of Google's stock at $537 (when I checked today), what is the weighted profit/loss as a function of price? The answer to that is to take the probability distribution of final prices and multiply it by the return distribution of the option to get the following plot:

What this graph tells us is what you expect to earn/lose over all the possible outcomes. Summing the values together will yield the expected value of this trade, or the amount of money you expect to make when making this bet over and over again. I've done that calculation and the number resulted in -$3.023/per share (and 1 contract is a multiple of 100 shares) meaning that you expect to lose $302.3 on average when making this trade.

But!! There is an upside to this however, in that you can instead sell a call contract instead of buying one and expect to make $302.3 instead and turning this into a profit.

Caveats

Obviously, if this were the case then everyone would already be doing this to make money in the stock market (and to some extent, there are people and organizations out there that are already doing this). So there are 2 important assumptions that you needs to be aware of, and those are:

that the motion of price can be modeled by a random Gaussian variable and
the mean and standard deviations are accurate and do not change

But there will be times when Google's stock price won't behave in the random manner that we modeled it as and we don't event know that numbers that were put into the model were right in the first place! This is the risk we take when using this calculation (and any other model for weighted profits and losses!).

So what is this good for?

This is a problem that plagues historical analysis when trying to make predictions of the future. But the most important take away from this is developing an understanding of the likely and unlikely future outcomes to help in making better decisions; in the long run, this is the most important part.

Now, even with these points of uncertainty, this could be used as a model outcomes based on your research and how you feel about it. The great power bestowed from this is being able to quantify your opinions of the future and risks involved by turning them into an expected value. This helps considerably in decision making instead of simply just going off on "a hunch."

There is obviously more work in modeling random variables and outcomes but I hope that you find this a useful tool in thinking about the future movement of a stock. Ideas and comments are most welcome.

Load Speeds are Important

2011-09-29T22:48:00.000+09:00

I bought some hosting space on iPage in February of 2011, the service was cheap, space was unlimited and I guess I got what I paid for when it came to speed; fairly slow load times. I wasn't happy enough with the load times to move the contents of my blog over to the site and domain that I've been playing with. But I'll make the appropriate announcement when I finally find a decent hosting solution.

I have been toying with getting onto Amazon AWS and setting up my own server and having my own space. The great thing is that I would have some persistent computing power available to me to run some automated applications. I really think that there are many fun things that one can do with having a server, without having the need to own the hardware to do things.

I am somewhat reluctant at using my main computing box at home as a server since I want something that I could completely crash with impunity but I am not really that interested in having a second full blown desktop computer at home. One option that I've been contemplating is using a MacMini as a server due to its really small foot print, the price is reasonable too at about $700 for a system with 4GB of memory.

Interestingly the same system sells for about $100 more when buying through the Japanese Mac Store. I think the exchange rate is playing with the price significantly as the Yen has sky rocketed (time for the arbitration??).

Time to do some research, but with some effort, it is possible to setup a server on AWS for about $110/year and have some fairly decent speeds. Storage goes for about $0.10/month meaning that I could have 4-5 Gigs of website data easily and still be only paying about $0.50/month in storage. Seems really reasonable to me.

All I would need to test is latency and bandwidth. If things look good, I'll probably make the switch.

Distractions are Productivity Killers

2011-09-28T21:27:00.000+09:00

My work setup currently has me working on a large rectangular open desk suited to 2xN people, where N is determined by the length of the desk. The office consists of a large room with M tables where everyone works at. The problem with this setup is that the office can get really noisy and I have a hell of a time concentrating when there are that many people around, all conversing about a wide range of different topics and phone going off every 15~20 minutes.

In an environment like that, memory and concentration intensive work is nearly impossible. To be honest, some of the best progress I've made with code was done while working the night shifts where no one was around and I was able to just zone out and simply focus on what was going on with the code. While programming, I really need to be able to keep track of variable names, the structure of the program and algorithms in my head. The instant my concentration is broken, it takes time to reorient myself to figure out what I was doing again. This is actually a classic coding problem, especially when looking at code that one wrote days earlier.

To me I am not a huge fan of large open offices. There are some people out there that are able to focus in such environments, however, I am just not one of those people. I'd much rather have headphones on and be dead to the world while getting work done.

I am quite sure that offices around the world are setup in some manner like this, but I think it is rather counter productive. With laptops being used a primary computing devices, I really don't mind having a main desk somewhere, but I'd be much happier to have access to a silent stall where I can allocate blocks of 2-3 hours to get some serious work done and return to my desk to get less mentally intensive work done and be accessible to people. I am sure that there has been plenty of research done in this field and I am curious at how much of this research is implemented in offices around the world.

I would venture to bet that an architect that is able to create highly productive environments would be paid big bucks!

All You Need is the Right Trigger to End Microsoft

2011-09-27T22:00:00.000+09:00

I started using Ubuntu Linux about 3 years ago and currently going on to my 4th year using the operating system. There were some hiccups at first when switching over, but now looking back, I have found the process to been a worth while one.

The greatest benefits to me about using a Linux system is the ease of reinstalling software and getting original settings back after a fresh install. I still remember times when working on a Windows computer where I would reinstall the OS once or twice a year to freshen things up and spend 2~3 days getting the system back up again with all the software that I use and configuring everything to my liking. What a pain that was.

The difference with working on Linux and having a home directory where all one's settings are stored in (somewhat) hidden folders, is that as soon as program is reinstalled, the home folder is the first place the program looks for to get the setting information. So long as you backed up the home directory, the instant you reinstall software, it is automatically configured. The other great thing is that software can be batch installed from repositories, and instead of having to deal with installation dialogues, you just tell the computer to install the software and it automatically installs them to a binaries folder and things are automatically setup. If you have multiple users, everyone's home directory holds each user's personal settings so there is no clashing whatsoever. Everything is simple to install and fast to restore.

The other great thing is having free software and having a repository to download whatever you need from. Just download the software, give it a try and if it works great then keep it. If not then uninstall it and try something else and most free software out there is "good enough" for all practical purposes.

There have been times that using free software didn't work out for me because of some minor formatting or incompatibility issues. For example, while open office can open and read MS Office files, there are times when the formatting of documents get a little off. I had the unfortunate problem of trying to write and update resumes in open office to find that the alignment of text was just off enough to make the document look a little unprofessional. As a result of that, I've become a little bit of a fan of PDFs, once you convert the document into a PDF, it should display the same across all systems.

If worse comes to worse, with the advent of OS virtualization, I am now able to run Windows inside of Linux or Linux inside Windows or what have you. I think this is a great boon to trying out new OSes and also getting access to just that one application that you can't find on your native OS. Rebooting is such a pain as I generally don't like needing to close all my applications and then open them all again after restarting the computer. Right now I use Win XP in side of Ubuntu just for MS Office on the rare occasions to edit documents, but other then that I don't use Windows for much else while at home.

To be honest, I don't see any real argument for the need to be dependent on a single operating system anymore. Most of the best software are multi-platform now and using them is just a matter of installing them onto whatever OS that you're on. Even writing multi-platform applications is not terribly hard using languages like Java or even Python since just-in-time compiling is sufficient to create native code for the applications to run.

So there really is no real need to be totally dependent on a single operating system anymore for any reason other than for the comfort of a familiar interface. To be honest, I think that Microsoft is walking on hollow ground; the instant the ground gives, I think it will be all over for them.

Is Facebook killing blogs?

2011-09-26T20:59:00.004+09:00

One thing I've noticed recently is that I am far more spontaneous when posting pictures and updates to Facebook as opposed to writing on the blog. Uploading pictures to Facebook is dirt easy, albums are automatically created and linked to my profile and it is quite easy to make short posts while on the go. That and compared to the blogging medium, one may have to upload pictures to a separate site, then add comments and then link back to the blog. It's the multi-step work that is a certain killjoy to me.

I've tried blogging while on the go (on places like the train) but found that doing things like looking up multiple webpages, adding links and doing basic editing (like moving a few paragraphs around) to be quite cumbersome compared to writing short status updates.

Which brings me to the point that I believe that traditional blogs are dying out compared to posing things on the fly as smart phones become the norm. The vast majority of my friends have since stopped updating their blogs and mainly post news only on Facebook, which has been the trend for quite some time.

What will happen to blogs in the future, I am not entirely sure. Perhaps one day there might be a big enough backlash against Facebook that people may return to realm of blogging. Who really knows. I for one will continue to write, and actually make a point of spending more time posting to the blog compared to writing on Facebook. Perhaps the next best thing is stronger integration into (out of?) Facebook through blogs.

To be honest, there isn't a whole lot of original content on Facebook anyways and much of my best thinking happens while doing "real" writing.

Applied Stock maket Stochastics

2011-09-25T23:29:00.000+09:00

A stochastic variable is pretty much a non-deterministic variable, in other words a random variable. I've been thinking about stock market in terms of statistics to better quantify risks and expected values because it really boils down to understanding the risk-reward profile and creating a portfolio with the desired characteristics given the available financial vehicles.

The challenge is then quantifying the risks and rewards of which is a black art. I doubt that anyone has a perfect system of setting these values a priori because risk and reward are particularly fuzzy numbers until after the fact.

Given the current fragility of the world markets, one occurring theme that I've been contemplating is the mechanics of hedging. There are of course several ways of not losing money before a market downturn may include:

Selling off equities
Buying bonds
Shorting stocks, selling calls or buying puts
Buying short ETFs

The list is not extensive but those are some of the options that one would have during market uncertainty. The next question becomes, what do you think will happen in the next time frame (which can include spans of days, months or years). The idea will be to track current and future issues and setup a portfolio that tracks your sentiment based on the data collected and your interpretation of them and make bets on sentiments that you feel the most confident about.

Suppose that you don't believe that the stock market is quite volatile but not in a position to make huge gains. If you have securities already and already made significant gains but don't want to incur capital gains taxes, selling your securities might not be the best course of action. In this case it might be wise to lock in gains by hedging against future drops.

This begs 2 questions:

What is the probability of the stock dropping n%?
Give the probability of a drop, how much should you pay for insurance?

Now obviously without part 1 of the answer, answering part 2 is going to be hard. So I've started first looking at part 1 of the problem. What is the probability of the stock market moving in some direction in some time frame?

The most obvious method that came to mind was to first get the probability distribution function of a stock market for the range of all monthly percent changes (I could have done daily, but I already crunched the monthly data), The graph looks like the one below:

The above plot contains all monthly data from 1928. The plot looks fairly Gaussian yielding a mean of 0.764% and a standard deviation of 4.445.

The next step that one takes with this data is determining where things may go in the next few months. The easiest method to calculating what may happen in the next 6 months is to create a random Gaussian variable with the mean and variance of the Dow Jones index and multiply them together to determine the probability of what may happen in the next 6 months. The result is the following graph:

Given the cdf of the next 6 month, the next question to answer is what is the probability of seeing the Dow increasing or decreasing by some factor? This requires some integration but I've done some sample calculations and those results are provided below:

Based on the information below, we have a general idea on the probability of seeing growth in the Dow based on all historical data within the next 6 months. What would be more important is to find means and standard deviations that represents the market right now; which could be done through calculating the distributions of %changes in price over a more recent time fame, or more importantly, values that they might expect in some future time frame based on current research.

The idea will be to create a portfolio that represents a personal bias on individual stocks and the stock market in general to create an adequately hedged portfolio against catastrophic crashes while exposing one's to certain amount of gains to mitigate risks while trying to aim for the most optimal profits. Modeling the motions of the stock market in general are important tools to help create a portfolio that matches one's risk-reward profile.

This is still a work in progress with many imperfections but it is a start to creating portfolios with explicit characteristics.

Why you shouldn't try to become the perfect trader

2011-08-21T04:20:00.008+09:00

I've been going through historical data in the Dow Jones recently to try and better understand past trends. I've mentioned before that I generally try to stay away from technical analysis but I've decided to put away that hat and decided to see where this takes me. The one good thing that looking at historical data is understanding the scale for change, things are measured in months and years instead of days and hours. I have a terrible habit of looking at daily market data, thinking that motions in the day are indicative of where things are going to be in the long term. So I'm going to stop looking at the market like that (though it can be fun sometimes... sort of my equivalent of "watching sports").

I've simply started plotting through the Dow Jones looking at annual returns year by year. Looking at annual returns is a really arbitrary thing to do since I am picking a price at some "random" date and comparing the ratio of it to some other date in the future. I am sure that picking different windows will have different values but what else am I to do, eh? Not a whole lot until I figure something out, but for now this is it:

I've grabbed annual data back till 1971 and the funny thing about looking at 40 years or annual data is that there is so little data to look at. The first thing that pops out at me is the all negative change years occur after a +10% change year (though not after all 10% years). The second thing is that 5 out of 7 negative years turn positive the year after. So what would you do in this case? One could surmise that it might be a good idea to take out some money after a +10% year and put it elsewhere as a contingent against a negative year (Ie. Bonds or something). The next thing is that you might once a negative year occurs, you have a good probability of catching a positive year after a negative year. Something like that could work but the problem is that you will still need to optimize against having 2 consecutive bad years. There is also a period in the 1990 where there was nearly no negative growth years, but let's say that you've optimized your strategy to factor that in too.

Now suppose that based this 40 years of data you've made a fairly optimized portfolio where you take a small chunk of money during good years and put them into bonds when you make more than 10% when the new year hits and made a rule that if there is a negative year, that you would plunk (say) about 60% of the money you had in bonds into the market during a bad year to catch the probability of a good year the year after and it left you with enough funds to deal with 2 consecutive bad years you could leverage yourself since the probability of having 3 bad years according to this data is zero (the alarm bells should already be going off here).

Suppose that this strategy worked well against this data, you might be fairly tapped out if there are 2 consecutive bad years and the strategy provides for very good gains. Unfortunately, if you look far back enough in history, there is a period right after 1929 where there was 4 consecutive bad years. If a black-swan event like that happened, everything in your portfolio would be gone. Unlike poker, you don't generally have "rebuys" in the stock market (unless you have really rich and possibly foolish relatives), If you're going to want to play this for the long game, developing a methodology to understand what is going on around you, knowing your options, knowing your limits and knowing what exactly you are willing to risk is important. Having good well founded rules to maximize your success are important and you should, in general, play by self set rules that are designed to maximize your chance of success (and though, just like poker, success isn't guaranteed). And just as a note, there are 4 years of consecutive negative growth around 1929 that might have wiped this strategy out (then again, it probably wiped out nearly everyone during that time). In addition, you may want to have a contingency to catastrophic events.

I had this knowledge internalized in me in poker so well that I had forgotten to be aware of my general rules that made me a generally strong player. Most of my poker playing rules were also mathematically generated too, the game of poker (un)fortunately is really easy to analyze. All you need is some general understanding of probability, expected value calculations and knowledge of permutations and you pretty much had it made. The stock market is different and choices and information is so much more vast such that it nearly impossible to understand the entire system... though I still do try (though I am off on a tangent).

Rule number 1, though, it to have a set of rules to play by and more importantly, have a set of mental interlocks to prevent yourself from going "tilt." Getting into the stock market will make one incredibly emotional. A bad stock market period is like PMS for men (that is not to say that only men participate in the market) and you honestly need to get a hold of yourself during those time because you are going to be prone to doing stupid things.

This coming of the second crash (?) has given me a great opportunity to better look in to and handle crashes. The first stock market crash that I have ever had to have deal was the 2008 one, and my, that was a doosie. I wished I was old enough to have participated in the 2000 dot com bust, because I would have been much younger and had a lot less money to lose. Unfortunately, I was too young and had very little real capital to my name during that time (and the cost of trading was way too expensive during that time; $29 for a single trade!). Anyways if/when I do have kids, I am giving them money to blow in the stock market, they might as well learn to lose everything early and fast (and I hope to not make chronic gambling kids while at it too).

Fail Safe (avoiding gambler's ruin)

Unlike other games, the stock market is the kind of game where you play on one "life." The idea that you're supposed to be in the market till about 65 ish (or longer) and not screw up and lose nearly one thing is actually daunting. It's like pretty much saying that for 30-40 years that you are not allowed to fuck up. But that is pretty much it, lose once in the middle of it and you are possibly screwed.

I've come to the realization that optimizing for gain and building a robust portfolio are (to me at least) opposing factors. Optimization is basically trying to allocate resources perfectly to come out with the most perfect result as possible (which is easy in games, hard in real life) because it can equate to doing things with minimal margin for error and going for possibly unrealistic perfect results. When it comes to investing, you are never going to get the most of that up swing, nor are you going to know when the market has hit bottom. The question is not a matter of making the perfect play (because it is impossible), can you make decisions such that are "good enough"? The next important thing is that can you make consistently make "good enough" decisions? Because if you can, then you are on your way to becoming a good investor and I think this is often an overlooked point.

The important thing about investing is being able to quantify the risks of every decision that you make and have a general idea of what you are going to do if things turn bad. How much loss are you willing to take? if losses do occur, are there other assets that can balance against that loss? Is the portfolio setup in a way that you won't be psychologically affected if things go to bad (or good?). Risk quantification in a portfolio is really tough and it is a really fuzzy science but it is very necessary. One of those lessons learned from my crazy crash and burn in 2008.

How am I faring?

I think I was lucky this time because I wasn't overly invested in equities and though the equities that I did hold lost a lot of value, I am sleeping better compared to last time. The other thing I should note was that I was a little impatient in getting back into equities a few months ago when it seemed to me that things were going to start taking off. The equities that I did end up buying were over priced because I thought things were going to be a little better than they currently are right now. I should have been more patient but that is the way how hindsight works.

I've also thankfully kept a good amount of cash out of the markets and will be patient for things to cool down in the market before looking at getting back in. The volatility will be here for a few months until Europe sorts it self out and people get over the S&P downgrade of the US (which ironically, did something good for US treasury sales). There will be plenty of sucker rallies and swings coming up... but then again, that is already pretty normal for the stock market.

If you can make sense of the following data, then you deserve to be a millionaire.

A look at some past crashes

2011-08-16T03:59:00.007+09:00

I had some spare time over the last few days so I've been going through some of the Dow Jones historical data with some programs to study volatility during some previous stock market crashes. I first plotted out daily percent changes in price and volume to look for large swings in price and volume. The result of that plot is provided below:
Looking at the percent change in stock price, it become fairly easy to pick out periods of time of high volatility and we see a few points in 1987, 1989,1997,1999-2003,2008-present. By the looks of the graphs, it takes a few months until volatility subsides. Though as of today, it seems that the stock markets are up, I highly doubt that the choppiness seen in the Dow last week will be the last.

The % change in volume plot doesn't capture well changes in volume since it could simply mean that volume could have suddenly risen and remained constantly high during a crashing period. Though from the plot, we do see that volume can vary quite wildly.

Given the % change in Price plot for the Dow Jones, I had a program go through the data looking for points in time where a > 5% decrease in the index occurred and extracted 4 months of data before the crash and 6 months after the crash to get about 10 months of data. I also had the program mark points where a > 5% decrease occurred and came out with 7 plots:

Black Monday Crash

Friday the 13th Mini-Crash
Asian Financial Crisis

Russian Financial Crisis

Dot-Com Bubble

Sub-prime loan Crash

US Debt Down grade Crash

The data seems to indicate that during the pre-2000 eras of crashes, volatility was far lower compared to what we are seeing in the markets now. This is probably the result of the HFT algorithms going at it in the market place. The most recent crash is showing far more volatility compared to all but the 2008 stock market crash as we have seen in a single week, some very large price movements in the Dow Jones Index. There was also a large spike in trading about a week before volatility picked up and I am not sure what this represents.

Not really sure where the volatility is headed from here, but there could be more on the way if things this time around play out in a similar way to 2008. We'll have to wait and see I guess.

Where does "net worth" go during a stock market crash?

2011-08-11T03:44:00.002+09:00

Understanding your "opponent's" trading account is akin to keeping track of the number of chips your opponents have during a game of poker-- the amount of cash your opponents have will have an impact on the way they play the game.

One thing about stock market crashes that fascinated me is trying to understand where the money and capitalization value goes. I have been working over the idea of the existence of "phantom capitalization" in the stock market as a result of stock prices being bid up and I was interested in trying to understand the mechanics of a stock market bubble. So I was basically trying to model a feedback loop were people start pouring more and more money (and leverage) into the stock market, thus creating the illusion that everyone has more equity via "phantom equity." The process repeats until it is unsustainable and things fall apart.

I believe that there is a disconnect with this mode of thought since people are fooled into thinking that this equity is real and the harsh reality sinks in when the inflated stock prices tumble which wipes people out. The real objective of participating in the stock market is to buy and sell stocks to eventually cash out with more money than you started. By playing by any other rules instead, I would argue that the investor is fooling themselves when participating in the stock market. Unfortunately, the calculation methods used to value equity is somewhat faulty and is prone to enticing people into inflating stock market bubbles.

The first thought that came to mind was to try and simulate players and their interactions in the stock market; trying to understand the impact of their buying and selling of equity and how it would look like in their trading accounts. Obviously, this is a pretty hard problem as the real stock market has many individuals and tradable securities. Going for the full monty is obviously too complicated so I decided to sandbox the whole issue and have 2 players with cash and a single tradable stock which both players owned at the beginning. When one player issued a buy order, the other had to sell at the same price and volume and by using this simple model, I wanted to see what a stock bubble would look like under this isolated situation.

I managed concoct a trading pattern between 2 players where they could buy and sell and equity at increasing prices while keeping the amount of money and the number of stocks they had on hand constant. However, through the act of trading and using the standard method of calculating the "average action price" of each person, I was able to inflate the price of the stock and thus was able to create phantom equity for each individual. I have included a screen shot of the simulation I performed in the image below (click for the full sized image).

In this rather simple simulation where I kept the volume constant, I noticed that the swings in the "cash" column got larger and larger as the price the stock increased. In more realistic situations, I would presume that the volume would taper downwards to reduce such large swings in one's capital account. I would surmise that this also wouldn't apply to just individual investor but is also true for computer systems engaged in high frequency trading. But the main point is that it is possible to observe the inflation of networth of 2 parties trading in a single market and that it should be possible to extend this model to a many people market with many different stocks.

For the above calculation, I didn't take it all the way to the point where both persons were leveraged to continue bidding up the equity price, but it should be quite evident that the swings would get larger until the point where neither person A or B would be able to leverage themselves any further. Supposing that a dormant person C existed, any sell off by this person would result in a catastrophic crash as margin calls are made.

From a technical trading standpoint, I think it would be an interesting exercise to find ways of watching the volume and price as an indicator of bubbles in the stock market. One concept would be to watch for decreasing volume while prices rise as an indicator of a bubble since higher prices will equate to lower volumes as a stock is traded up. I would also expect that the converse be true in that volume would likely increase when stock prices are low since it takes more of the same stock to move the same amount of money.

What I am very interested in trying to figure out as a result of this simple calculation is an empirical method of guessing at the "action prices" of other participants in the stock market. I have a hunch that people will have a tendency to sell off their stocks after turning a profit and will likely end up spending their profits on buying other (inflated) stocks. The whole process continues until a bubble occurs and it pops, leading to the destruction of phantom equity.

One method of making money in the stock market is simply knowing when to get out and staying out when a bubble occurs. And the trick to understanding when a bubble is occurring is figuring out when there is too much phantom equity vs the amount hard cash available.

The Unwinding yen carry trade?

2011-08-09T13:17:00.001+09:00

The yen is skyrocketing recently against the USD, CAD and the AUD over the last few days. The chart for the JPY vs the AUD is especially astounding as people are unwinding their carry trades. The recent spike is astounding. The other thing, however is that I am quite certain that the rising valuation of the yen is unsustainable against other currencies and there should be a correction happening eventually. The question is at what price and how volitile things will be during this time period.

One thing that I appreciate after coming back from Cambodia and Vietnam is an understanding of the bartering system, where the prices of goods is often bartered between the customer and the store clerk. Prices do vary between shops and it is very interesting to see how prices can be very fluid between places. Everything is driven by the sentiment of the store owner, the customers and other market circumstances and this experience is equally applicable to the stockmarket.

One intersting observation that I made about the merchants that I bartered is that they liked to ask where I was from. Basically, what they were trying to discern is what price would I be willing to buy something by the living standard of the country that I lived in. A Chinese person might price things differently from an Europeian and etc. When it comes to the stock market, instead filtering the buying and selling patterns of people by country, one would need to understand the trading motivations of different financial institutions. Meaning that different people would have different ideas of what a fair price for the same financial instrument.

Anyways, I am very tempted to sell more of what yen I have into other currencies, but the question I am trying to figure out is how long will the sell off in the stock market will continue. I believe that the yen will continue to strengthen the worse things get in the stock market. The question is when will the sell off stop, at what price will people start selling yen and etc. All of this is really hard to figure out.

When it comes to the financial market, one cannot look at companies in isolation but also other traders and their motivations in the market. I have a new appreciation for that and very interested in developing tools in trying to understand the financial conditions of the other players in the market. As in poker, you have to be able to keep track of the number of chips everyone else has because how people play is also dependent on the number of chips they have.

In Cambodia

2011-08-01T01:49:00.001+09:00

This is probably out of the blue, but I am in Cambodia right now. My work had a scheduled 1 week shut down of the office and facilities as a result of power conservation efforts in Tokyo. Apparently, companies have a sort of power usage limit between now and until mid-September and going over the power limit will result in fines based on the number of hours a company is over the limit. There are other details but I won't bother getting into them as it will probably detract from the main contents of this post, as in me being in Cambodia.

Dispite backpacking through South-East Asia 3 years ago, Cambodia is one country that was on my list to goto, but I never made it to because of time constraints. This time around, with the 1 week holiday from work I would go and hit Hanoi and Siem Reap to see the Angkor Wat temples. After a bit of a turbulent jounrey from Vietnam to Cambodia (not to mention that they also block facebook there), here I am in Cambodia and I have some pretty positive things to say about the country.

The people are incredibly warm and after getting out of the airport there weren't a bunch of taxis waiting outside waiting to over charge travelers to get into the city. We also had a tuk-tuk (a motorbike connected to a sort of trolley) that waited patiently even as my flight into the country was delayed by 2 hours. Compared to other SE-asian countries, there is a lot less price gouging here; then again, the cost of living here compared to other countries is so much cheaper that even if they raised prices, I might not even notice.

The price for a dish or something from a local stall ranges in the $1-$2 range, a foot massage for 30 mins is $2.5 and a full body massage for 30 mins is $3, just to put things in to perspective. Staying at really nice hotel runs for about $30/day, which is really reasonable. My imagination runs wild with the idea of running off here and not returning to work for simply years while staying out here.

The other nice thing about people here is that they are generally good natured and even if they can't get a sale out of you, they still are nice to you. Which is a rather nice touch. One of the guides I met is only but a teenager but has managed to learn a really good level of Japanese through 1.5 year of study and acts as a guide for many Japanese tourists that come out here. The determination I've seen in some of the youths here is simply astounding.

There is one sad thing that I've felt while being here is that there is a huge influx of tourists in Siem Reap and it feels like that many of them are just here for the cheap thrills of a country with a really low cost of living- there are some people out here that I've see that I would have a hard time imagining getting anything of value from visiting the angkor wat temples and at the same time, it feels a little sad that the locals are doing their best to cater to these people through the creation of bars and restaurants to turn a profit-- a perfectly good environment being ruined by the influx of money through the western world... and that was probably the most profound insight I had while walking around the city today. I took a bunch of photos, but haven't had the chance to download and sort them yet. Hopefully, I'll have some time to do so later on.

Being shipped off to Kyushu for a few months

2011-07-27T11:10:00.001+09:00

The current phase of the project that I am working on has ended and as a result of that, it seems that my expertise are required elsewhere in the company. Crunch time is coming up for a special Sept deadline and I was just called in by the boss, yesterday to learn that I am being shipped off to Kagoshima, Kyushyu for 2 months to help troubleshoot some process related issues.

They'll be putting me up in a hotel for the first week and then a mid-term housing arrangement while I am there. I'll likely be working a few graveyard shifts as well... the compensation for taking this job is still being hammered out but here is hoping that I get some good spending money out of this.

Working on a few other things on the side as of the moment and the weekends have been really busy with events.

In other news, I'll be taking a 1 week vacation in the first week of Aug to visit Vietnam and Cambodia. Posting will remain light...

We are in for an interesting FX ride

2011-07-12T19:03:00.001+09:00

Been following the news a little bit lately and it seems that Greece and Italy are having debt issues and the currency markets are going nuts.

The US will need to pass a bill to raise their debt ceiling on Aug 2 and what exactly will happen there is uncertain but I am quite sure that they will raise the debt ceiling, because it will have significant impacts on their ability to raise money in the future if they go into a mini-default.

As a result, I've just been looking at the currency charts recently and the yen is jumping upwards in the midst of this uncertainty. I've already diversified some of my funds out of yen an may look to purchase more foreign currency as I think that the markets will be fairly unstable.

It will be interesting to see how things will play out over the next few weeks, but I'll be lining up a cascading orders in CAD and AUD should the yen spikes up some more.

What does it take to run an E-business

2011-06-30T12:45:00.001+09:00

I've been mulling the idea of having some parts fabricated and sold online recently and got down to considering the challenges of logistics and putting together a small site to see what happens. It's more of an experiment than anything else but having sold some X-box's though E-bay before, I just thought: "how hard would it be to create an online store and be able to accept credit card payments for something."

Been looking around here and there but a cursory search indicates that one can open a merchant account for about $25/month through paypal and deal with them skimming off some of sales. Modeling selling things online is also quite fascinating and what I noticed is that sensitivity to profits is very high when margins are low. The most important thing is learning how to create a landing page and figure out how to drive traffic to a site to gague interest before starting.

Premature Optimization is Bad

2011-06-16T15:59:00.000+09:00

The title almost sounds dirty, but still, one shouldn't get too excited and ahead of them self when it comes to implementing programming solutions. One of the biggest challenges to programming is understanding the full scope of the problem that is dealing with. When I mean "problem," I don't mean it in a sense that something is broken, but in the sense of understanding the "specification of the system" of what one intends to build.

I am currently putting together a more advanced parser that can read text and extract values out of it from html. With the advent of content management systems (CMS), most of the data displayed on the internet follows a specific layout depending on the page. Writing parsers can be a tedious process as you need to do several things:

1.   Effectively isolate the block of text want to analyze
2.   Write a parser to target and extract the information you want to get, and
3.   Create a data structure to save that data

This process is time consuming because I would have to write additional code to parse different values and change the data structure to save everything. For every additional dependency there is in code, the probably for error goes up exponentially (a gut feeling). The less dependencies between modules there are, the less errors in code that you'll probably get. The best kind of code is code that automatically adapts itself to whatever you're doing... but I am going off on a tangent. Right... dealing with premature code optimization.

I have come up with an idea using html templates instead of writing code to parse data out of an html block. Writing procedural code requires one to think of a strategy to get at the data wanted and then codify the process in a program. That means that for every block of html, one would need to write code to get at the data. I've done that before and it can be a time consuming process (which can be made somewhat easier with the use of parsing libraries however).

Using templates it becomes pretty easy to specify the structure of the text and target certain sections of the data using keyword markers. Editing text and converting them into templates can be somewhat tedious but still far easier than writing code to extract data. The work required is just turning parts of the template into wild cards and entering keywords to certain parts to get at the data and I really like this solution, so much that I decided to write code that would semi-automatically take html text and convert it into a template. It took me over a day to try and put something together... and I realized that I still hadn't fully understand the use cases of the html templates, the possible forms of the html templates and just as important, the form of the templates can have small variations that could cause the code to not work.

So here I was, spending a god chunk of a day trying to write code to optimize a process that I hadn't fully understood yet. And I have a feeling that some of the biggest failures of software projects is a poor understanding of the use cases which may need extensive revisions to deal with unforeseen problems.

More important than making it work quickly, is that it works first. I ought to stick to that before putting effort into speeding up certain processes. Once I get this templating engine up and running, it'll be interesting to see what uses I can come up with for this code.

Building a web craling platform

2011-06-15T10:37:00.001+09:00

I've been doing some web-crawling over the last few years. I started off with some really rudimentary pattern matching stuff but over the last 2~3 years I've become much more comfortable with text processing. To the point where I am becoming able to programmatically to edit chunks of text that I write. The power of really good text processing is amazing, especially when it comes to structured textual information.

I have spent an innumerable amount of time trying to figure out the best way of writing programs to parse text and I have been thinking of programmatic methods of getting the data out the information out there. But the more and more time one spends with text, patterns and strategies arise that can be used over and over again to get at the information embedded in text. What I want to be able to do is to create a simple frame work that will allow me to quickly create parsers for whatever text document I want to get at and keep a library of them so I can stream data from a variety of websites. Eventually, I might be able to make it easy enough that even non-programmers can be able to write parsers too and that may have some interesting applications.

Running 4 km (almost) every weekday

2011-06-14T09:46:00.001+09:00

As a result of the commute, I've been working on getting the most out of my time. Getting home by about 8~9 pm everyday means that I don't have a lot of time to do much else. Considering that I am now making time to study for my CFA exam in December and trying to exercise everyday, that doesn't leave me with much time to do anything else after getting home.

I've noticed that I've put on a little weight, about 3 kg compared to last year. I attribute that to getting a big bag of almond chocolates from Costco and having a kitchen where I am cooking much meatier dinners as of recent. So I am needing some cardiovascular exercises to act against the recent bad diet (which I will get back into making more healthy).

I have a transfer stop on the way home where I need to change trains which I need to ride for a single stop before getting home. I've decided to take this opportunity to skip taking the transfer and jogging the rest of the distance home. I've been cycling on and off since the last year, usually doing long treks on the weekend (about 30 km) and I was quite pleasantly surprised to find that exercise on the bike transferred quite nicely to running as running 4 km. Progressing through the first weekhop of running nearly everyday, I have found that my stamina has increased and that I can push up my speed significantly. There are also daily variations on my ability to run, however I am feeling an improving trend. I should start timing my runs to see how I progress.

By the time I get home after a quick hop into the shower, I've got time to pump some iron using the weight training machine I have in my room. I also have some free weights in my room that I use from time to time and I've noticed that using free weights are great for full body workouts instead of working on isolated muscles. I still am working on trying to figure out the perfect routine to really push my muscles. It looks like I'll either have to hit a gym sometimes and talk to a trainer or consult some books.

Conquering Excel Macros

2011-06-08T18:38:00.001+09:00

VBA is a terrible language. It's archaic and idiosyncratic. I recently got back into programming in VBA since yesterday to help out a friend at work to process data faster. It would take him weeks to what this program will be able to do in minutes. The only kicker is that it costs me time to write this code.

I've transitioned over to Python as my main programming language about 4 years ago and I've learned a lot from it since starting. Functional programming and the map, reduce strategy to computing data. Basically, if you can write out a function to process one block of data, it's just a matter of looping through the rest of the data array to calculate everything.

What python (and several other languages) do is obsolete the loop and make them implicit. If you have a function and you have an array, then there is way to apply the function to all the elements of an array with one line of code. None of that

let's create counter,
create the loop structure,
create an output array,
pass the data to the function,
dump the output data into the array,
and increment the counter to get to the next piece of data

process. You have a spoon and there are many buckets of ice cream, what else do you need to know? Functions and list comprehensions work just like that. In a single line of code that 6 step process is gone. Code that can comprehend lists are amazingly short and the hard thing to do it limited to writing the function.

I work in Python and I think in Python when I program. Then I come back to VBA, it doesn't have the syntax to do that. What usually took me 1 short line of code takes me 4~7 lines of longer code to do the thing. It's a waste of time and prone to error.

So basically, I've spent the last 2 days making VBA more Python like by creating the data structures to alleviate the lower level management of data array comprehension to the point where I can throw arrays around and process them in 2~3 lines of short code. It's not a perfect solution, but much better and far more manageable. Compared to the old procedural code that I used to write, managing old procedural code is a nightmare to maintain. The great thing about list comprehension is that you can worry less about the state of the program because the code becomes stateless. None of that "what was the value of the counter?" and that pizzaz.

There are times when the state of the program is important however, like for instance a user application: What screen am I on, or what settings have I changed, or am I still connected to the internet or something like that. But still, the overhead of dealing with states associated with data greatly decreases with list comprehensions (there will be cases where the data will require state changes in a function, but the overhead is greatly reduced).

At the time, instead of having multiple lines to describe a process, you can have just one which is far easier to understand because the unimportant scaffolding is hidden; the only thing showing are the important parameters and the name of the function. That is how good code is written. So farewell to the terrible looping structure, I've gotten rid of most of them and now left with the descriptive code that tells me what it does and what it operates on.

That's the beauty of higher level computing languages, in that you're able to do a lot with saying a little. There are even higher level ones where you can define your own keywords and syntax to them. A not very well known language is Lisp that allows one to do just that. I have no idea what is possible when one is able to define one's own language to suit whatever problem they are tacking with but I am quite sure it would be a very fascinating adventure. I've already seen my programming skills improve greatly by using Python. I can only imagine what else is there to learn by learning higher level languages.

The mind is a lot fresher after spending time away

2011-06-07T10:49:00.001+09:00

I have found that I need to take 2 passes at a problem before getting it right and usually the second pass needs to be done after spending a day away from the task. My mind feels significantly fresher after spending time away from work and then coming back to a task. Things that I didn't notice before just jumps out at me, where as in other cases, I could look over something repeatedly and still overlook details.

I find this to be true to both writing and even programming. The first post of anything I make tends to filled with errors and even after double checking, I still tend to miss a lot of small mistakes. It's likely related to a bad habit of wanting to get something done as fast as possible and sometimes I find double-checking as a tedious task that gets in the way. One hitting the submit button, that feeling of needing to have something done as quickly as possible usually subsides and I am able to look at past work with a more critical eye.

There are times when I have been coding to find myself coming back to the code wondering to myself "what the hell was I thinking" when a much simpler solution exists. This is especially true when I make the mistake of designing a complex solution and thinking of the solution as clever.

For example, I managed to write a text parser with the feature that it could automatically detect if a text string was an integer, a floating point number, exponential or a date and automatically call the correct function to parse the data. The function was also extensible in that it could also be updated on the fly to auto-detect and parse other values.

I thought the smart thing to do was to create 2 functions, 1 to detect if a text string was of a certain type (returning true or false) and then a second function to perform the conversion and I had to make an elaborate system to keep the testing function and the parsing function paired together which I thought was kind of unweildy, but I managed to do it. A day later, I realized that I could just have 1 function that would either return a value or nothing depending if the parse failed or passed and used that as the indicator if the right function had been called.

I could have been bashing at a problem through an entire day with an inelegant solution... and I deplore inelegant solutions because why work hard on a bad solution when you could be spending time on a better one instead? Thinking like this is both a blessing and a curse because the resuts of my work varies between "really good" or "none."

Anyways, hopefully I'll be getting back to posting on a more regular basis. I have more pockets of time where I am able to think compared to before and I hope for this to continue.

Less time online and improved concentration

2011-06-06T15:22:00.002+09:00

I've been getting back in to studying for the CFA exam and one thing I noticed is how terrible my concentration was. Spending one's time reading online articles vs reading books is quite a different experience-- a large amount of information has been accumulated into a single place instead of scattered through a variety of webpages that one would have to usually have to search through. Having a good table of contents is also a boon to immediately pinpoint where in a document that you would need to go to get the information needed.

By having a large volume of high signal-to-noise information available, it significantly cuts down on the time required to search for information (which I think of as a distraction). The resulting effect is that having good books to do research from actually helps improves one's concentration instead of being distracted by looking for information (which may or may not be relevant).

One other interesting thing I've noticed about myself is that I tend to focus way better when I am working with other smart people. I believe that having the synergy of having a few smart people around to cover for information gaps or thought gaps cuts down distractions by the need to look for information. I believe that addictiveness of, say, video games in general can be attributed to having all of the information a player might need very accessible through a very intuitive interface or by having an environment where one can immediately figure things out with simple tests.

Right now, I think that the internet isn't an ideal place to learn in depth topics through websites as either the quality isn't there, finding good quality content is simply too much work or the information that you're looking for isn't covered in significant enough depth. I think that books and other resources fill those gaps.

One project that I would like to work on is to start creating a repository/network of high signal-to-noise sources and will be looking into a variety of tools to help me do that.

Cutting down on wasted time

2011-06-03T09:51:00.001+09:00

I've been working on cutting down wasted time recently, considering that now with the commute that I leave the home at 7:30 am and don't get home till past 8:30 pm, I have come to the realization that I have little time to myself. It is paramount that I make the best of it so I've been working on cutting down the number of distractions and time killers.

One of the best moves I've made is killing the habit of flipping on the computer immediately after getting home. I don't have a TV and have been living without one for the past 5 years without one and to take its place, the computer has become my media center. The problem is that the way I am spending time on the computer is just as bad as the way I used to spend on TV-- aimlessly doing nothing.

Instead of flipping on the computer to aimless browse at things, am becoming more stringent with my time allocation. First thing I do after getting home is either cook or exercise. I've also started making a new habit of not riding the train all the way home and getting off at 1 stop before I get home. Actually, it's a transfer stop and instead of spending time waiting for my transfer, I just hit the street and jog the rest of the 3 km home. I've managed to do the jog in about 15~16 mins and it's turning into a pretty good exercise routine.

Running this distance is actually quite significant when it comes to reducing body fat as I burn about 200 calories. One average meal for me is about 500~600 calories so I am burning off the equivalent of 33%~40% of a single meal. Assuming that I do this 3~4 times a week (ie when it doesn't rain) the reduction in the amount of calories the body absorbs is significant in addition to the health benefits of jogging.

I've also allocated about 1 hour almost every night to studying for the CFA exam (at a rate of about 20 pages/day) and have made this a prerequisite to turning on the computer. It's been about 3 years since the last course I took in university and it's rather refreshing to start studying again. One of the great things about reading study material is the much higher signal-to-noise ratio compared to reading articles on the net. The quality of information in books is far superior to what is published on the net. I've also noticed that as much as I like digitized information, I find that I like writing out my notes; there is simply a freedom in penning out notes that isn't available compared to typing out information (writing diagrams and sketching arrows is still ridiculously slow). Now that I think about it, if there was a great way to pen notes digitally and organize them, that would be awesome because I am one of those people that can easily generate volumes of notes.

I wonder if there are good note taking software where you could both type and use a stylus to sketch in other information. Like arrows, lines and even equations.

Once getting into a groove of "getting things done in rapid succession," I've found that a momentum kicks in. In the sense that I am far more likely to want to move on to the next item on my mind that needs to get done, without hesitation. Looking back at past behavior, the biggest hindrance to action is thinking about all the things that I ought to be doing and not knowing which of the things I should just simply do, because I have a tendency of worrying about "is the thing I am doing the right thing to be doing?" I've moved on to killing that by coming to the realization that time is scarce and doing something is better doing nothing. I've become far more effecting at using my time when getting at home, far more compare to when I was living closer to work and I find that ironic.

A change in environment is a good thing.

Working on my first patent

2011-06-02T11:25:00.001+09:00

I've been charged with the design of some new optical systems based on geometry and varied materials to improve the brightness of light emitting devices. One of the great things about doing simulation work is that I can create 3D models, change materials and configurations faster than it would take the people fabricating the devices to create and test the devices built, because semiconductor fabrication is tough work.

Meaning that I can likely iterate through far more designs and variations than the processing people can go through in the same amount of time, probably by factor of x5 ~ x10 easy.

As a result of the work I am currently doing, I've been able to write a library that allows me to quickly generate 3D models, simulate them and analize the data faster than the guy that taught me to use the software (and has about 2 years up on me in experience using the simulation program). We tried to bring in another guy from a different department to do simulation work, but after 6 months of work, he eventually burned out... but I digress...

Now after getting out of the clean-room and having reasonable amounts of time to spend thinking and analyzing information, I have already found ways of increading output of our devices between 20%~50% with out standard models and I have identified a possible way of pushing that up to the 60%-90% range with some designs.While I am at it, I figured that it would be very cool to put out a patent to have something under my name.

Applying for patents is an expensive process with fees for the application and additional fees to maintain the patent over it's period. As an individual, it might be prohivitive to do this kind of stuff, but if you're working for a large corporation that can do this, why not?

Unfortunately, the patent review process takes approximately 3 years to go through, but I figured that the earlier I get started, the better. I just need to get it through the internal review process and find time to talk to the legal department here... this is going to be interesting.

So many thing to learn, so little time

2011-05-31T14:26:00.001+09:00

One good thing about moving out from where I live now is that I value time much more compared to what I used to previously. Meaning that despite having less time, I am aiming to do more with it.

The current things I have on the chopping block are the following:

Take and pass the level 1 CFA exam in the winter
Work out 5 out of 7 days during the week for at least 30 mins
Pick up emacs (for text editing and programming)
Learn Ruby on Rails to understand how to develop web applications
Dedicate atleast 30 mins/day to learning Japanese for the JLPT exam in the winter

Time is incredibly scarce right now and I am pretty much at the point of even using small smidgens of time at work to help myself accomplish these goals. Just glad that I can make time to do things at work instead of doing some manual labour job.

Living with the commute and etc

2011-05-31T09:25:00.001+09:00

It's been about a month and a few weeks since making the move from the company dorms. I pay a higher rent now but having my own place where I can invite people over is nice. I now live about 20 minutes outside of Tokyo, instead of 1 hour. The trade off, however, is that it now takes me an hour to get to work instead of the 10 minutes I used to spend before. The time delta of 50 minutes times 2 is actually quite significant. The slate has an article on commuting entitled "your commute is killing you," explaining that the further away you live, the more your health, social life, productivity and marriage suffers. Fairly obvious stuff, considering that you have less time to do everything else. 2 hours out of 16 waking hours spent on commuting is more than a 10% of productive time, which is painful.

On the flip side however, I do enjoy living closer to Tokyo and it has made getting out into the city on the weekends far more convenient instead of the 1 hour ride if I wanted to get out there. I can spend more time with friends compared to what I used to be able to do before. Just as important, however, is my ability to network with people in Tokyo will improve as Tokyo is far more accessible to me.

What I'll be looking forward to doing over the course of the next year is to get acquainted with the hacker community in Tokyo, hopefully find some intelligent, driven people to work on some projects and bounce around ideas. I've already found the Tokyo Hacker space where they do meet ups and work on projects and also a Tokyo Startup Weekend event which happened last week. Unfortunately I wasn't aware about it and it ended just a few weeks ago but I'll be looking to become more involved with the entrepreneurial communities here to find cool people to do stuff with.

I've been meeting lots of people recently, but there aren't a lot of people that are interested in doing business or have a business like mind so I'll be spending this year searching for entrepreneurial communities.

Am alive, just been busy moving

2011-05-16T00:21:00.001+09:00

Completed the move to the new place just last month. Will have some more details about it in a bit. Been quite busy getting the furniture and unpacked. Normal semi regular posting will resume in a bit.

Financial wealth vs physical wealth

2011-04-25T16:16:00.001+09:00

Been reading about the commodity bubble and crash of 2008. It is interesting to try to understand how the motion of money that causes the commodity bubble. At first, I was thinking that money can move into a commodity class (as in be stored) but in reality, someone is exchanging a commodity for that money, which breaks that model. I wanted to model a change in price as a function of the amount of money that goes into an equity, but that doesn't seem to be the case. What basically happens is that the money is exchanged for a commodity or equity and the transaction ends there.

The value associated with an equity is the result of it's instantaneous trading price and there is a disconnect when the value of the equity is tied directly into the "real" money supply and the physical commodity.

Assume a closed system of $1 billion dollars and 100,000,000 tonnes of copper:

Let the price of copper be $10/tonne, the total value of this market is $2 Billion
Let the price of copper be $30/tonne, the total value of this market is $4 Billion

Just because the price of copper was more expensive in the second case at $30/tonne vs $10/tonne does not necessairly mean that one economy is more "rich" when the amount of physical resources remain the same. I would be much happier to if I (and others could) a good $100,000 house versus a terrible $500,000 bad (top of the line) one.

I think that there is a gap if the measurement of "wealth" is "money."

The balance of living in a one room apartment

2011-04-13T09:51:00.002+09:00

Another 3 days to move into the apartment and packing is underway. The number of belongings are less than expected; I regret not haggling with the moving company for an even cheaper price. I looked up the weather this weekend and there is a 50% chance of rain. I've never had to move in the rain before and it will be interesting to see how the moving company deals with it.

I did some more research last weekend and found out that renting a truck for a day was 8000 yen compared to the moving fee of 40,000 yen that I am paying. The only catch is though my driver’s license in Japan allows me to drive stick shift, I have little experience at it (perhaps 2~3 times). I would be interested in renting a manual car and learning stick because saving 30,000 yen for a single day's work is worth the effort.

I spent the last weekend looking at furniture for the new room as it will be a much bigger compared to what I have now. I can fit a sofa, bed, a desk and my weight training machine and the question is, do I really want to buy so many things? One of the things that I really liked about living light on belongings is that it is easy for move around (though I haven't been taking advantage of that).

Though it has been tempting to buy furniture in advance and have everything arrive on the same day, I have decided that it would be better to live in an empty space first to get a feel for the room before loading it up with stuff; I would rather have bought things to realize that it doesn't fit in well with the room. I'll be pretty happy with moving as it'll be really nice to be able to invite friends over.

On preparing to move

2011-04-12T10:46:00.000+09:00

There are hidden costs (and benefits) to everything. I've been living in the company dorms for the past 3 years and by virtue of the age limit I am moving out by the end of the week to my own place. Leaving is a blessing because despite the savings in rent ($200 monthly), I probably have incurred plenty of social costs by living in the dorms. As a result of living in the company dorms, I am far away from Tokyo and can't invite guests over because of the building rules. As a result, I only see friends when invited to an event or when I host something elsewhere.

One part to living is the social aspects of living is knowing people and being able to do things/interact with them. The living conditions up until now have prevented me from doing as much as I would like to have done. Though in retrospect, I did end up doing more traveling around Japan from the savings from really cheap rent.

I have managed to find an apartment just outside of Tokyo, a brand new building and I will be the first tenant into my apartment. It is a fairly spacious 1 room apartment with an insulated floor and I have managed to select a corner room, where to one side is a wall shared with the outside and the other wall is shared with an elevator, meaning that I have free reign to play music and watch movies as loud as I would like and am incredibly happy about this.

I face one other dilemma; in that the room is really spacious and I would like to buy some nice furniture go into where I live. The problem I fear is having too many large belongings that eventually tie me down to a place.

The best solution I can come up with so far is to make more money to make the costs of buying the furniture insignificant to my income, making what I buy "disposable."

The post earthquake situation in Tokyo

2011-04-11T09:44:00.002+09:00

For the large part, life in Tokyo has largely returned to normal. There are still some noticeable after effects of the earthquake still lingering in the city however. Since the shutdown of the Fukushima reactors, the entire Kanto region (the region surrounding Tokyo) is running at reduced power capacity and the entire region has managed to cut power consumption by at least 25% since the earthquake, alleviating the need for rolling blackouts through the region.

A link to the power charts for the Kanto region is provided here. The blue dotted line represent's last year's power consumption, the small blue bar is the total peak power available and the pinkish line represents yesterday's power consumption and the blue bars represents power consumption for the for the current day.

It's rather fascinating to see how much more power people and companies are able to cut and still maintain fairly "normal" operation. I doubt that you will see this level of power consumption elsewhere in the world however, as the attention to detail and the character to voluntarily reduce consumption is the same with other countries. The above graph could be a representation of the amount of "disposable power" Japan has.

In more detail, office buildings have reduced power to lighting, reduced the number of elevators and escalators operating. The trains have turned off the air conditioning and turned off lights inside the cabins during the days (it gets incredibly dark when going through short tunnels and really shaded regions). I expect power consumption to slowly rise as industrial regions will slowly rise as companies start brining more equipment and facilities online.

We have yet to determine if the power shortages will continue through the summer as one additional power plant is scheduled (since before the earthquake) to be taken offline for inspection through the summer, possibly bringing back the blackouts. There have been talks through the news that this might be not necessary as we have been seeing unprecedented levels of power conservation.

Supermarkets are currently short on food products as the shelves for certain products are either empty or only half full; coupling that with the current dim lightning resulting from power conservation, it presents a rather dreary atmosphere. The supermarkets are short on certain vegetables, milk, soy bean products and bottled water (which were quickly snapped up because of fears of water contamination).

The schedule for the current project that I am working on is being pushed back by about 6 months (and we are really rushing it at this rate). I expect that much production in Japan be significantly reduced and research delayed. Since the currency jump after the earthquake, I have decided to move some of my funds out of yen and into other currencies as I expect the yen to weaken throughout the year. Already in less than a month, the yen has dropped significantly.

I expect the earthquake will send slow shockwaves throughout many technology companies throughout the world as Japan produces many high tech components in many electronic devices. This might be an opportunity for other emerging market players to pick up the slack for reduced production from Japan. Right now, this probably is a very big opportunity for other Asian countries like Korea and China to play technological catch up with Japan. I expect competition to be fierce once Japan gets back on its feet.

Macbook

2011-03-26T20:00:00.001+09:00

So I managed to get my hands on a macbook recently and started learning how to use it. There are obvious differences when it comes to using one of these machines compared to a Windows or Linux machine. One thing that drives me up the wall is that I still haven't figured out how folder navigation works other than double clicking on icons to open them; the standard "hitting enter" on files or folders doesn't work here.

There are other notable differences; for example, the function (F1~F12 keys) are actually mapped primairly to notebook functions (like screen brightness, volume control and etc) and you have to use a function key to activate their "normal" behaviors (ie function-F5 for refresh). There are, of course more things that I need to get used to, like how applications are installed and removed (things work slightly differently compared to usual Linux machines) so it'll take some time to figure those things out.

I still remember my first venture into the linux world after buying the first version of the EEE-PC. It took a good week of tinkering to really figure out how to do software installation and figure out what packages I needed for things like multi-lingual support (I also ended up killing the OS a few times by deleting the wrong things too). I suspect that I'll probably fall through similar pitfalls while learning how to use the mac.

One thing that I do like about using the mac, however is the polished multimedia editing tools, which are leaps and bounds superior compared to the open source alternatives. I guess when it comes to software that you pay for, it is what you get.

Interms of the interface, I like it because it has a cleaner look compared to the usual windows interface and the applications look pretty good. Still haven't had a lot of time to tinker with things but will be looking forward to playing around with the computer some more when I have time. After pretty much graduating from playing games on Windows, there really isn't much of a reason to stick around to a single platform anymore.

The effects of radiation being felt in Tokyo

2011-03-15T18:48:00.001+09:00

A friend of mine posted that the "air tasted like metal," though unaware of it, those are signs of elevated radiation. The government had reported elevated levels (though not physically damaging [yeah right?]) of radiation in Tokyo this afternoon. Work has been cancelled for tomorrow and the day after so I am making plans to relocate out of Tokyo for at least until this weekend.

CNN's coverage of the Earthquate is crapy

2011-03-13T17:05:00.004+09:00

I generally abhor news reporting just because they always focus on disaster, scandal and damage all the time. Maybe it is an American thing, but for the first time in a long while, I am currently at a friend's house watching the news with a Japanese news feed and CNN's news feed side by side. And I have to say, I am in awe of how shoddy CNN's reporting is. Nothing but the same clips of the Fukushima explosion over and over again, exceptionally dumbed down explanation (read, elementary level?) explanation of a nuclear power plant and more. I am surprised that they do not have any bilingual reporters translating and reporting on the direct press conferences from the Japanese government.

Though the Japanese news stations, through do replay some clips of the damage out here, I am far more aware of the evacuation process and the condition of people around the Sendai area; which I think is far more important compared to just talking about the "massive damage" from the earthquake.

News organizations have too much power over the way people form their opinions about the world and I don't think they are good enough to handle that kind of responsibility.

Backing up data as a precaution

2011-03-13T10:58:00.000+09:00

Going to be leaving the home for a bit to meet up with some family. Decided to backup some critical data to the cloud, just incase I won't be able to return. In disaster, the internet is incredibly handy.

More info from a friend in Sendai

2011-03-13T10:53:00.000+09:00

More people are coming online on Facebook reporting that they are ok. Seems like shelters are taking care of people there. One friend was able to cook several days worth of food. My friend reports that there is no power or electricity. Seems like power is up and down depending on location.

People tweeting real time on the news

2011-03-12T22:06:00.000+09:00

Been watching the news online and there is an integrated tweet feed. Most tweets are pretty useless and I wish there was a way to block them.

Note to self: look at sources of reliable and unreliable tweets.

From the Sendai City webpage

2011-03-12T21:46:00.000+09:00

For those that are interested. Sendai city home page is up:

http://www.city.sendai.jp/

-Bus service is intermittent in some places, operational in others
-Hospitals accepting patients in severe conditions
-People are recommended to stay home
-Natural gas service is intermittent
-People without water can have access through sources at schools
-Garbage collection not in service

Hitting the supermarket in the morning was smart

2011-03-12T20:23:00.000+09:00

The super markets and convienance stores were busy today. People have been stocking up on food since the quake. I now have about 1 weeks worth of food cooked and prepared. Other friends have mentioned that some stores are empty of food that can be immediately consumed.

Japanese reporters on radiation levels and evacuation distances

2011-03-12T20:18:00.001+09:00

Looks like the reporters are seeding panic by wondering if the evacuations distances are sufficient. They are worried about the government covering up information and if the evacuation distances are sufficient. Exceptionally pissed off at the talking heads that they aren't doing any research.

I wish there was some kind of market to force them into a bet... where I can bet against them.

Sendai Status via Twitter

2011-03-12T19:43:00.000+09:00

For those that know Japanese:

Via Tetu on Twitter: http://twitter.com/mahhya94/status/46425985838620672

青葉区五橋の姉と連絡つきました。13：22現在避難はしていない様子。街中は建物もそんなに壊れてないと。 30分前13時ごろ電気復旧。水も復旧

Translation:

Go in touch with sister in Aobaku-Itsutsubashi. As of 13:22 does not seem to be evacuated. Buildings do not seem to be that damaged. At about 13:00, power restored, water running

Via Tetu on twitter: http://twitter.com/tetumemo/status/46476055724298240

【追加情報仙台市の友人から】仙台市役所前のみ電力供給あり、取引先の会社へ避難中。　長町南から長町駅、商店街付近は建物の倒壊は無いものの道路がうねり、交通が困難

Translation:

(Further information from friend in Sendai)Power only available in front of city hall, taken refuge in company building. At south Naga-machi, objects on streets not from fallen buildings on streets. Traffic is hard.

I've heard further reports that water isn't running in some areas...

Fukushima Nuclear plant explosion:

2011-03-12T19:08:00.000+09:00

Calculations on the radiation levels:
Current reported dosage levels @ 1000*normal.
Normal radiation dosage @ 2.4 mSv/year
need 1~2 Sv dosage to start to feel nausea or vomiting -> time to that occurs is about 5 ~10 months assuming that you are *at* the power plant.

Dosage levels will probably drop in an inverse squared power law away from the power plant. Meaning we've got plenty of time to react.

Sources:

Radiation rate @ 1000 x normal: http://bit.ly/eGht2S
Radiation poisoning levels: http://bit.ly/bRheP6

Quake notes

2011-03-12T15:08:00.000+09:00

Random news articles:

News Streams:

Facebook, faster in disasters

2011-03-12T10:00:00.000+09:00

Spent most of the evening and night yesterday on Facebook reading from friends reporting in that their families are ok in Sendai. SMS and cell phone contact is really slow and I thought it was ironic that it was faster for me to Skype home and contact people through facebook. I'll make a mental note of that for the future. So much information just came so fast that I hadn't the chance to organize it into a post... the damage in Sendai is extensive and we are still having aftershocks.

I think I am going to take it easy today... wonder if the grocery stores still have food.

Earthquake, I am ok

2011-03-11T18:16:00.000+09:00

Incredibly sad the earthquake hit Sendai. I've been to some of the places they've shown on TV with damage. I am ok at least. Lots of information going through Facebook right now. Checking with friends in Sendai to see if they are ok.

Thinking about the Monkey Portfolio

2011-03-01T08:40:00.000+09:00

In the old days of high school we used to have multiple choice scantron tests and we used to call the monkey score the score you would get if you just randomly started choosing answers. We used to have 5 possible choices on these tests so the monkey score would be 20%.

I was having a conversation with a friend recently about financial planning and the guy is a financial planner actually. During the discussion my friend proclaimed to me that he could structure a portfolio for clients with different risk characteristics with averaged annual returns ranging between 6%~12% and that he would recommend a 12% portfolio for young people for fast growth. His reasoning behind that was that you could take any 10 year window of the stock market and find that the market compounds at an average of 10%. Since he worked at a financial institution, they had better tools to get better returns which warranted that they could get that 12%. I decided to take his statements to task to really see if this was the case... unfortunately it wasn't.

To be generous, I decided to take a nearly 30 year span of the stock market starting in June 1981 until Feb 2011 (chosen at random) to calculate compounding rate. As of June 6 1981 the Dow Jones index was at 976.88 compared to Feb 1 2011 value of 12130.45, which is a 12.4x increase. Calulating for the compounding interest rate, all I would have to do it take the root of this between the time span to find that the compounding rate to be approximately 8.86% over a 30 year period. The result is striking to me that a 10% compounding rate is optimistic at best and 12% is a bit out there.

The Dow Jones Index since June 1981 to Feb 2011

So next, I decided to do take this a step further and assumed that I invested into the market at a constant rate of $1000/month for whatever the price the Dow Jones Index was going for until the beginning of Feb 2011, what would the annual compounding rate be after ~ 30 years? The result is illustrated in the graph below:

Compounding rate calculated while making constant contributions

Sure there is a lot of movement in the compounding rate in the initial few months but as time slogs along the compounding rate begins to stabilize at 6% but drops down to 4.7% after the recent economic crash. From the results we are nowhere close to achieving the 10% compounding rate that was initially proposed. Just as a note, the graph bottom axis is in months but has the same time span as the above Dow Jones plot, I was just too lazy to change the axis.

The gains however are still significant even at a lower compounding rate though, assuming a $1000 contribution into the fun over 356 months, the total contribution turns out to be $357k (assuming +$1000 at month 0) with a portfolio value of $1.38 million after nearly 30 years of constant investing. Sure the market might have had compound growth in the upper 8% range, but the dollar cost averaged purchases of stocks over the past 30 years tells a different story. The guy was obviously making a pitch at me and setting reference points is a common sales tactic, but the reality of the matter is that a 10% continual compounding rate is optimistic. When it comes to financial advice and products, keep your wits about you.

Some notes on web programming

2011-02-15T22:19:00.003+09:00

It seems to be taking forever for my web domain to get registered. After registering for a webhost on Sunday, 2 days have passed and my application to get a web domain is still pending. Not particularly happy about that but there isn't much else I can do except wait, though 2 days seems like an eternity to me.

While doing the waiting, I am looking into the following things:

How to setup an Apache server and get CGI includes for Python working
How to create interactive applications for webpages
How to manage data using a SQL type database
How to create web applications on infrastructure on the Google App Engine or Amazon's EC2
How to program websites with AJAX (since I don't like web pages reloading on me after doing a query)
How to use Python to generate html templates or generate CSS templates through Python
How to create menus, buttons and images through programs since I am too lazy to use things like Photoshop to do tasks like these manually

I have a feeling that we are going to see a trend away from desktop data management happening as online servers will be dealing with data management and home computers will become more like clients in the future.

I need to certify myself as fluent in Japanese

2011-02-14T21:43:00.002+09:00

Learning a second language is a pain in the ass, especially when you have access to internet in English. It's a terrible habit of mine, that I exist in a bubble of English and use my Japanese only when necessary. For being here for 5 years, I should be able to do better.

I've also been looking at new job opportunities, people that speak both English and Japanese fluently have a leg up, there are lots of interesting opportunities out here to work in a multinational firm or setup new businesses, especially when foreign companies need people that understand the culture and want to setup base here. I have had my nose buried too deep in work and haven't been working on my Japanese as seriously as I should. My bad.

I want to raise my salary by 25~50% by the end of this year by improving my Japanese skills and picking up some marketable skills for use in different environments. One thing I have learned about working in a corporate environment is that you'll get nowhere if you're a bad communicator.

Foray into web publishing

2011-02-14T01:15:00.002+09:00

I've finally broken down and bought an account at a webhost today. The site I chose was iPage, which allowed me to register with them, get a free domain name and hosting for a year for $54. Not bad considering that I now want to learn the basics of hosting web content on the net. I've done this kind of stuff before, but on my personal machine in the past. If I am to create more intricate web pages and possibly applications, I should learn how to host them on the internet, which means learning how to use a webhost.

I made one mistake with registering with these guys however, the problem being that they do not have SSH access into their servers, meaning that I cannot run command line related stuff to check the version of the software they have, but they seem to have a large repository of interesting packages to play around with. Later on, I may see about using Google's app engine or Amazon's EC2 frameworks for scalable software. It will take me time to learn things but I think there is interesting work to be done in this field. This blog may or may not move, but I am in the process of registering my own domain name and interested in developing my own web presence.

Contemplating getting a macbook

2011-02-14T00:57:00.002+09:00

The last time I was in Canada, I was back home to be the best-man for a friend's wedding. My dad, being a Mac convert had a few Macs lying around the house and I found out that their iMovie software was pretty damn good and made a slideshow movie through using the software. I was thoroughly impressed with the bundle of good media software with the Mac. It is as if the Macs were specifically designed for media kind of work. I never thought twice about getting a Mac since I switched over to Ubuntu-Linux, which is a packaged version of Linux. For free, it's pretty damn good and has nearly all of the basic functionality I need.

I've noticed, recently there are some rough edges that need some smoothing out, mainly in the section of professional software. The great thing about linux software is that it is free and they work pretty well-- I can pretty much do nearly just as much, if not more on a linux machine compared to a windows box, for free and even better. I really like how fast Ubuntu boots up, how quickly I can log in and how responsive the machine is even right after logging in (there isn't very heavy background software loading after you log in). It's pretty tight software. The only thing that is lacking, perhaps is a really good video editing suite or media management editing suite. The one thing that I liked about Apple software is the level of polish that goes into the software available for the Macs-- they simply work and usually are quite cutting edge.

Compared with the Microsoft ecosystem of software, I've found that looking for windows based software on the internet to be rather spammy and sometimes the software sites on the net can sometimes even seem a little dubious. Perhaps it is a result of a smaller and tighter-knit Apple perhaps? I am not entirely sure, but we are starting to see something similar happening to the iPhone/iTouch app store where sometimes dubious software sometimes does pop up (though putting software up there is done through an approval process which has wrought ire from some people).

Anyways, there is a bit of a premium when it comes to buying Mac hardware, but since the base of their software is Linux based, it shouldn't be too hard for me to pick up. The question is if I want to start off with a laptop or just an iMac mini (which I think it kind of pricy at $799!).

Keep a spare keyboard

2011-02-14T00:49:00.002+09:00

The keyboard managed to die out on me while doing some administrative stuff on the computer. I've been working on a wireless keyboard and mouse since the summer and have been really enjoying it. Sure the battery dies once every 3 months but it wasn't much of a big deal, until the keyboard died on me that is.

After wondering if the keyboard was in a bad position, the receiver was in a strange location and wondering if the battery was dead, I have pronounced the keyboard dead. Generally, indicator LEDs aren't standard with wireless keyboards because they shorten battery life, making debugging rather tricky. I eventually had to dig out an cabled keyboard to connect to the desktop... only to find that the keys were rather sticky. Then it struck me... I spilled some really sticky liquor on the keyboard since the last time I used it and quickly took it to the shower where I doused it in hot water.

Suffice to say, I won't be able to use my desktop computer today and am currently using the internet through my netbook. I've put an order for a new wireless keyboard online and the total cost for a new one is about $25 bucks, including free shipping. Not bad, it's something that I can live with.

Sometimes, you have to entice people to commit

2011-02-11T09:49:00.003+09:00

Working alone is tough, there is only so much memory, attention and knowledge that a single person has. Not to mention that the cutting edge of technology is a moving target-- what you don't release this year might be outdated by the next. If you want to make a killer product of service, you have to make something good and do it quickly. My best analogy to this would be like learning to ski well-- learn to lean forward, even to the point that you think you might do a barrel roll, do that.... and lean forward some more.

I've been in a recruitment phase for finding a community of smart people to work on projects with and eventually turn those into a viable service or product and there are great opportunities out there for data mining/business intelligence type applications. There is so much on the internet that we have just begun to scratch the surface, but people know that the market is there and there are many startups that are moving into this realm.

There is only so much that I can do on my own and that has been one of the biggest realizations for me. Yes, it is true that I could probably learn what I needed to know to do something, but it takes time to be a master at sometime. If you truly want to do something big, you're going to need a team of people to do it, a tight group of smart people.

I've been making some programmer friends recently and I've made several pitches to try and get a project off the ground. Yes, I can convince them that some of my ideas are interesting, I also enjoy bouncing ideas to find new avenues of use. I have some code written and some proof of concept, but I need to learn how to make people to commit. I tend to generally be soft on people, like when I tell them "spare whatever time you have" or "get things done whenever you can," to be honest going easy on people like that is a mistake and you'll get no real commitment from them; unless of course, that the people you are talking to implicitly feel strongly that you are going to succeed with or without them (but still, it is a weak play).

If you want people to commit to working on a project with you, you do have to entice them. Show them that the project has momentum, that you are strong willed and believe that it is a good idea and that if they want to join up that they have a short but reasonable time to do so, because people usually don't do things unless there is a deadline to for decision; it is simply human nature (oddly enough, I think it is similar to dating and that you shouldn't give a person that you really like the opportunity to go out with you whenever they feel like it because you are implicitly saying that you don't value yourself that highly... but I digress).

In short, you have to act with urgency and find ways of making what you do a priority for other people if you want them to join up or else things will happen "whenever," which translates to "never."

Now a fan of audio books

2011-02-09T21:17:00.002+09:00

I had an American friend when I was back at university in Sendai that loved his audio books. While working in the clean room and processing his devices, he'd be in his own world listening to whatever audio book in his iPod. I never really understood audio books back then because I read much faster compared to then I could listen and I would get irritated by listening at the pace of the speaker.

Fast forward 3 years, I started listening to a few audio books while exercising, riding my bike and while on the train. The lack of having a (sometimes) large, heavy book with me was definitely a good thing, because I had less heavy things to carry around with me and I'd be learning during times when I had nothing else to do, which means time productively spent.

An audio book can take about 6 hours to listen to (about 1 hour/chapter) but it isn't bad considering that for a normal commute to Tokyo takes about 1 hour one way meaning that I can finish 1/3 of a book just on a trip out to Tokyo and that doesn't include the time for me while walking around. The other nice thing about listening to audio books is that I find the information content from books to be much more nourishing as the topics covered in books are covered with greater depth compared to what you generally find on the internet. The only disadvantage for audio books is that books discussing complicated math or needs graphs to explain ideas becomes very hard to comprehend, so topics that do not require a lot of "rigor" are usually good to listen to. Hopefully I'll find some good Japanese audio books to further develop my Japanese skills.

Updating the resume

2011-02-08T12:32:00.002+09:00

It's been a long while since the last time I updated the resume. The last time I had updated it was nearly 3 years ago, after graduating from my masters program. I am (un)offically in the hunt for a career adventure. I believe that I can earn more and grow faster than what the managers think that the average person can over here.

I mentioned earlier that I wasn't so hot about waiting 4~6 years for a promotion. I think that one of the biggest faults of an organization is limiting the speed at which a person can climb the ranks is a fatal mistake, since no one will have an incentive to take on more responsibility than they have right now. But hey, that's bureaucracy and politics for you.

Interestingly for the past 3~4 months, I have been looking at other opportunities in different organizations, the pay is dramatically better and pretty good for even a young person like me. I've started sending out applications... no real bites for positions just yet.

It's been a long while since I did any real work hunting, preferably, this time in a more "Western" organization as I find my mental capacities are much sharper when I work and communicate in English. Good Japanese skills here are very important and can even improve the value of a worker too and I intend to work on that as well... I just don't have an incentive to do anything at the place where I am currently employed. But I digress...

What I realized after getting into the working world again (since the last time I did any kind of job hunting when I was graduating from my undergrad studies) is that the nature of the resume has changed. I was good at marketing myself as a student with my resume, which was padded with a lot of good academic achievements, what I realize now is that I can't play those up as much any more and have had to chance the format of my resume to reflect the kind of work and my new work interests.

Not in all cases, can one send a cover letter out for each application as for the recruitment sites that I've been working through now only use the resume to evaluate a candidate, to which I realize my resume is too engineering heavy for the positions that I've been looking at.

The other thing about looking at job postings, especially at recruitment sites is that they have concrete job descriptions and approximate salary ranges for the kinds of people they are looking to hire, which is really different compared to the standard Japanese system where you just apply, then they accept you and then figure out where in the company they want to send you. I am far more comfortable with the normal western hiring practices.

I've made the mistake of sending out too poorly tailored resumes to too many job openings that there aren't any new good ones at the moment so I am going to have to slow down and think of strategies of improving my skills set to get a good position elsewhere. It is an interesting experience to be back in the work search world again.

Giving myself no choice but to succeed was a very strong motivator

2011-01-31T12:47:00.003+09:00

[no text]

Success is the result of doing worth while things for people

2011-01-30T19:35:00.005+09:00

I throw a large events a few times a year, something sort of a reason to get friends together to hang out. Yesterday, I threw a late post new year's party for friends that haven't gotten together for a while and invited out a few more friends to the party. The event was fun as I had about 30 people attending.

One thing I miss about living back home is the proximity of friends and the ease at which everyone can get together at a moment's notice. One point my old friends and I used to pride ourselves upon, was our ability to get together and do something in a very ad-hoc manner. When living in Tokyo, things are a little different.

I probably mentioned this a few times before, how much of a pain it is to get people in Tokyo together to do something. For a meet up to actually occur, people have to make time usually a week in advance else things either get filled up or people aren't prepared to make the time investment to come out to meet meet up. When a one way commute is only 10~15 minutes to meet up, the time investment is trivial. However, when a one way commute takes from 45~60 minutes, things aren't as trivial anymore. I balk at the idea of an impromptu meet up sometimes because the time investment is quite expensive. The dynamics change when you live closer to the city, of course, but most people are surprised at my ability to coordinate events while living far from the city. It's one of those things that I just have to deal with.

The other idea I was pondering today was "what is the primary factor which allows a person to be able to gather people to events?" I have several friends here that are event promoters, either for clubbing events or musical events. I feel for these guys as they are either mailing through their cell phones, on Facebook or through twitter to encourage friends to come out. I know how tough it can be when it comes to putting an event together. A place needs to be booked, friends need to be contacted, schedules need to be checked and there are questions that need to be responded to. Sometimes the most annoying thing is when you contact everyone and no one gets back to you and you end up scurrying to contact everyone directly to double check if they are coming out or not, or managed to forget about the invitation. Organizing events can be a real pain honestly, it's like doing a PhD-- you really have to like being a host to go through with the issues that comes with the territory. I for one, am happy when people get together.

I mentioned earlier that I have friends that are event coordinators/promoters by profession. The interesting thing I learned through Facebook was about event coordination-- the impact of mass mailing friends about an upcoming event. The general response to a mass invite of people on a friend list is abysmal, I wager that maybe 10% people confirm in advance if you're lucky. Other people are wishy-washy about committing to the event and list themselves as "maybe" and the people that say "no I can't attend" are usually fast to say so because they have prior commitments. Getting people to commit time to an event or anything is a challenging problem. And personally to me, a 10% confirmation percentage is really depressing. Unless you're really popular or have a very big pool of friends, avoid using large group invites.

I've noticed over the years that I am most successful at getting friends together to something was when I contacted small groups of them at a time. My friends generally divided into different circles so I end up contacting each circle directly or even people directly to invite them out. Response rates when you contact people directly are much higher and since they are actually responding to a direct contact from you, the probability of them saying that they'll come is also higher. After switching to this technique of inviting people out, I've felt much more comfortable running events as people are responding to me timely and I have a better idea of the general number of people coming out in advance so I can make things run smoothly.

Another important factor about hosting a successful event is the reason why you're running it. I am quite certain that there are people that we know are promoters of themselves. There have been friends that have sent out mails to invite friends to their musical performances and people in the real estate world that have ran events to promote themselves and spammed their friends to attend. My knee-jerk reaction to these invitations is one of disgust, in the sense that these promoters see me simply as a source of potential revenue and that I value myself more then that.

So I started thinking about what is it that makes my friends want to come out to my events and I've come to the conclusion that my friends want to attend my events because my intentions are different-- I am hosting the event not for myself, but as something for them to enjoy. I also like inviting new friends out to these gatherings so they have opportunities to meet new people and hopefully make new friends. In a sense, I have created a brand for the events that I run which makes my friends want to come out to the events that I host, in that these events are for them, more so than it is for me.

Oddly enough, thinking back at some of my earlier business successes that I've had directly correlated to a desire of me wanting to do things for other people. One of my fondest memories of an event was a donation drive salsa night between the Nursing and Engineering Department. I told the nurses to market the event as a good cause and I told the engineers that the nurses were coming. Suffice to say, that event was a resounding success as we made good money in a single night.

I would go on to start my own exam preparation series that turned out to a resounding hit that made the Engineering Physics student society the richest student society on campus. A story that I look back as one of my biggest successes while in university. I've probably recounted this story more than I should, but my dream was to run my own successful exam preparation business, more than making money. I ended up talking to my physics professors to get old final exams and told other students in my graduating class of my vision and I ended up recruiting a team of 8 people (including myself) to run the thing together.

Putting this event together was a passion for me, I wanted to make the lecture series as cheap as possible so I charged only $15/seminar session. I ended up making $4500 in profits in 2 weeks. I donated the money away and passed on the program to the student group, which resulted in a good recommendation letter from the head of the physics department which helped me get a scholarship to Japan and also would help me through an interview process that would also help me get my current job. What I gave away, came back to me in multiples and in ways that I could never imagine.

I have started reading books about human dynamics recently and about the rise of powerful people. The idea of having power has been defined in a variety of different ways. One definition is the ability to control people to get them to do what you want; another and more subtle version is the ability to have people depend on you. Neither of these 2 definitions sit well with me because they seem very Machiavellian, so I propose a third definition, the impact and the ability of what a person is able to give to others. The framing the concept of power like this make sense to me as it characterizes many of the most influential people I know and even some of the most successful and respected companies in technology that I can think of.

My train of thought stops here, but I feel that this is a strong idea.

Stand close to the edge to prepare for change

2011-01-28T16:28:00.004+09:00

There has to be one point in your life where you are willing to put your self on the line and be ready to accept failure as a path to success. To be honest, there are many more things I want now than I have the means to have. I want to be young and successful, (possibly even rich), I want to live in a nice fancy apartment for a while and have the drive and freedom to create a successful venture.

I have looked at my financial numbers for the last few years. They are pretty good, better than most for people in Japan at least, but it isn't sufficient for me. I can do better. Nearly all the successful people I know, the routes to their rise have never been through the traditional paths, there is something different about them. I used to be like that at one point when I was younger, but I fear, that the edge that I once had has become somewhat dull.

I can't say that I blame the system for that, the system is what it is and it really is my choice to go with it or not. To be honest, I've stopped caring about what it is about. It's not a Japanese thing, it's not a North American thing or a corporate thing, the system is simply what everyone else does, which to me is the most eloquent way for me to describe it.

I have to move out of the company dorms within the next few months and from there I can expect my savings rate to decrease, meaning that raising capital for me to do things will be harder in the future... which also reminds me that I still have to look through my data from last year. I've gone through most of it but haven't finished putting a sort of report together.

I also have the issue of time to deal with; where I want to be allocating my time. I do want to get back into investing, but don't have the time to look deeply as I would like to in the market. I also want to pick up on my programming skills to get into the foray into the world of online services, eventually going independent. I also want to take the level 1 Japanese Language Proficiency Test, the highest level Japanese test so I have the piece of paper certifying that I am proficient with my Japanese. At the same time, I do also want to balance my social life, work out and spend time reading too.

I've starting reading a Japanese book about life past 35 in conjunction about the economic conditions in Japan, some ideas of marriage, family, preparation for the future and the state of social security. It's a challenging read, but I am happy that I am actually able to grasp most of what they say. But I think all of the advice is the same though, mainly that you should be prepared to depend on your self as depending on social programs is dubious. One comment the author made in the book is that one's 30's is probably best defined by sacrifices. I turn 30 this year and I can already tell that I just don't have time to do everything and that my time now is incredibly important. More so than I was in my early-mid 20's.

Maybe I never tasted the sweet taste of success the right way when I was younger to push myself like the young entrepreneurs of some of the hot start up companies that are out there right now. But having that hunger now, is terribly important, the sooner that I make time now, the more dividends it pays off in the future.

Time is of the essence.

Dealing with older people and creativity

2011-01-22T02:32:00.003+09:00

I've been mulling a thought recently, the company I work is internally looking for people with fresh ideas and that can execute them. I have to be honest, though, there aren't a lot of people where I work that can do that. I don't feel the creative vibe or the fire to get something done. Upper management is preaching that we need more "creative people" that we need to make more "innovative products" and etc etc. Their ideals and the reality of the working environment speak is a world of difference.

I've been reading a lot about start up culture and the most important thing is to hire people that creates the right culture that sets the course of a company, the attitude of people and the things they can do and think about. At one point, when you start hiring and working with mediocre people, the cultural assets of a company is diluted causing trouble for the future. Things that can't be quantified with numbers are often hard to measure, but one does feel it.

The one thing I've realized, recently is how adverse older people are to new ideas, not that they are against them but being able to come up with new ideas or being able to accept them. Once they get used to doing things a certain way, it's all they know. The worst thing for young people is to be infected with that way of thinking. In hierarchical cultures (both social and working) I think this is sort of an infectious poison and the people that end up coming up with and executing revolutionizing ideas are people that aren't cast from the usual social mold.

I've had the some interesting opportunities to talk with HR people within the company to find that the average age of the company is starting to rise quickly, it's probably the result of the Japanese "baby boomers" getting older. In some departments, the average age is hitting close to the 40 year old mark. When you have so many old people at the top, I think it is hard for young people to find other young people with similar ideas to work with. The other thing is social politics and when you are around people with more authority than you, it becomes harder to do your own thing. Which brings me back to the fast paced change I've been seeing in the software industry recently, fueled by youthful people going their own way and creating startups. I am an avid reader of what the people at Y Combinator have been doing and the programming languages they've been using. Most of these guys that build web applications are young, using new languages and software platforms that large corporations have not even come to touching on a large scale. In the programming world, you can pretty much learn a lot about the culture of the company by the programming languages they use, thus looking at job postings is an interesting way of learning about your competitors.

Many of the biggest corporations now have been started up by young people at one point and the question is if the company culture ages well with passing time. The companies that are hot, are generally the relatively young ones, the ones like Facebook, Google, and Twitter as "hot," and we hardly think of companies like Microsoft, Oracle or IBM in the same way.

I've been working for older people for too long with their own ideas for too long and I think it's time for me to cut and move on. What they think isn't the end all and be all, but there are things that you can learn from them. It's just that you have to be good at seeing through some of their out dated thoughts. It's a tricky position to be in politically as a young person, but I've come to appreciate the need to become politically adept in dealing with people.

I've also been watching a lot of youtube recently and have been really amazed with some of the quality work that is being put out there by young people interested in music and video editing. It is a place where hardly older people create good content for. I miss that vibe and I realize more and more that it's time for me to get out of where I work.

Live Deliberately

2011-01-16T00:12:00.002+09:00

I made a post earlier about avoiding distractions to improve productivity. Things like youtube and social bookmarking sites can be a total time sink as it is easy to bounce from video to video or link to link. For the most part, not all articles are all that mentally nurturing, but there is a strange addiction to wanting to see or read more. Some call this a sort of "information addiction," others call this a resistance of the mind to change context and do other things, whatever the problem is called, I would be more happy to be rid of this issue.

Of the times that I have been most productive, I've noticed that I had concrete deadlines and my actions were exceptionally deliberate at getting the the tasks to get whatever I needed done. Without that level of focus, it is too easy for the mind to wander around and randomly do unproductive things.

I've started a new habit of shutting down the computer while not using it as it was too easy for me to find myself clicking on firefox and browsing around the internet aimlessly. By shutting down the computer, I am forcing myself to think and define exactly what I want to do before powering the system up. Without a concrete task defined, it is too easy to just start goofing off. So far, I have found that this is effective, though I hope my mind doesn't decide to find work arounds to this habit. I have noticed that I have been turning to my iPhone for some light browsing, but the magnitude of what I can look at is a little more limited compared to reading things on the computer (though the iPhone still does a pretty good job in it self).

By forcing my mind to think about what I want to do before doing something, it has made me realize that many things that I used to do was simply a result of me following random urges to use the computer or do things to kill time. A lot of the bad habits I have gone unnoticed and being more deliberate about my actions is helping me pick up on these habits.

I am also trying to keep track of what I've been spending time on to help me identify on what I am spending my time on to determine if there are more optimal ways of allocating my time. If I can do this with my annual expenditures (of which I still need to publish) then I should be able to track my time as well. As the saying goes, "time is money," thus I should treat is as a valuable resource.

The US's Trusted Identity system?

2011-01-11T22:49:00.004+09:00

The US's trusted identity program is picking up some steam recently. Arstechnica has a short article about it, where the gist of it is that you have certain entities that exists to verify your identity using digital authentication. The example they give is of a person using their cell phone as their ID device to get access to medical records, when it some how gets confirmed through some kind of network.

Obviously there are security issues if the cell phone you carry gets stolen, which probably means that you probably should password it... which ironically is sometimes dealt with when the people with access to your medical profile could simply ask about your medical history to verify who you are. What does this additional layer of "security" add? I have no clue.

One other point of contention is that of having a "single point failure" system, where in fact, you could be locked out of society if the network refuses to verify who you are.

A cursory look at web hosting prices

2011-01-03T15:56:00.008+09:00

I've been looking at getting my own domain + server space for some tinkering around as I am interested in doing some web based programming work. There are lots of interesting data mining applications out there and one that just caught my attention was Spokeo, which is an online directory of people with information scraped from all sorts of locations. Consider it like a public version of a CIA database of people, which has personal information including you address, phone number and home value. Obviously, there are some privacy concerns by having this kind of information available, but none the less, an interesting application of data mining.

I am quite interested in learning how to create web services, at least for myself and perhaps expand that to other people if there is time for that in the future. One of my favorite cloud services I use is Dropbox, where I have several computer synced to my dropbox account and have file synced between all of them. Integration is seemless and all operations happen as if you were normally manipulating files through your usual file manager.

I am still on their free account (2 GB limit) and they charge you $9.99/month for 10 GB of storage and then $19.99 for 50 GB/month of space. I think that it is a little on the steep side, but they are running on Amazon's servers which is probably driving costs up. For more space and the ability to share files with myself, I decided that it might be a good idea to look at your standard vanilla web hosts to see what were the prices were like. What I found was that using a web hosting solution (those that were based in North America) were vastly cheaper compared to Dropbox and Japanese web hosts.

I've tabulated the results of what I looked at (from sites that had information that was easily accessible):

Table of web hosts

Interestingly, dropbox when compared with other web hosting services is expensive. Japanese web hosts in general, tend to use tiered pricing for their services and don't provide any unlimited plans which seem to have become popular with North American hosting providers (well, the 2 that I listed, there were more but I didn't bother to list them). With this result, I will want to go with a non-Japanese host as I will get the most bang for my buck. Interestingly, these hosts look ideal as an offsite backup of data, in the case of catastrophic damage, which is a good thing if you want to keep data safe.

Pictures of people between 0-100

2011-01-03T01:44:00.003+09:00

Found this site just recently, looks like a photo project of pictures of people between 0-100. An interesting look at the progression of aging in people. It is a good reminder of one's own mortality and the value of time.

Search Engine Optimization and Social Networks

2010-12-29T23:39:00.007+09:00

Search engine optimization (SEO) has been a trending buzz word on the sites I've been visiting recently. I generally believe that there is a big market for business intelligence related internet work as there are so many people online today searching for so many different things. The key point is to make sure that your website is the most prominent to drive potential customers to your site and convert them into paying customers. In simple terms:

customers = site visitors * conversion rate

and what people in the internet business industry want to do is increase the number of visitors and conversion rate to make a business profitable. The problem itself is very fascinating as it requires a lot of data mining and profiling to determine where your customers are, drive them to your site and convert them into paying customers.

Consider that Amazon it self had $22 billion dollars in revenue for the year ending in 2009, that the whole total of online sales for 2009 is much bigger then that. Meaning that the business for improving online sales is big money.

For those of you have been on the internet since 2000, some of you might have noticed invisible text hidden away at the bottom of a site or in the html of others. The idea of optimizing one's site to rank higher in the index of internet search engines is an old idea. Google had an interesting take of the practice by both rating the number of external links leading into the site in addition to the content of the site it self when determining the ranking of a site thus improving the quality of sites they returned from their searches. I believe with the rise of social networks and social based information exchange sites, that the nature of search will change in the coming years.

I started thinking of first seed of this idea in a former post in June entitled Friends as Filters, where friends based on their interests will filter and pass along useful information within a circle of friends. With the rise of ease of sharing information through social networks, we find that social networks are having a significant impact on how information is passed along. We now have a new class of "viral" videos or webpages that become massively popular by dissemination through social networks instead of being ranked at the top of a search engine.

I find that Facebook is an interesting avenue for sharing information with friends, whether it be through the sharing of things they do or the interesting links they post. I do admit that I need to organize my contact list and start filtering people for interesting content for to improve the quality of information passed to me. But the basic concept is there, which is similar to social linking sites such as Reddit, (the now waining) Digg and more. Unfortunately as the number of users grow at these social sites, degradation in the quality of posting does occur, resulting in meme posts that are popularly voted up as a result to an appeal to the greatest unified demographic; in these cases, it need not be the majority controlling the "main page," assuming the demographics of a site is sufficiently fragmented enough to allow a minority group to have the most control over the popularity of the links posted. It would be akin to allowing a minority governmental party wielding majority like power inside a parliamentary house.

With this problem at hand, online communities have started reorganizing to support sub-communities where relevant information to sub-groups are passed along or posted only in that sub-group. What we are starting to find now is that things are a little different compared to the times of say Usnet when posted information was publicly available compared to the wall-gardened sites such as Facebook. What many spammers used to do was mine Usnet posts for e-mail address and spam everyone with advertisements. With the advent of more modern social networking sites, stronger linking of information is now possible however that information is not publicly available unless you are an app developer or the administrator of these sites themselves. What people are finding is the information in these sites is valuable when mined and summarized correctly. I think this will lead to a change from SEO related work to Social Network Optimization work in the future.

I think this field is still in it's infancy and to get information worth mining you are either going to get access to data in established social networks or create a killer application to create a network of your own, but I think there is good money to be made jumping into the social networking site for business intelligence applications.

Edit: Some typos and had 22 trillion instead of billion. I have a terrible habit with skimming material too quickly it seems.

Mastercard and Visa shouldn't be controlling who you can do business with

2010-12-24T15:53:00.002+09:00

There exists file sharing sites that share that contain copyrighted content, but the idea that financial transaction institutions such as Mastercard and Visa working with the MPAA and the RIAA to preemptively block financial transactions with organizations they don't like sets a chilling standard with regards to the freedom of to whom you are allowed to do business with. We are seeing the same thing happen with Mastercard and Visa blocking financial transactions to Wikileaks because of the inconvenient nature of the material they are publishing online. Supporters from around the world are willing and wanting to donate to Julian Assange and his cause through both financial and technical support.

There is nothing more democratic when it comes to people voting with their money, which is exactly something that the US claims to be a staunch supporter and defender of. Unfortunately this simply isn't a battle of ideals, free markets or rights. It's simply organizations trying to change the game to keep other people out of their political/economic turf.

Facebook and more intrusive ads

2010-12-24T12:33:00.003+09:00

I generally use Facebook to keep track and stay in touch with friends. The biggest motivations to get into Facebook was to get reconnected with people that I had lost touch with and be able to keep up with them. I believe that the social graphs of people in today's society is simply much larger compared to what it was before as a result of the internet.

I've recently upgraded to their new profile and was aghast with the prominent display of ads on my profile page. To me, it's a terrible eye sore and I am not very happy about having a personal space being used as an advertisement space. That is a digital line that I would much not like to have crossed. I have since installed and enabled the adblock add into firefox and the ads have since disappeared. For now, using ad-block is a viable solution, but there ought to be better ways of maintaining a social graph, content and access instead of using a closed platform. One of those alternatives is Disapora, which is still under development. It will be interesting to see what happens from there.

On Digital Money Usage in Japan

2010-12-24T10:25:00.006+09:00

Of of the interesting thing about about Japan is the use of digital money. I used to think that digital money was close to inconceivable because I much prefer the feel and sight of having cash in hand when making payments. The physical and visual queue of using money is that has greatly helped me in keeping my spending in check. After moving to Tokyo, I've become accustomed to the prevalence of using digital money.

The cafeteria at my office operates on a digital money system. We have several thousand employees at the research center so the lunch lines can get pretty long if you get to the cafeteria right at noon. When I was a university student, it would take a while to get your food and then take another long while as everyone fumbles with coins and bills to pay for the food. Credit cards are one solution to that problem, but I believe that digital cash cards are superior compared to credit cards.

One electronic cash card system being used here is called Edy, which is a wireless rechargeable cash card that are also integrated into some cell phones in Japan. These cash cards can be charged using recharge kiosks where you can place your card or cell phone on to and deposit money into the card. Another interesting feature for cell phones is that you can log into your bank account from the cell phone and have money wired into the chip in phone. One point of risk for the system however is hacking of the cards to digitally manipulate the amount of money stored. I am not entirely sure about the security features used in the card to prevent people from tampering with the device, however.

The Edy cards are currently accepted at nearly all convenience stores and some retail chains, so I would not say that market penetration of the cards for consumer usage is not that deep. However for mass transit usage (trains, buses and etc.) pretty everyone uses these cards as you do not need to look up train fairs and pay for it every time you go through the gates. Throughput of people in Japan through the train gates in Tokyo is quite amazing as a result of this.

The one nice thing about these digital money cards, is that they are not necessarily tied to the identity of the user, for example in the case of a credit card. The loss of a credit card usually incurs the headache of contacting the credit card company, deactivating the card and waiting for a new card to be made and sent to you. Without having this identity connection, a card loss is simply equivalent of physical loss of the money, meaning that you could just buy a new card (which is about $10 ish), reload it and start using it immediately.

One of those things I've been getting annoyed about recently is the size of my wallet with the number of cards I carry. Women, in particular, I notice have huge wallets to carry all the point and membership cards they use for shopping. As of recent, I've been thinking about how nice it would be to be able to carry a much less boated wallet with me.

The financial incentives for managing a digital money system is obviously there as money charged into these cards can be placed in a holdings account where the institution can make 1~2% interest in very low risk investments. When it comes to creating digital consumer ID cards, I am not entirely sure that there is sufficient incentive yet to make that jump yet, though I believe that work is trivial.

I would expect that ID information will eventually all become digital and embedded within an ubiquitous device like a cell phone. The interesting thing to ask is what are the risks of digitizing all ID and monetary data? Furthermore, I would wonder what kind of system would manage it? For it to be socially acceptable, I would be willing to venture a guess that the management system would have to be a decentralized one. Maybe we will see that transition in the next 5~10 years after the first "killer application" that drives this transition is created.

Intangibles

2010-12-19T23:54:00.005+09:00

I am oft known as a frugal person, in that I don't generally like spending money if I have to. Optimizing/reducing cost is one of those things that I've been good at doing; however there are things that do go missing from the equations, the intangibles.

Tell me that you need a hammer and I'll tell you where you can get a sufficiently cheap and good one. Tell me that you need an affordable computer and I'll set you up with the specs for a system and where to get the parts for the right price.

It has been mentioned to me recently (and repetitively) that apparently, I need to "improve my fashion." Like in this XKCD comic, "my normal approach is useless here." I was dragged out this weekend to shopping in search of new threads. Why is it that one piece of paper scribbled by Vincent Van Gogh is far more valuable than something scribbled upon by a random person? Art, unfortunately is one of those strange realms that numerical analysis is rather useless upon.

The one thing I know is that I spent a lot for shoes, pants, shirts and a jacket. The other interesting question is, what interesting intangible benefits of "looking sharply presentable" have? Apparently, I have feeling that the answer would simply be best described by as "a lot."

Friendship map of the world

2010-12-17T00:42:00.002+09:00

I've always been a fan of finding interesting ways of mashing up data from the internet. One project I did was downloading performance ratings of CPUs and combining them with pricing data to calculate price/performance ratios before selecting a part.

Recently, an intern at facebook stripped out friendship data from their servers and made a map of connections between cities, by mapping what city they were from into geological coordinates and drawing lines between the points. After some artistic manipulations of the lines, his work resulted in this.

A very cool map indeed. Some interesting notable dark spots are South America, Russia, Mongolia and China, parts of Japan and along the Canadian and US border. If you're interested in the details of this map, a link to the original facebook article is available here.

What would you pay for a set of games?

2010-12-15T09:39:00.003+09:00

The band Radiohead once did this and this is the second time that an indie game developer is doing this. It's the humble indie game bundle, where you get to choose how much you would pay for a set of 5 games and they are posting live information on the how much people are paying and by what platform. The trend seems to be the average "donation" is Linux > Mac > Windows. Where Linux users pay 2.5x more than what people pay on the Windows platform. Ironic that people that use a free OS would pay more for software.

An interesting article on if you should get a PHD

2010-12-14T01:23:00.001+09:00

The link to the article is here. It's a good read and I haven't finished reading all of it yet (not that it's long).

Web based programming is the next big thing

2010-12-09T22:33:00.002+09:00

Programs used to work in isolation on a single PC somewhere before the internet became prolific. Computer science classes at university used to be all about programming in C or C++ or something like scheme. But now with the advent of the internet, most of the programming skills we learn at universities are obsolete.

The thing I remember the most about my time in programming classes was dealing with how to store data, how to process it and then how to display it to the user. I still think that these things are fundamental to the core of programming, but I think there is a big demand gap when it comes to finding skilled people that are able to get data to feed their programs, process large data sets and come out with meaningful results. This is the stuff that business intelligence is made out of.

Over the last few years, there has been a big transformation in the world of programming, databases and languages have now been developed with the paradigm that given certain data, there are specific functions calculations that you want to run on all the data and summarize them into a single result. This thinking framework is otherwise known as MapReduce. The great benefit of this methodology is that is is also functional programming friendly, meaning that you can define compound functions that you want to run on a specific data set instead of dealing with loops, counters and how you are going to save the data. The basic idea is that you make a list of data, process it and out comes a list of data. Then after that, you reduce that list into a single value or something.

Modern languages languages like Python, Erlang, Lisp and more are designed to handle programming like this (in additional to dealing with the usual procedural type programming and object oriented programming). Older languages like C or C++ can do the same thing, but you have to build the functions and the scaffolding to do this on your own or call libraries to do this kind of programming which make C based programs longer and more error prone to write.

One of the reasons why I was surprised in an earlier post to find some large corporations doing data processing still wanting C++ programmers. The other added benefit of using more recent languages is that they also have libraries to access web based data built in, making it easier to feed applications with more data to process. The of the most famous startup companies are the direct result of this effect. You won't learn most of this stuff from your ordinary university class, the only way to get into this stuff is probably on your own.

Learning how to work with open software projects

2010-12-08T14:51:00.005+09:00

I've seen some interview questions and job postings that mentioned that working on open software projects is a nice to have skill over the last few years. I never understood how much of an impact that working on an open software project might have on a person's ability to code until just recently.

I have been following the programming scene in sort of a cursory manner, mainly through reading at hacker news, which is a good site for programming and web development kind of information. What has been coming up repeatedly is the development of a program project hosting site called github, where the collaborative projects are hosted. The ability to code well parallels with learning to write well; in that reading good code helps one write good code.

I have cursory looked at sites like source forge in the past to download code to look at, but one of my biggest problems is trying to figure out how large projects are structured and what code is where, doing what. I used to do most of my coding through a simple text editor before because configuring and trying to figure out these modern fanged integrated development environments (IDEs) would get on my nerves when I wanted to learn and compile something simple. I've used to code through a simple text editor for the last 2.5 years until realizing that the debugging facilities and managing large chunks of code became time consuming and brutal.

I have since switched over to the freely available Eclipse IDE with an integrated debugger and the ability to read locally created libraries and allows for auto code completion. It is a godsend compared to what I was doing before. Commercial packages like Microsoft's Visual Studio probably has functionality like these built in, but learning coding and figuring out IDEs on your own at the same time is a little much. The great thing about Eclipse and some of the new open source IDEs is that they are starting to add functionality to connect and download projects from code repositories from places like github. I have just finished configuring IDE to do just that and downloaded the libraries to a data-mining spider called scrapy. There are other libraries available that does the same thing but I have the intentions of learning how the program was coded, the programming styles used and possibly using the project itself for my own data-mining projects. After looking at some of the code, I can understand how some people would look highly upon people that participate with open source projects as they are exposed to working in an environment with many people, many lines of code and looking at a lot of good and bad code. I will be spending time at looking at more programming projects to further hone my own skills. The idea that companies look at people with open software experience brings up the interesting concept of profiling people and their skills through secondary activities and traits. It's an old idea however, but it would be interesting to see the correlation of coding skills to open source project participation or say a person's affinity to knitting and a (negative correlation) to say, sky diving. Unfortunately, statistics when applied like this is like opening a can of worms.

Can't make heads or tails of silver

2010-12-06T22:58:00.006+09:00

Apparently there seems to be a campaign on the way to crash J.P. Morgan via the purchase of silver as they supposedly hold a short position of 3.3 billion ounces of silver and are currently under investigation of silver price manipulation. In the last few months, silver has been trending upwards for the last 3 months with a 47% gain. Gold hasn't seen that much of a price increase.

Silver ETF price for the last 3 month

When it comes to financial information on the internet, there is a lot of garbage out there, especially for a hot topic like gold and silver. The leading argument to buy silver comes from a guy by the name of Max Keiser, in general, I don't trust financial pundits and whatever they start talking about, I generally tend to ignore but make a note to look into on my own.

The burning question that needs to be answered is, "is silver relatively cheap or expensive right now?" I've added to snap shots for the last 10 years on gold and silver prices from kitko.com.

Price of Gold

Price of Silver

Metal prices have gone up significantly over the last 10 years, but I'll skip the commentary on that:

Gold in 2000: $300 (approx), Silver in 2000: $5 (approx) Ratio: 60:1
Gold in 2010: $1414, Silver in 2010: 29.58 (Dec 6 prices) Ratio: 47.8 :1

Historically, it seems that the price of silver has been much lower compared to gold in recent times and I find it hard to believe that this ratio should suddenly change. I've also been looking at the relative production rates and total produced numbers of silver and gold and get wildly different ratios compared to the prices (of course the numbers themselves might not be reliable too).

From gold-eagle.com: I calculate that by 323,476 Tons (by 1999) have been extracted. I guestimate that about 450,000 tons have been mined by 2010.
From wikipedia: They mention that 165,000 Tons have been mined by 2009.

Meaning that there is a 1 : 2.77 availability ratio of gold to silver. I've also looked at the production ratios and the numbers work out to be 1 : 4.8 for gold and silver. I think that stockpile availability might be the key factor and the difference between supply and demand might be causing the price differentials between gold and silver, but I still can't figure out the reason why gold is that much more expensive than silver, presently and historically, unless the numbers I've been looking at are bad (which is a possibility). Personally, for the bigger price differential between gold and silver to be supported, there ought to be a greater differential in availability between the 2 metals.

The other forward looking argument to be made is, suppose that people are buying silver to crash J.P. Morgan and that it does happen. And then what? I would say that eventually everyone would have to sell their stocks of silver to get their money back out of it thus dropping down the price of silver. The question is, when do you start selling? If you can't figure that out then you shouldn't be buying in the first place.

It'll be interesting to see what happens, but I have a feeling that there is a bubble in silver happening right now.

Addendum:

I'll add supply data from kitco.com here since I think the data from here seems a little more reliable and up to date.

Gold supply (2009): 117 Million Ounces, Silver Supply: 810 million Ounces Ratio: 6.9
The ratio is still fairly small compared to price ratio between gold and silver. What is the pricing mechanism for gold and silver? I still don't know.

Reducing distractions and the 10,000 hour rule

2010-12-04T19:11:00.002+09:00

I am in the middle of a transition period in life; I've got another 6 months till I turn 30 and the big question of "where do I go from here" that I need to figure out. One of my colleagues turned 40 a few months ago and the only thing he was muttering the whole time was "I just can't believe I'm an old man." The interesting thing about my life is that I get along well with people that are older than I am; it's always been that way when I was young and the best part of having friends that are older than you is that you get a glimpse of the future so I can prepare for it now.

As of recent, I've been thinking much about my future; or at least, a strategy for one. Getting prepared for the future isn't like getting prepared for a trip or a disaster, where you just spend a few days going out and buying what you need. Preparing with skills, knowledge and experience is a different game, where you pay for it in the currency of time.

The sub-prime crisis and the following recession/depression fiasco has wreaked indiscriminate financial catastrophe to all walks of life; the young, the middle aged and the old. What was interesting was how each group was affected by the economic downturn. For example, the young cannot find jobs, the middle aged are have debts to deal with, and the old had their savings hit. What this economic downturn has taught me is that you need to keep a good backup plan to protect yourself from being stuck between a rock and a hard place, especially if you've become old (or transitioning to being old) and had your savings wiped out (naturally, you would have been investing in "secure" investments).

So where does this all fit in with me? At this point in life, I've come to realize that with growing age, the number of options I will have available will start to shrink in then next decade-- the number of new fields I will be able to jump into and get really good at will dramatically shrink. So it is now that I should start looking into explicit skills that I can enjoy and be really good at to make a good living. It has been oft quoted from by Malcom Gladwell's book "Outliers," that the number of hours it takes for a person to become an expert at something is 10,000 hours.

Doing the math, to become a master it will take:

27.4 years at 1 hour/day of practice
13.7 years at 2 hours/day of practice
or 6.8 years at 4 hours/day of practice

This calculation also puts a lower bound on how long it takes to be really good at something and also the number of skilled expertise that a person can have over the course of a life time. The other thing is the earlier you have more skills, the more they will pay off in the long run. Meaning, if you start training hard by the time you are 50 years old, you might finally become an expert by the time you are 60, however that doesn't leave a lot of time to put those skills to use.

The other interesting thing about skill sets is that they also come outdated as time goes on. Someone that might have been a really good switchboard operator/manager in the early 50's would find their skills useless in today's modern world. Or perhaps a little differently, some computer gamers might find their skills in Starcraft 1 be outdated by the release of Starcraft 2. You might have some transferable skills, but you won't be an expert off the bat. This brings up the interesting point that transferable skill erosion occurs with every new generation of technology, the further along we go in generation, the number of skills we can transfer will negatively compound. Though it might be a little early for me to start thinking about this, it is something that I should keep on mind.

So here is the thing, suppose that I do practice something at 4 hours/day, it will still probably take me 7 years to be really good at it. What it also means is that if you want to be good at something in a reasonably good time, you really don't have a lot of time to waste.

After coming to this realization. I have decided that I am going to institute a system to reduce the number of unproductive distractions so I can spend more time learning. Mainly less checking of news sites, less youtube, less computer games and etc. This doesn't mean that I will be aiming to live a boring life, far from it actually. What it means is that I should more effectively use my time by doing things that brings me memorable value; which could include picking up skills or spending time traveling to somewhere exotic, learning more about wine or whatnot.

I think there was an Indian saying that went like this:

When you are young, you make your habits.
When you are old, your habits make you.

Time to make some productive habits while I still can.

People still use C++!?

2010-12-01T22:38:00.003+09:00

I find it amusing that people still use C++. I am a big python programmer, which was a transition for me from C++. The python programming language is just far more extensible, easy to program and dynamic variable typing is so easy to deal with that I just can't fathom that many people still program in C++. Many of the best startups on the internet don't touch C++ at all. Do I brush up on old skills or keep moving with the new stuff, a bit of a dilemma.

The problem with disclosing income

2010-12-01T21:15:00.002+09:00

I've been doing some reading recently about the issue about disclosing one's income during an interview. The problem is that it give the employer an incentive to underpay you knowing your payment threshold. I've been on both sides of fence recently, knowing what pay looks like in the finance world thanks to a few contacts I was able to target for price at par with other people in the industry. Unfortunately things didn't work out as they had someone without a job that was willing to take the job, but that was OK. I would much rather not take a job if I knew I was going to be underpaid -- because one of the reasons for moving to a new position is to leverage your skills for a higher salary.

Since that deal had just recently fell through, I've just been introduced to some recruitment agencies and in the process of sending out applications for new jobs. I decided this time, that I would see what happens if I put something close to my real salary. I was soon contacted by a representative that would pass my resume to a few other positions. The guy quickly asked about my salary inspirations and I gave him my ball park number, he immediately balked at the idea of me looking for a 40%~50% salary increase, citing that I had a much lower salary listed. The moral of the story here is to not disclose your salary or put down a higher number.

I had qualms at first about inflating my salary before, but after seeing this reaction, it is an unfortunate necessity of the negotiation process.

Time to pick up some database skills

2010-11-30T23:24:00.002+09:00

I've been dabbling on and off about some data mining work for sometime now. The one thing I am kind of surprised about myself is that I haven't picked up any decent database skills. Some IT related work came by my way, the guys are looking for someone with C++ skills and and database related stuff.

I haven't touched a low level language like that in ages, the last time I did something with C++ was try to write my own poker bot. I actually had a clever system setup as I made a hack into the RAM to find out where the card data was stored and figured how the card data was stored.

I haven't done any sort of enterprise kind of programming before; most of the work I do with programming is scripting and numerical analysis kind of work; I run pretty much everything through the command line to get at data. But IT work for banks and getting into analysis kind of work seems kind of interesting. The problem is that the firms are Japanese and I sort of have doubts on the amount of money they would be paying. I've worked in one Japanese corporation already and I believe that I should be fetching a higher salary, oddly not being paid enough seems to be strong negative incentive to me-- there were times I worked harder as a student than I work in the company.

There are a lot of skills I need/want to pick up right now. I just have to hunker down and pursue a few things with great intensity. I intend to make programming one of my fortes.

JLPT Level 1

2010-11-29T00:45:00.003+09:00

The JLPT is the Japanese Language Proficiency Test, basically a bench mark for the language ability of foreign people to comprehend the Japanese language. I don't have any qualifications as I have just picked up everything I needed along the way, but I have since come to the realization that it may come in handy for future employment opportunities. I am debating on picking up a few other qualifications on my own depending on the availability of my time.

I have decided to prepare myself for a move for a new career to better leverage my skills and improve income compared to the work I am doing now; even if it involves me picking up some new skills. The job market in Japan is interesting for people with international skills and I intend on breaking into that market career wise. There are probably other interesting opportunities in other Asian countries like Singapore, Taiwan and Hong Kong for international talent. One of those requirements to break into this field is a necessity for good multi-lingual skills in addition to either business or technical skills.

The need for international recognition as a baseline for certain skills may play an important role here. I will be looking into at least getting my JLPT level 1 certificate within the next year. I've lived here for 5 years now and I should at least get that certification.

Career planning is tricky

2010-11-20T17:17:00.001+09:00

I am now in the process of looking for a new job. People might call me crazy to consider looking for new work with the economic down turn and all, but some how I am confident that I can find something better, that also pays more than what I am getting right now.

The job that I do isn't all that bad, and the working environment compared to other Japanese companies is pretty good, but I believe that I can do better and that I can pick up the skills needed to advance faster.

I finished reading a nook earlier this month called "kitchen confidential" written by Anthony Bourdain, a first hand biographical look into the nitty gritty of the culinary world. One interesting note that he made was that he could have been a better chef if he hadn't spent so much time chasing money, where as another talented chef went from Italian to pastries and through other fields of cooking, starting from close to nothing to learn everything he could. The fusion of knowledge would eventually make him a renowned cook.

Right know, I can admit that I am chasing the money, wanting to pick up the skills to advance into better paid positions. Raising one's income isn't a necessarily a bad thing, but what is it that one unwittingly gives up in this conquest?

Modeled numbers are easy to predict, this is the realm of accountants and mathematicians, but there is also a hidden lie beneath it all, we assume that the model doesn't change.

Accountants cannot account for the power of a technological or cultural vision becoming a reality. Like poker, predicting or creating the future isn't just about a numbers game, though knowing how to crunch the number does help. Are there intangible things that I am neglecting?

And so, I will probably come to a crossroad, there will be jobs that I can pick up skills for that may take me away from my world of research and development that I believe that I want to remain in or do I go into the world of finance or other uncharted territory that may pay better.

There will be more job offers coming my way in the coming months and more skills I will need to pick up to position my self for better opportunities. The question is, to which direction do I walk and why? Assuming that this is the right question to ask in the right place, sometimes it is just nice to just walk in the first place.

If there is a time to do it, it is now; I hear from other people it does get a little harder to find new work the older you get.

The Search for a New Apartment Begins

2010-11-16T13:05:00.001+09:00

As mentioned in the previous post, I've been living in the company
dorm for the past 3 years and it's time for me to go. In Japan the
moving season is in March-April because the university year starts and
new hires start work at this period of time.

I've been looking on and off for places to for the past month through
the internet and after living in the dorms for a while, I am ready for
a nicer place to live. Somewhere with at least 2 rooms and a kitchen,
and relatively closer to the city in a nice building. It is going to
be much more expensive to the dorms that I've been living in, but I
will be able to invite friends over and host events, which should cut
down on my entertainment costs significantly.

The added benefit is that I'll be able to dedicate a room to a
study/work out area, meaning that I'll be able to do more activities
at home. I would like to see if the partition between having a study
versus having a bedroom + study will have an impact on my working
habits. Either way, having a new environment will be mentally
stimulating and I am quite certain that the change in environment will
have a positive impact.

At the same time, having increased associated living costs will force
me to keep myself to sharpen my skills, advance and look for new
opportunities. I believe that it's easy to stagnate if one gets too
comfortable-- because if things are pretty secure already, why try
harder? This is a habit that I am going to work on breaking, but I
believe that it requires some external "stress" to be applied to
myself.

Fortunately, I've still been tracking my personal expenses so I have a
general idea of what I can afford. In Japan, compared to North
America, there is a wider selection of places and prices of places to
live in. Where as in Vancouver, the cheapest place that you will be
lucky to find will be in the $800 range, going upwards. Living about 1
hour out of the city by train, you can probably get away with
something in the $500-$600 range if you wanted to save money. I'll be
aiming for something nicer and closer to the city. It'll be more
expensive, but I will be aiming to shoot for a career upgrade within
1.5 years for a respectable increase in salary. I have a general idea
of what other companies might be looking for in an international
candidate now so I will be pushing to get certified in the level 1
Japanese Language Proficiency Test and one other certification.

To accomplish these goals, I will be looking to reduce the number of
hours to focus on personal development. The idea is to invest in
myself and reap compounding rewards the more skilled I get. I think
this is a far better solution compared to working longer hours-- as I
loathe to think of myself as a monkey. Other commitments I'll need to
make is to cut down on the number of hours that I spend unproductively
and I am in the process of setting up a timer system where I focus on
a task for a specific amount of time before giving myself a break to
do whatever. I'll also be looking to find a way to tabulate what I
learned or accomplished during these times to determine if I was
working effectively.

Time to move out of the company dorm

2010-11-14T23:44:00.002+09:00

Living in a dorm of international students was great, living in a dorm with co-workers is pretty damn boring. I exist in a different world, even when in Japan with international expats, Japanese people that have lived abroad and came back, I even really like the country folk here as well. The thing is that I just can't connect with the ordinary traditional salary-people here. The just don't jive with me, not that I don't like them; I am just bored around them. I generally do my best to fit in and I am usually pretty good at fitting in, but there are times when you simply just need to put your foot down and get out.

Yeah the rent for the company dorm is really cheap, they charge about $250/month in fees with utilities included. Dirt cheap and thanks to that price, I've saved up a good chunk of money that would have otherwise gone to the landlord. But at the same time, you also pay a price, that you can't invite outside friends and the other thing is that I live away from Tokyo, from where all the cool stuff is happening. I've just started to meet many interesting people and decided that I need to get closer to Tokyo and make more free time to do interesting work and meet interesting people-- the surrounding environment plays a very important influencing factor. You can tell that things are different when you listen to people with interesting ideas and trying to startup a new business, whether it is in silicon valley or in Singapore; you think depending on your environment.

So I've decided that I need to get out and get myself into a more happening place. I'll probably have to deal with a commute to get to work, but it is one of those things I'll probably have to live with. In ideal circumstances, I'll find a new venue of work where it is close to where I live, somewhere happening. I've been living too "defensively" for the last few years by making decisions based on how much it I can "financially save." I want to from "saving" to "making".

A weekend of relaxation

2010-11-07T22:08:00.008+09:00

After a wild Halloween and more weekend events (one of which included me dancing in a museum and the curators getting mad at me), I've decided to spend this weekend at home for a change. Most of my friends would have a hard time imagining me spending time at home instead of going out, but yes there are times when it is really nice to just not go out; in this case I had a lot of errands that needed catching up on and I figured that it would be now or never to get them done.

Stuff like cleaning the room a little, changing the battery on my scooter, hitting the hardware store to look for a can of rust remover and finding spray can of grease (interestingly) for my bicycle and such, I remind my self that there are probably a lot of small little maintenance things that I ought to be doing but just haven't really been finding the time to do recently. So sometimes, it is necessary to put time aside to get things done and time to just chill out as well too.

The last week has been marked by a string of food indulgence, as not many people might know, I live without a kitchen so I've created my own little kitchen with a table and a hot plate that lets me fry meats and make all sorts of other dishes. Having good food waiting for me at home is a great incentive to come home early; I often have a hard time staying late at work when given the choice of eating at the company cafeteria (which isn't bad, by the way) versus something home cooked.

It's been a while since I've done any real picture posting-- I still remember that I have more photos of Hakodate that I should have posted and other things that I should have done but haven't gotten around to doing yet. Slacking isn't a really good habit, but there are only so many productive hours in a day that a person has (or so I believe).

So first off, since February of this year, I bought a new cross bike and have been riding it from time to time to the beach on the weekends. Round trip of 30 km makes for great riding. The entire distance can be covered in about 1.5 hours but I like to take a breather at the beach, where I like to buy a can of orange juice and watch the beach goers play volleyball and surf over there. With autumn dawning upon us, the sun sets much earlier and the sky much clearer compared to the hazy humid sky that one would typically see.

The setting sun

The beach center

The other nice thing about autumn in Japan is that it is still relatively warm-- I can still venture out in a t-shirt on a sunny day, which makes doing exercising in warm but cool weather perfect.

I've also been doing a lot of youtube watching recently. My favorite videos to watch on youtube are of the cooking variety. I've already posted recently about one of my favorite youtube chef at foodwishes but in addition I also really enjoy watching Anthony Bourdain's series called No Reservations; a show that I highly recommend for anyone that enjoys both travel and good food. One of his shows aired way back was a "Techniques Special" from which I used as a base to replicate a beef stew.

The recipe is rather simple:

Take a hunk of shoulder cut beef, chop it up, season with salt and pepper.

Fry until brown

Add onions and cook until semi-translucent

Add some red wine.

Simmer for about 1.5 hours

Add carrots and cook until tender

Serve with a side of rice, bread or whatever you have lying around.

I did all this while cleaning the room and gave up on cleaning after the stew was ready. An incredibly easy but tasty dish. Would probably be nice to add some potatoes as well. I've got enough food in the fridge to last several days. Eating leftovers for 1 might get a little repetitious sometimes, but with enough variety it beats eating expensive take out any day.

Stay on your toes

2010-10-28T22:36:00.002+09:00

I'll be writing some more later on but for now, there will be a short lull in writing. Work is busy at the moment and a lot of things are going on. Will be back later!

forget the limiting, focus on the enabling

2010-10-23T14:56:00.002+09:00

I've posted plenty about my gripes on working in a corporation and doing research in an institution. I've decided that it's time to stop worrying about all the annoying things that bug me about these establishments and free myself of the "me versus them mindset" and become more fluid in dealing with roadblocks.

Perhaps I was naive when I was younger, when I saw a system or an organization that I thought was ineffective, that it would be easy to encourage change. After trying with little real results, I have come to the realization that change is hard and not worth the time. It's more effective to create a pocket in an organization where you can work effectively and if things are not optimal, then it's time to go. When finally getting into the real world, the scale that you deal with things changes, dramatically.

Before, life would revolve around the academic environment, going to elementary school or high school with 1,200 students and only interacting with mostly people in the same grade (say several hundred people) means that you have a really small point of reference. Going to a university of 40,000 people is a little different. When you finally get out of university and into the real world of millions of people the scale of what you deal with changes. When you're dealing with just a few hundred people and you're stuck with them for many years, that was the environment that you're stuck in and it's rather hard to change. You had to live with the things that you liked and didn't like. After getting out of school, the world is a big place and you don't have to put the effort into changing an environment to make it more palatable. If it's something that you don't like then you can easily move on.

And that's the thing when it comes to attracting smart people, keeping them and creating the right environment to getting good work done. I don't have time to fight the system, to convince people that doing things a different way is better and then doing it. Life is too short for that and I just want to find the right people and environment to be successful, because at the end of it, it takes way less effort to fight within a system and do something than to disengage and do it yourself. The strategy is to find the right way to be successful on your own. If the only taught that at school...

Performance review

2010-10-23T14:13:00.003+09:00

I just had my performance review yesterday. In 6 months, I did a decent amount of work and at the same time, I also reduced the number of over time hours I've been putting in at work. I was pulling about 55 hours of overtime/month last year and this year, I've brought it down to about 30 hours/month. Having an extra 24 hours/month to yourself does make a difference in quality of life. Don't underestimate the value of time. I'm going off on a tangent however...

After my performance review, I asked my boss what does it take to get a promotion. How things work in a corporation is that you get raises based on the number of years you've been around, on top of that, after a certain number of years you can be recommended for a grade promotion. His response was that I need to take more of a leadership role and that I could be considered to get a promotion after my 4th year. Hearing something like that was shocking and disappointing. Why does it have to take at least 4 years to get considered for a promotion? I believe that promotions should be performance based, depending on the capacity of the person to handle responsibility and their ability to do good work. To be stuck in a position for 4 years and then take say another 5~6 years for another promotion is pretty much career advancement at a snail's pace.

I don't want to be half-way dead by the time I get into middle-upper management.

There has to be a better way at being successful instead of lining up in the seniority queue for a promotion. That way of thinking has to die and if you want to have a successful company, "promotions" shouldn't be limited to the number of upper level vacancies that need to be filled with someone. Instead, I argue that everyone should have a salary that scales with the level of their performance. I would rather have an environment where everyone can be rewarded for excellent performance instead of people waiting for someone to leave to get upgraded.

Real talent can't be kept waiting, because they will find other places to move upwards into and the company will be at a net loss in talent.

When it comes to making a successful company, having the right people is everything. More important than the amount of money that's bankrolled into the company. Smart, resourceful people will usually find a way to succeed. When your most talented members are gone, companies that were once great will be doomed.

Having a backup computer is important

2010-10-17T23:35:00.004+09:00

I've had previous experiences with computer hardware failure, namely having a laptop computer dying on me, leaving me without a PC to do any sort work or connection to the internet. Having a single point of failure for tools or equipment I use on a very regular basis can be a severe vulnerability and it is necessary to have a backup.

I learned this lesson through experience, after my first jaunt where my primary computing device was a laptop. It was great for about the first 1.5 years, then it died on me after expiry of my warranty, the extended care package also did not cover additional repairs at 100% and every time that something would fail, it would always be diagnosed as a motherboard "failure" which would cost just enough so I would seriously consider getting a new laptop instead.

After having enough of that experience, I decided that laptops had a too high cost of failure in terms of time (as repairs had a 1 month turn around time) and financial cost (ie. $700~$1000 for a motherboard replacement) to repair. Being experienced with the assembly of desktop computers, I know that I can generally replace damaged computer parts for far, far cheaper. I decided to switch back to having a desktop as my primary working system 3 years ago and put together my own system. The machine was working great until today when the computer started acting up when the power started cutting itself out during use. I am highly suspicious that the power supply needs to be replaced and I just put an order for a new part online for about $40 for a new unit.

The difference between $40 and $700~$1000 is staggering, and in addition, instead of having to wait 3~4 weeks for the repair, I'll have the parts arriving in the next few days to make the replacement.

When it comes to mobile computing, I have opted to have a cheap $400 netbook for use when I am on the road. Equipped with 160 GB in hard drive space and an atom processor with 7~8 hours of battery life, the machine has sufficient enough kick for me to watch movies on the go, digitally enhance and store travel photos, and the usual net browsing and e-mail applications. Should anything should happen to my little netbook, I would be far more comfortable with discarding it and getting a new once for about the same price. I am very happy with the decision to change my computing setup years ago, because it is finally paying off.