Time to pick up some database skills

I've been dabbling on and off about some data mining work for sometime now. The one thing I am kind of surprised about myself is that I haven't picked up any decent database skills. Some IT related work came by my way, the guys are looking for someone with C++ skills and and database related stuff.

I haven't touched a low level language like that in ages, the last time I did something with C++ was try to write my own poker bot. I actually had a clever system setup as I made a hack into the RAM to find out where the card data was stored and figured how the card data was stored.

I haven't done any sort of enterprise kind of programming before; most of the work I do with programming is scripting and numerical analysis kind of work; I run pretty much everything through the command line to get at data. But IT work for banks and getting into analysis kind of work seems kind of interesting. The problem is that the firms are Japanese and I sort of have doubts on the amount of money they would be paying. I've worked in one Japanese corporation already and I believe that I should be fetching a higher salary, oddly not being paid enough seems to be strong negative incentive to me-- there were times I worked harder as a student than I work in the company.

There are a lot of skills I need/want to pick up right now. I just have to hunker down and pursue a few things with great intensity. I intend to make programming one of my fortes.

JLPT Level 1

The JLPT is the Japanese Language Proficiency Test, basically a bench mark for the language ability of foreign people to comprehend the Japanese language. I don't have any qualifications as I have just picked up everything I needed along the way, but I have since come to the realization that it may come in handy for future employment opportunities. I am debating on picking up a few other qualifications on my own depending on the availability of my time.

I have decided to prepare myself for a move for a new career to better leverage my skills and improve income compared to the work I am doing now; even if it involves me picking up some new skills. The job market in Japan is interesting for people with international skills and I intend on breaking into that market career wise. There are probably other interesting opportunities in other Asian countries like Singapore, Taiwan and Hong Kong for international talent. One of those requirements to break into this field is a necessity for good multi-lingual skills in addition to either business or technical skills.

The need for international recognition as a baseline for certain skills may play an important role here. I will be looking into at least getting my JLPT level 1 certificate within the next year. I've lived here for 5 years now and I should at least get that certification.

Career planning is tricky

I am now in the process of looking for a new job. People might call me crazy to consider looking for new work with the economic down turn and all, but some how I am confident that I can find something better, that also pays more than what I am getting right now.

The job that I do isn't all that bad, and the working environment compared to other Japanese companies is pretty good, but I believe that I can do better and that I can pick up the skills needed to advance faster.

I finished reading a nook earlier this month called "kitchen confidential" written by Anthony Bourdain, a first hand biographical look into the nitty gritty of the culinary world. One interesting note that he made was that he could have been a better chef if he hadn't spent so much time chasing money, where as another talented chef went from Italian to pastries and through other fields of cooking, starting from close to nothing to learn everything he could. The fusion of knowledge would eventually make him a renowned cook.

Right know, I can admit that I am chasing the money, wanting to pick up the skills to advance into better paid positions. Raising one's income isn't a necessarily a bad thing, but what is it that one unwittingly gives up in this conquest?

Modeled numbers are easy to predict, this is the realm of accountants and mathematicians, but there is also a hidden lie beneath it all, we assume that the model doesn't change.

Accountants cannot account for the power of a technological or cultural vision becoming a reality. Like poker, predicting or creating the future isn't just about a numbers game, though knowing how to crunch the number does help. Are there intangible things that I am neglecting?

And so, I will probably come to a crossroad, there will be jobs that I can pick up skills for that may take me away from my world of research and development that I believe that I want to remain in or do I go into the world of finance or other uncharted territory that may pay better.

There will be more job offers coming my way in the coming months and more skills I will need to pick up to position my self for better opportunities. The question is, to which direction do I walk and why? Assuming that this is the right question to ask in the right place, sometimes it is just nice to just walk in the first place.

If there is a time to do it, it is now; I hear from other people it does get a little harder to find new work the older you get.

The Search for a New Apartment Begins

As mentioned in the previous post, I've been living in the company
dorm for the past 3 years and it's time for me to go. In Japan the
moving season is in March-April because the university year starts and
new hires start work at this period of time.

I've been looking on and off for places to for the past month through
the internet and after living in the dorms for a while, I am ready for
a nicer place to live. Somewhere with at least 2 rooms and a kitchen,
and relatively closer to the city in a nice building. It is going to
be much more expensive to the dorms that I've been living in, but I
will be able to invite friends over and host events, which should cut
down on my entertainment costs significantly.

The added benefit is that I'll be able to dedicate a room to a
study/work out area, meaning that I'll be able to do more activities
at home. I would like to see if the partition between having a study
versus having a bedroom + study will have an impact on my working
habits. Either way, having a new environment will be mentally
stimulating and I am quite certain that the change in environment will
have a positive impact.

At the same time, having increased associated living costs will force
me to keep myself to sharpen my skills, advance and look for new
opportunities. I believe that it's easy to stagnate if one gets too
comfortable-- because if things are pretty secure already, why try
harder? This is a habit that I am going to work on breaking, but I
believe that it requires some external "stress" to be applied to
myself.

Fortunately, I've still been tracking my personal expenses so I have a
general idea of what I can afford. In Japan, compared to North
America, there is a wider selection of places and prices of places to
live in. Where as in Vancouver, the cheapest place that you will be
lucky to find will be in the $800 range, going upwards. Living about 1
hour out of the city by train, you can probably get away with
something in the $500-$600 range if you wanted to save money. I'll be
aiming for something nicer and closer to the city. It'll be more
expensive, but I will be aiming to shoot for a career upgrade within
1.5 years for a respectable increase in salary. I have a general idea
of what other companies might be looking for in an international
candidate now so I will be pushing to get certified in the level 1
Japanese Language Proficiency Test and one other certification.

To accomplish these goals, I will be looking to reduce the number of
hours to focus on personal development. The idea is to invest in
myself and reap compounding rewards the more skilled I get. I think
this is a far better solution compared to working longer hours-- as I
loathe to think of myself as a monkey. Other commitments I'll need to
make is to cut down on the number of hours that I spend unproductively
and I am in the process of setting up a timer system where I focus on
a task for a specific amount of time before giving myself a break to
do whatever. I'll also be looking to find a way to tabulate what I
learned or accomplished during these times to determine if I was
working effectively.

Time to move out of the company dorm

Living in a dorm of international students was great, living in a dorm with co-workers is pretty damn boring. I exist in a different world, even when in Japan with international expats, Japanese people that have lived abroad and came back, I even really like the country folk here as well. The thing is that I just can't connect with the ordinary traditional salary-people here. The just don't jive with me, not that I don't like them; I am just bored around them. I generally do my best to fit in and I am usually pretty good at fitting in, but there are times when you simply just need to put your foot down and get out.

Yeah the rent for the company dorm is really cheap, they charge about $250/month in fees with utilities included. Dirt cheap and thanks to that price, I've saved up a good chunk of money that would have otherwise gone to the landlord. But at the same time, you also pay a price, that you can't invite outside friends and the other thing is that I live away from Tokyo, from where all the cool stuff is happening. I've just started to meet many interesting people and decided that I need to get closer to Tokyo and make more free time to do interesting work and meet interesting people-- the surrounding environment plays a very important influencing factor. You can tell that things are different when you listen to people with interesting ideas and trying to startup a new business, whether it is in silicon valley or in Singapore; you think depending on your environment.

So I've decided that I need to get out and get myself into a more happening place. I'll probably have to deal with a commute to get to work, but it is one of those things I'll probably have to live with. In ideal circumstances, I'll find a new venue of work where it is close to where I live, somewhere happening. I've been living too "defensively" for the last few years by making decisions based on how much it I can "financially save." I want to from "saving" to "making".

A weekend of relaxation

After a wild Halloween and more weekend events (one of which included me dancing in a museum and the curators getting mad at me), I've decided to spend this weekend at home for a change. Most of my friends would have a hard time imagining me spending time at home instead of going out, but yes there are times when it is really nice to just not go out; in this case I had a lot of errands that needed catching up on and I figured that it would be now or never to get them done.

Stuff like cleaning the room a little, changing the battery on my scooter, hitting the hardware store to look for a can of rust remover and finding spray can of grease (interestingly) for my bicycle and such, I remind my self that there are probably a lot of small little maintenance things that I ought to be doing but just haven't really been finding the time to do recently. So sometimes, it is necessary to put time aside to get things done and time to just chill out as well too.

The last week has been marked by a string of food indulgence, as not many people might know, I live without a kitchen so I've created my own little kitchen with a table and a hot plate that lets me fry meats and make all sorts of other dishes. Having good food waiting for me at home is a great incentive to come home early; I often have a hard time staying late at work when given the choice of eating at the company cafeteria (which isn't bad, by the way) versus something home cooked.

It's been a while since I've done any real picture posting-- I still remember that I have more photos of Hakodate that I should have posted and other things that I should have done but haven't gotten around to doing yet. Slacking isn't a really good habit, but there are only so many productive hours in a day that a person has (or so I believe).

So first off, since February of this year, I bought a new cross bike and have been riding it from time to time to the beach on the weekends. Round trip of 30 km makes for great riding. The entire distance can be covered in about 1.5 hours but I like to take a breather at the beach, where I like to buy a can of orange juice and watch the beach goers play volleyball and surf over there. With autumn dawning upon us, the sun sets much earlier and the sky much clearer compared to the hazy humid sky that one would typically see.

The setting sun

The beach center

The other nice thing about autumn in Japan is that it is still relatively warm-- I can still venture out in a t-shirt on a sunny day, which makes doing exercising in warm but cool weather perfect.

I've also been doing a lot of youtube watching recently. My favorite videos to watch on youtube are of the cooking variety. I've already posted recently about one of my favorite youtube chef at foodwishes but in addition I also really enjoy watching Anthony Bourdain's series called No Reservations; a show that I highly recommend for anyone that enjoys both travel and good food. One of his shows aired way back was a "Techniques Special" from which I used as a base to replicate a beef stew.

The recipe is rather simple:

Take a hunk of shoulder cut beef, chop it up, season with salt and pepper.

Fry until brown

Add onions and cook until semi-translucent

Add some red wine.

Simmer for about 1.5 hours

Add carrots and cook until tender

Serve with a side of rice, bread or whatever you have lying around.

I did all this while cleaning the room and gave up on cleaning after the stew was ready. An incredibly easy but tasty dish. Would probably be nice to add some potatoes as well. I've got enough food in the fridge to last several days. Eating leftovers for 1 might get a little repetitious sometimes, but with enough variety it beats eating expensive take out any day.

Stay on your toes

I'll be writing some more later on but for now, there will be a short lull in writing. Work is busy at the moment and a lot of things are going on. Will be back later!

forget the limiting, focus on the enabling

I've posted plenty about my gripes on working in a corporation and doing research in an institution. I've decided that it's time to stop worrying about all the annoying things that bug me about these establishments and free myself of the "me versus them mindset" and become more fluid in dealing with roadblocks.

Perhaps I was naive when I was younger, when I saw a system or an organization that I thought was ineffective, that it would be easy to encourage change. After trying with little real results, I have come to the realization that change is hard and not worth the time. It's more effective to create a pocket in an organization where you can work effectively and if things are not optimal, then it's time to go. When finally getting into the real world, the scale that you deal with things changes, dramatically.

Before, life would revolve around the academic environment, going to elementary school or high school with 1,200 students and only interacting with mostly people in the same grade (say several hundred people) means that you have a really small point of reference. Going to a university of 40,000 people is a little different. When you finally get out of university and into the real world of millions of people the scale of what you deal with changes. When you're dealing with just a few hundred people and you're stuck with them for many years, that was the environment that you're stuck in and it's rather hard to change. You had to live with the things that you liked and didn't like. After getting out of school, the world is a big place and you don't have to put the effort into changing an environment to make it more palatable. If it's something that you don't like then you can easily move on.

And that's the thing when it comes to attracting smart people, keeping them and creating the right environment to getting good work done. I don't have time to fight the system, to convince people that doing things a different way is better and then doing it. Life is too short for that and I just want to find the right people and environment to be successful, because at the end of it, it takes way less effort to fight within a system and do something than to disengage and do it yourself. The strategy is to find the right way to be successful on your own. If the only taught that at school...

Performance review

I just had my performance review yesterday. In 6 months, I did a decent amount of work and at the same time, I also reduced the number of over time hours I've been putting in at work. I was pulling about 55 hours of overtime/month last year and this year, I've brought it down to about 30 hours/month. Having an extra 24 hours/month to yourself does make a difference in quality of life. Don't underestimate the value of time. I'm going off on a tangent however...

After my performance review, I asked my boss what does it take to get a promotion. How things work in a corporation is that you get raises based on the number of years you've been around, on top of that, after a certain number of years you can be recommended for a grade promotion. His response was that I need to take more of a leadership role and that I could be considered to get a promotion after my 4th year. Hearing something like that was shocking and disappointing. Why does it have to take at least 4 years to get considered for a promotion? I believe that promotions should be performance based, depending on the capacity of the person to handle responsibility and their ability to do good work. To be stuck in a position for 4 years and then take say another 5~6 years for another promotion is pretty much career advancement at a snail's pace.

I don't want to be half-way dead by the time I get into middle-upper management.

There has to be a better way at being successful instead of lining up in the seniority queue for a promotion. That way of thinking has to die and if you want to have a successful company, "promotions" shouldn't be limited to the number of upper level vacancies that need to be filled with someone. Instead, I argue that everyone should have a salary that scales with the level of their performance. I would rather have an environment where everyone can be rewarded for excellent performance instead of people waiting for someone to leave to get upgraded.

Real talent can't be kept waiting, because they will find other places to move upwards into and the company will be at a net loss in talent.

When it comes to making a successful company, having the right people is everything. More important than the amount of money that's bankrolled into the company. Smart, resourceful people will usually find a way to succeed. When your most talented members are gone, companies that were once great will be doomed.

Having a backup computer is important

I've had previous experiences with computer hardware failure, namely having a laptop computer dying on me, leaving me without a PC to do any sort work or connection to the internet. Having a single point of failure for tools or equipment I use on a very regular basis can be a severe vulnerability and it is necessary to have a backup.

I learned this lesson through experience, after my first jaunt where my primary computing device was a laptop. It was great for about the first 1.5 years, then it died on me after expiry of my warranty, the extended care package also did not cover additional repairs at 100% and every time that something would fail, it would always be diagnosed as a motherboard "failure" which would cost just enough so I would seriously consider getting a new laptop instead.

After having enough of that experience, I decided that laptops had a too high cost of failure in terms of time (as repairs had a 1 month turn around time) and financial cost (ie. $700~$1000 for a motherboard replacement) to repair. Being experienced with the assembly of desktop computers, I know that I can generally replace damaged computer parts for far, far cheaper. I decided to switch back to having a desktop as my primary working system 3 years ago and put together my own system. The machine was working great until today when the computer started acting up when the power started cutting itself out during use. I am highly suspicious that the power supply needs to be replaced and I just put an order for a new part online for about $40 for a new unit.

The difference between $40 and $700~$1000 is staggering, and in addition, instead of having to wait 3~4 weeks for the repair, I'll have the parts arriving in the next few days to make the replacement.

When it comes to mobile computing, I have opted to have a cheap $400 netbook for use when I am on the road. Equipped with 160 GB in hard drive space and an atom processor with 7~8 hours of battery life, the machine has sufficient enough kick for me to watch movies on the go, digitally enhance and store travel photos, and the usual net browsing and e-mail applications. Should anything should happen to my little netbook, I would be far more comfortable with discarding it and getting a new once for about the same price. I am very happy with the decision to change my computing setup years ago, because it is finally paying off.

The beginnings of a bookmark list

I incredibly happy about the news aggregator sites like slashdot, reddit and now hacker news. With so many articles being posted everyday, and sometimes, degradation of communities with growing number of people, I have decided that I would like to create a bookmark list of good articles that I have read to share with other people. It may simply be a page on this site for now, but I would like to more functionality to create feeds with other peoples bookmark lists to enrich breadth of reading topics and material.

Being with the right peoole is rejuvinating

Yesterday, I had the pleasure of going to a small gathering of Tokyo based programmers working or interested in start ups. Though interaction with everyone wasn't technical in nature, it was great to meet people working to starting their own businesses.

The great part of everyone was how supportive to each other they were. I would have thought that the world of startups be all about competition but it isn't, it is also about people helping each other out. I sense of altruism and community resonates well with me and I want to work in an environment like that.

I talked about being in a place with the right kind of culture in an earlier post, that community is the kind of place where I want to be.

Know your tier 1 friends

Life is short and the things is that you should you know in life is know who you can depend on. When push cones to shove, there aren't a whole lot of people in life that you can depend on in life.

There are people in life that will call upon in you when they need you, and there are people that won't come to help you when it comes time when you need help. Maybe when a person grows up is becomes necessary to separate the wheat from the chaff. It is necessary to know who you can depend on. Those that you can depend on, treasure them. The rest, is a decision call. But if you can't depend on them then there isn't a point to being friends with them, but that us the cold truth. You might as well know who you can depend on now then waste your time on useless people later.

If this sounds cold, then realize that the truth is equally cold, because this is what you must answer to everyday. If you want to have real friends then find real people that you can depend on.

Company culture is important

I went to a company seminar for junior employees of the company I work for today. I was throttled with disappointment and have reservations about the future direction of the corporation that I work at. The environment and the kind of people you hire are essential to create the right kind of culture/atmosphere that makes a successful company. We face a creativity and innovation crisis and I am starting to see that part of the problem is a result of the culture of the company -- there aren't a lot of people with the right kind of creative flare, follow through and leadership to pull off innovative and visionary projects. Hell, I don't even think that there is a real vision at work.

You know when you are in trouble when the executives are telling employees that "we have to be more creative," thinking that just telling people to be a certain way will enact change. When you have executives that can't lead, set good examples or communicate inspirationally throughout the company then there are very fundamental problems. The problem is even worse when there are superiors that don't listen or are stubborn and you have weak subordinates that know when it's time to push against the grain to get things done. We lost 6 months on one project because people weren't able to speak up. But back to the seminar I went to today.

The seminar I attended is a career planning seminar for junior employees (though I would not like to consider myself one). The idea that over the next few years, we are supposed to be preparing a general vision of ourselves, the kind of person that we are to evolve to be and present that vision to our project managers and eventually to the division executives when we get to our 4th year at the company. Feels like being back in high school by the sound of it.

I feel that many managers are in search of an answer for the next big thing. What many of them can do to a certain degree is identify them when they squeeze ideas out of their employees. I also find that so few of them have true visionary ideas of their own. The only difference between them and their employees is that they're subjected to more ideas and can pretend that they know more-- there is know easier way to stick out your chest and look like you are an "alpha" by appearing to know more than your colleagues. It is by nature of their profession that managers act this way. Dilbert comics make a good point about this with the pointy haired boss, know much about buzz words but little behind their meanings. To be a good manager, I believe that there is more to just knowing, but the ability to get things done on your own. If you want to inspire and attract the right kind of people, you have to start off as being the right kind of person.

Changing corporate culture, or culture in general is a tough process. I've tried it to little avail once before when I was a student at a Japanese lab while doing my masters. I've seen and experienced different things as a westerner and people are afraid of trying new things as old habits are hard to break. I talked about how we how it would be a good idea to start having better ways of exchanging ideas between members of the lab, (the lab I was a part of was really big, with 50 researchers) and better initial training to bring new members up to speed instead of fending for themselves. I eventually ran into a brick wall, with insufficient support and with looming deadlines coming up, I gritted my teeth and decided that I wanted to be the best possible person that I could be instead of changing my environment and did my own things.

Sometimes there isn't time to fight the fight and other times, it just isn't worth it. One of the things I learned over the years is to not waste time fighting over small things. If the effort it too much for some small return, then it isn't worth it to waste time over it-- simply moving on and finding better people to deal with is far more rewarding. I think that a lot of smart people are aware of this and when it become too much of a pain to do interesting work, you will start to lose your smartest people. Google, Microsoft and other technology companies know how valuable smart people are that they even had an agreement to not poach each other's company. If you want to be successful, part of it is knowing how to attract and keep the right kind of people to work with. When it comes to success, people and culture is crucial.

Financial Report Aggregation

Started reading some financial reports from Amazon and Apple today. What was interesting about the both of these companies is how they've deployed their capital in international markets to hedge against the declining USD. Amazon, in their 2010 Annual report would stand to lose money if the USD appreciated. Apple seems to be hedging against the declining USD as noted in their July annual report.

It would be interesting to see what kind of economic sentiment one can pickup from aggregating information by reading financial reports to figure out how businesses are positioning themselves for the future. So a question I've started mulling is how does one aggregate financial information from many sources well? The amount of information that I can process is limited, is there a collaborative strategy or a digital processing techniques?

Will need to do more reading to figure out a strategy.

The "Life goes on" Stance

Maybe it's because I think too much, but I am the kind of person that likes making optimal decisions. I wonder if I am using the most optical technique for math problems, I wonder if the data structures and the functional relations are setup to write clean code, and I wonder if moving out to a more lively place, while costing more is better than living cheaper further way from the city.

Decisions, decisions, decisions. The problem is that one doesn't really know the end effects of a decision until having made them. For whatever known or unknown circumstances things may or may not go as according as planned. The question, is it really worth it worrying about all that stuff?

It's been ages since I played any Texas Hold'em poker, but I took away some very important lessons in life from this game. Texas hold'em poker is an incomplete information game, that is the beauty of this game-- the idea that you have to make decisions without knowing every piece of information available. You could still make the "best" decision given the information available and still lose and that is where the "risk management" aspects of the game kicks in. Sure you can have a 90% to win probability, but if you're playing all-in all the time then you are going to have your ass handed to you eventually. The moral of the story is that for whatever decision you make, even if it goes terribly wrong, you ought to be able to take the hits and keep on going (Rocky 6 reference).

An that's sort of the thing with life-- you do things, somethings fail some of them work out. Unforeseen circumstances kick you in the rear and sometimes pleasant unexpected surprises fall from the sky. But that's really all there is to it, no matter what happens, for better or worse, life simply goes on and there isn't much that you worrying can solve.

Being robust is more important than being optimal

There are too many things in life to worry about and worrying takes up precious time. It would be nice to reduce the amount of time worrying and increase productive time and state of mind. The simple solution is to make robust decisions, where the expected value if positive and failure is not catastrophic. That is not to say that one shouldn't make bold decisions; but for the bold decisions that you do make, you ought to be able to take the hit and still be able to stand if things don't work out. The interesting thing though about a lot of optimization mechanics is that risk/robustness analysis is often forgotten and left as a hidden variable-- things work out nicely unless something fails.

Being robust also means that you can set yourself up for some high yield/high risk (relative) situations since you could have a fall back setup should things don't work out. I'll say it here, but knowing how to be robust and knowing how to take chances and succeeding at them is a key ingredient when it comes to being successful (and even sexy).

The point is to not let failures be hangups or crippling and keep moving on. Oddly enough, this almost sounds like an investment strategy.

Goofing off pays off

Richard Feynman, the famous physicist, had mentioned that he had lost the inspiration to find new and useful ideas in physics. He also apparently was in a rut for it, where upon he just gave up on thinking hard about it and decided to let his mind meander to do whatever it felt like. The ends result was that he ended up winning a Nobel prize for his mind meandering when he came across a student in the Cornell cafeteria spinning plates and noticed that the plate had a 2:1 wobble to spin ratio. After performing the physical calculations and being chided by one of his colleagues that the analysis he was doing was "useless," Feynman would go on to win a Nobel Prize for a derivative work from the calculations he did from the wobbling place as it was related to electron spins in quantum electro-dynamics. For those that are interested, a small part of the story can be found here, and if you do have a chance, I highly recommend Feynman's book, "Surely You're Joking, Mr. Feynman!" as a great read.

I too would run into a similar problem these last few weeks after being thrown into the flame of writing simulation scripts using a terrible scripting language. The syntax is strict, variable names are strange and the compiler doesn't accept lowercase letters in the code. The code looks like Perl on bad crack and it's surprising how people can get work done in this language.

I have a colleague working on the simulation project and he's build up a code base over the last 2 years, he's resigned to bashing through this programming language to run simulations. I've tried to read the code and fell asleep multiple times and got several headaches as a result of trying to follow a convoluted path of variables. split across several files to figure out what it going on.

Traumatized, I gave up trying to write code in this language. After 2 days of trying to understand the code, I was pretty much in a rut with this terrible code and resigned myself to clearing my head by working on other projects and taking really long breaks. Nearly a week later, after getting a handle on the fundamentals, I was struck with an idea-- basically, I would write a program to writes the simulation script and I would by pass all the terrible variable handling by natively managing variables in a different language. I managed to clean up the interface considerably, write a main processing loop where I just throw cleanly designed data sets to generate complicated physical structures with simple commands.

What my colleague would need nearly a week to testing and debugging, I could do the same thing in an afternoon. After joining my colleague in the simulation team, we're going to throw out all the old code and redesign everything using my framework.

Perhaps it is an inherent trait in me, that I hate using an inelegant system to do something and my natural instinct is to run away from the problem until I find an elegant way of dealing with the problem. Sure, I may have been goofing off for the past week, but I've figured out to nearly multiply my productivity by a factor of 4~5 over my colleague and will be bringing him up to my speed after I teach him how to my code. Increasing the productivity of 2 people by a factor of 4~5 is huge when it comes to the amount of cost savings the company yields. I will be writing about this in my winter evaluation report no doubt.

Though this story may sound like that it serves a didactic anecdote, that running away from hard problems is a good idea, I assure you that it doesn't and there are times when it is necessary to roll up the sleeves and to the heavy duty grunt work. The question is knowing when to run and look for better solutions vs getting on with it is key. Maybe I'm just lazy, but I generally like attacking problems when I have an idea to make it easy to solve, otherwise, I'll be doing other things.

You can't Schedule Productivity

I tested, debugged and coded for most of the the day, by the afternoon, I was just too mentally tired to do anything else-- the afternoon was somewhat of a write off as I could not bring myself to learn a language for a light simulation program that I need to learn. I am coming to the conclusion that I have a limited attention span for certain activities, so why push it when you aren't going to be able to get any productive work done?

With that idea in mind, I have second thoughts about the 9-5 pm work day. Thing is that we aren't necessarily at our mental peak during this time period and hammering at a problem when not mentally prepared could be a waste of time. Just because a set time is designated to do something does not necessarily mean that it is the best time to do it. The human mind and emotions can be so fickle, unfortunately that it is just beyond the comprehension of logic-- it would be nice to be able to do something exactly when one decides to, but it doesn't work that way.

So I am here, stuck at the office, tired after programming and not in the mood to more work. It's 5 pm right now and it's just a matter of time before I can pack up shop and head home. The question I repeatedly ask myself, is time spent really the right metric to measure how much a company should pay a person?
Many of the brightest people I know do not follow the 9-5 syndrome. These people are writers, artists, engineers and hackers. Ideas come to them where ever they are and they work as long as they have to do whatever they have in mind. I've worked on projects well into the night because I thought it was something interesting to so, I've done the same for writing when an interesting idea hits. One of the things that I enjoy is having my iPhone where I am able to write immediately when a thought is fresh in my head instead of it becoming dulled after getting home.

Though it is important to have a form of daily rhythm/schedule, there are somethings that can't be scheduled for; and one of those things is setting a time when you are going to be mentally productive. Perhaps there are mental exercises to improve one's concentration and memory, but I have yet to find something that helps one to get into that "productivity zone." Some reading into human psychology might be something interesting to look into, but for now, I am going to give up on the notion that I am going to get a lot of productive coding work done and focus on other things instead.

Charging for what would usually be free software

I've started getting to simulation for a project that I am doing at work. Long story short is that the language for the simulation program is pretty bad and I have decided to use python in between to use the simulator to work with. The problem is that the $30,000 simulator (which is only 104 MB large) is rather buggy and requires me to use a specific version of python. They recommended that I use a prepackaged version of python that comes from a company called active state.

The thing about python is that there are many optional free libraries that can be used together. In general you should have no problem using the libraries as they have installers to get the thing working. The product/service that active-state provides is sort of a automatic version management system to keep track of the different libraries installed and automates the update process and provides a repository that you can download modules for different programming languages. Their module tracking software for python is called pypm (or Python Manager).

Community Edition vs Business Edition

The distinction between the two version is sketchy at best. On their website they trumpet the use of Financial Modules, Database modules and more. I downloaded the community version since I was recommended by the developer of the simulation software to use it. I then tried to install the usual free numerical calculation modules (numpy, scipy, and matplotlib) for python from their repository and was prompted that I needed a business license to get it; the price for that license is $999, to get access to that package and other packages that required a business license to use. I can't tell if this is a great business idea or a total scam.

I instead went online to download the numerical calculation packages and install them. For the version of python I had before, it worked perfectly. After installing the Active state version of python and installing the packages I regularly use, the plotting package I usually use can't be imported because of a missing library. This doesn't instill a lot of confidence in their software: suppose that I download the libraries from them and it didn't work, I'd be short $999 and very unhappy. The difference between the community edition vs the business edition is "whatever software package they have available, except for some well used ones that we want you to pay for."

On the Linux front, software package management is all done via the freely available repositories available on the internet. I am generally informed automatically of the updates and updating is done automatically. Management of the python libraries is also done automatically and for free. Though I am quite certain that adoption of Linux based systems is still fairly low, should adoption of software repositories and management of installed software be done through these packages be done on Windows, a business like the one ActiveState runs would be killed off fairly quickly.

Maintaining code for other people sucks

Perhaps I might not be the best programmer in the world, but I can do enough to get things done. Last month, I managed to write a macro that does fitting for measurement curves to extract system parameters for some devices we are making.

We've got a production line going making devices that needed some assistance for device characterization, so I decided to pitch in and write a program that does the fitting for them. It took about a week to write, test, debug and install into the factory line and things were going great until I got an e-mail this week saying that the program wasn't working well. It turns out that the new batch of devices that are being fabricated have characteristics that make it hard to fit nicely and the mathematical functions that is used to fit the data against is somewhat unstable under certain conditions that causes it to blow up, meaning that the solver that comes pre-equipped in Microsoft Excel runs into a brick wall when the function blows up on it.

I've decided to write a new solver to deal with the problem and I need to go through the entire writing, testing and debugging phase again to add in this new module. As a result, I am not particularly happy with this predicament as it sucks up time from me doing anything else.

So I'll say that I've learned a valuable lesson here: I've written useful code for someone else that doesn't know how to maintain it, resulting in me having to keep track of things when conditions change or when things go wrong. The code doesn't benefit me and it costs me time in after care. Perhaps learning about code after care is important as well, but I would much rather maintain code that is a direct benefit to me; like dealing with 2 birds with one stone. In this case, it's my stone being used to hunt for someone else's bird.