Sustainable software development II : rewrite and star performer

Rewrite
Developers usually like new project, sometimes called green field project, in the sense developers have more freedom to create (think of a white paper). In the same token, from time to time people will start a “full rewrite” project as the old project (code base) is “so messed up” and beyond repair. Is that real? A very high profile “rewrite” failure is the Netscape rewrite (see Joel Spolsky’s post here). And recently yours truly was involved in such “rewrite effort” as well. The result is also similar. I think not just developers, some managers also like the “rewrite” as well, due to various reasons. Some are valid, and some are not. Software (if it’s true software) usually lasts longer than people thought. In 2002, I visited my former workplace, a manufacturing place in Shanghai, and surprised to found they are still running my Foxbase/Foxpro based report generator, the the OS is Windows 3.1. At the time I worked on this (1994/1995), it saved 2 accounts, a few days of work (type calculators), each month. I bet they still used it for sometime after 2002, but I really doubt that will last till today, and the printer (Epison pin based) may not find punched hole printing papers :-)

Star Performer
I thought this problem for a while. I happened to listen to one podcast from “this agile life“, and read this article The surprising danger of being good at your job from yahoo/business insider, both helped me think more about this problem.

I think in software development world, it’s a very common phenomena, like @codinghorror said in this post “two types of progammers“, basically he is saying there 20% of developers are passionate about software development, 80% of them are there to get a pay check. But the star’s power and magic needs to be balanced or controlled, so that the teammates can also thrive and together the team can deliver something truly good.

Another related phenomena, is Lebron James in NBA. A few months ago we witnessed his heroics again. But Cavs did not win eventually. One reason I think is they are one man team. While their opponent Warriors are a real team.

Some observations of 401k in last 10 years

New job also means 401k investment election time. Here are my choices. Large cap: value 10%, index 30%, growth 20%, small cap: value 15%, index 15%, growth 15%. All US stocks :-)

I took a look at my #Siemens 401k 5 year performance, stock funds did better than all other, international stock was weak compared to US :-( The good thing is I keep all (98%) in stock funds.

Ideally we should look at 10 year or longer horizon for 401k, so I looked at as well, international stock was weak in last 10 years too :-(

The reason why international stock did not do well in last 10 years (note by international I mean mostly Japan and Europe), is the economy in both places grew very slowly, much slower than the US rate of about 3%. Japan may have negative growth rate.

New job, offense and defense in software world

I changed job again recently, to be more precise, I made the change at beginning of the month. So today marks the conclusion of first 3 weeks.

I felt fairly good so far, both in terms of environment and the work. It’s a bit different from what I did in last few years, as I engaged more in product support (still software world). I consider this to be more defense type of work, esp. the part of trying to keep application up and running, vs. the development work I did in the past (more like offense). Both are important. Some new tasks include handling customer requests (technical, second or third level if we look it from level of support). We can say this is a bit like defense too, because sometime people literally threw problems or questions at you (or me).

I did give it some thought before making the change. I’ve been doing software development for the most part in last 15 years (first 8 years at Siemens PLM Software formerly UGS or Unigraphics). I’ve done some customer support here or there, mostly when I was with Autodesk. Most recently I was doing development at Mercy. I still like development. But at the same time, at this stage of my career, I like to think about what I am good at and what I like to do more. I think troubleshooting, and problem solving are definitely in my comfort zone. I also need to step out my comfort zone a bit (aka customer support). And I like to learn more on production, operation, devops (the new buzz words, essentially the automation of deployment and production), and last but not least security. So here comes my rationale. I think in some of the places like google, the job is really refined and one thing I am interested is “site reliability engineering“. Another good thing about current place is it’s a decent size company and has some interesting problems: both technical and non-technical. So here comes challenges.

Let’s see how I do as time goes :-)

PS: I heard about the news about my old company’s incoming layoff. We went through a similar exercise last year when I was there. At that time I was relatively new at the old place. And I was a bit scared. I think this expansion/shrinking is a way of IT life nowadays. Ironically, I got call from a recruiter and she told me the news I already heard. I recall last year some recruiter even sent email to my working account. Just a fact of post linkedIn world. Also another of my old company had a layoff recently too. See my post at uudaddy.

PS2: I think in software development there are also both defense and offense. Maintenance, fix bugs, and refactoring can be considered as defense. New features, bells and whistles can be considered as offense. Both are important as if we don’t refactor, or maintain, soon or later the code or the software will become unmaintainable :-)

PS3: in the soccer (or football outside of US), we have seen players moved from middle field to defender position as they age. Similar can be said for software work, from development to support, from new development to maintenance.

PS4: did Sony release Play Station 4, really? Just kidding. Another thing I want to say is this transition does not mean “I am no longer a coder”, a more precise description is I will do more diverse things, from coding to anything else related to product support (applications, and customer), whatever it takes. In a fancy term, I will get a more holistic view of application in production. :-)

Switch from iPhone 5s to Amazon Fire Phone

I did a switch from iPhone 5s to Amazon Fire Phone about a week ago. This is not the first time I switched from iPhone to Android (Nexus 4), I did a switch for a few months in Summer 2013. It was ok, but I really missed iPhone for some of the features, and its smoothness. I think I still miss some of the things on iPhone, e.g., the “Cellular” in Settings tells me immediately how much data I used on iPhone. On Fire Phone (android), the UI is not as good. There were some rough edges esp. when switching apps, it slows down or when I tap on something, the response is not immediate sometimes (Safari). But for the most part, for a $80 phone (the price was $179, and Amazon offer a year prime membership there, which costs $99 by itself), I think it’s understandable.

I did get a Fitbit Charge from Microsoft store because Fire Phone does not come with pedometer. I didn’t intend to keep the Lumia Phone, so I sold it on Amazon right away.

By the way, since Amazon released Prime Music Players, I was fascinated by it. Essentially I don’t need Apple iTunes anymore :-) The Amazon app store does have some limitation compared to Google Play store, and some popular apps are not there yet, e.g., BoA and weChat apps. I read it’s possible to side load play store apps, but haven’t done so myself. Also, I still have the original iPad mini, so I can still use iOS apps there.

Some of strong points of Fire Phone
1) Large screen compared to iPhone 5s: 4.7″ vs. 4.0″, that’s meaningful.

2) Large storage room too 32 gb vs 16 gb, considering the OS and app sizes, the real usable space for phone/video/podcast is much bigger.

3) The camera is also good.

4) Some fun factor added on Fire Phone (or android), e.g. the daily changing theme (home screen background), for one.

Some migration tips
1) The contacts: I tried to put them on Gmail contacts, it syncs automatically.

2) Prime music: I installed the app (note the pre-installed music app is not Prime music app), and download the songs/album I like to have again.

3) Data usage: there is no way for Fire Phone to control the usage of cellular data per app, which iPhone does. I just turn off and on Cellular Data as time goes.

4) Fitbit: with the Fitbit on hand, the measurements of my steps are more accurate, as I did not carry iPhone 5s all the time. I do wear Fitbit all the time.

Maybe I will write a review for Fitbit charge sometime. The one thing I did (after I learned from Amazon product review) is I bought this fasteners to make sure I don’t lose it.

How I unlocked t-mobile iPhone 5s and transfer to H2O wireless

I have thought about this for a while, especially after hearing about republic wireless $25 month plan from my friend Luke and H2O wireless (thanks to mitbbs penny savers forum). Previously I only heard about the StraightTalk $50 plan ($45 now, it appears T-mobile also dropped its $50 plan to $40). A little background, I had an iPhone 5s for t-mobile, full price bought from Apple store in Oct/Nov 2013. I only used it for T-mobile prepaid legacy plan, which means “pay as you go”, 10 cents a min for phone call, and 10 cents per text message. I like to have the 3G data, like I had before from AT&T. Here are the steps I followed:

1) Unlock the iPhone 5s. Although I paid the full price, by default the phone is still locked to t-mobile network. The quickest way is to chat with t-mobile, which I did. Actually I did twice, because the first time they did not do the job. The second time they sent me notification saying my phone is unlocked.

2) Get a sim card for H2O wireless from Amazon. I made a mistake again. Without much thought I got this micro sim card first, and I quickly realized iPhone 5s needs nano sim.
h2o_wireless_sim
3) Last but not least, when my t-mobile prepaid balance came very low, and when it’s weekend (not critical time), I started transfer. I did online on h2o wireless website first. Haven’t seen the number transferred after a bit over a day. So I called. One problem is since my t-mobile is prepaid legacy plan, there is no account number. And I did not give correct pin per H2O rep. Got it solved. The next thing they asked me to install couple profiles for internet (because I want data plan). Then talk to the billing department setting up monthly pay, they gave me 10% off for setting up month bill auto pay via credit card, make the final bill $27. Note H2O actually uses AT&T network, and the data usage is available in real time from “Cellular” setting of the phone.

(update) The main reason I ditched T-mobile is its poor coverage. Sometimes I don’t get signal in the condo neighborhood. The wifi-call feature helped out a bit by tapping into the Wifi network, but it has two drawbacks: 1) It still uses the regular minutes; 2) When at work and I walk around, as the Wifi hot spots changes, the phone will disconnect or lose signal.

Also, how to setup Internet and LTE for H2O. The operator texted me two links. Install the first profile for initial Internet setup. The second one was sent via Dropbox link. No need to install Dropbox app, just tap top right, “direct save”, and install. It will complain about the first profile just installed. Go ahead and remove that one, the second one should install. This is not very intuitive as I did not hear instructions clearly from the operator, nor did I bother to look up their website for FAQ. From my own experience, the first one is probably just for verification or something. Also the operator asked if I want multimedia message, I said no because she mentioned I need to install some “jailbreak” app or something.

Also, I heard from my other friend, if one is really budget conscious, Ting is also a good choice, they use both Sprint and T-mobile network.

Silly mistakes in web service implementation

Came across a strange problem this week using a web service from 3rd party. It’s a soap based web service. When we pass the XML with the required data elements in certain order, it works. If we did not follow that order, it complains invalid parameter or something. I guess in the code they have something like this:

if parameter_1 is invalid
return bad_parameter

if parameter_2 is invalid
return bad_parameter

In my opinion, we can do a simple fix as shown below:

if (parameter_1 is invalid) || (parameter_2 is invalid) || …

To be fair, I have made this kind of mistake myself, probably a while ago :-) I think it’s important for experienced developers have empathy towards people who make those mistakes, and understand how they come to the imperfect solution, and how to avoid those problems in the future.

(Update 05-09-2015) It looks like a common problem in web service, as explained in this thread. Quote:…Some vendors may grab SOAP parameter values by element name (the proper way) and others may grab the values by node position (what you’re probably running into)…

Another post on stackoverflow about similar problem on WCF SOAP web service.

Sustainable software development : I

I am starting a series of “Sustainable software development” blog posts. Like many series I did in the past, it could be a series of one and only, or it could be a true fall classic (WS series).

Back to the topic, I have been doing software development for almost 15 years, been through many organizations, sometimes move between different organizations inside one company. And I think among all these agile, water fall, off shore, near shore, on shore etc., the most important of all is: consistently deliver values to customer. The key of what I said is “consistently” and “value”. Note I am not trying to add another one or two buzz words here. By that I mean everybody should be happy: employee or contractor don’t get overworked, feel they got the short end of the stick, managers don’t feel they got squeezed between sales people and the end customer, the customer does not feel been lied to (by the “used car” salesman). How do we achieve that? I think it’s both a management and engineering question or discipline. Note I said “sustainable” in the topic, if any of the one conditions (symptoms) is true in real world, what will happen? Employee or contractor will leave, manager will feel depressed (they may not leave as fast because it’s usually harder to find a manager position), customer will get angry and make threatening calls…

Note I will mentioned “software”, not “application” or something. Software is a bit older word than application, or “apps” the new buzz word popularized by Apple (Steve Jobs?). Good old fashion software. A lot of times when I saw “applications” people developed, it’s merely a configuration or customization of some prepackaged software. I used to work for company does that sort of thing too. Don’t get me wrong, there is value in customization or configuration. But that is NOT “software development”, that is “software customization” or “configuration”. Also, there is a drawback to this approach, the “upgrade nightmare”, because it’s hard to upgrade to the original software due to poor customization. Customization, if properly done, I think the changes can be localized, and the application can be maintained. On the other hand, if it’s not properly done, just like “hard code” things in the code, it becomes harder and harder to catch up with the vendor as time goes. It’s very likely those kinds of system will generate a lot of outages and support calls, and we know most developers don’t like handle production support problem all day (and night). They like development better.

This reminds me a related question I came across on Quora How do you make programmers work 60-80 hours per week? The short answer is don’t do that. In today’s world, we still think about a problem when in shower (just like Warren Buffett got the idea to buy BoA stock in shower :-), as Ryan Matte said in Quora: a programmer never really turns it off. So from management point of view, they should really encourage the hard working developers to go off to take a break. I recall my middle school teacher had very similar philosophy.

So this is my 2 cents to get started on this topic. I will hopefully get back to this soon…maybe talk about how to use agile and team size to make software development more sustainable:-)

So long…

Java app memory leak, performance

A collection, I will add my comments later. Symptom: PermGen run out of space

Memory leak
Overview (Mark Thomas)
http://people.apache.org/~markt/presentations/2010-11-04-Memory-Leaks-60mins.pdf

Memory leak happened in multiple deployment (more likely happens in development)
http://stackoverflow.com/questions/7788280/memory-leak-when-redeploying-application-in-tomcat

Tomcat 7 (Mark Thomas)
http://java.dzone.com/articles/memory-leak-protection-tomcat

quote…
Application or library code that can trigger this situation include:
JDBC driver registration
Some logging frameworks
Storing objects in ThreadLocals and not removing them
Starting threads and not stopping them

There are also a number of places that using the standard Java API can trigger a similar issue. These include:
Using the javax.imageio API (the Google Web Toolkit can trigger this)
Using java.beans.Introspector.flushCaches() (Tomcat does this to prevent memory leaks caused by this caching)
Using XML parsing (the root cause is unknown due to a bug in the JRE)
Using RMI (somewhat ironically, causes a leak related to the garbage collector)
Reading resources from JAR files

Overview and use VisualVM to debug memory leak (Colm Divilly)
https://cdivilly.wordpress.com/2012/04/23/permgen-memory-leak/

Nikita Salnikov-Tarnovski
https://plumbr.eu/blog/what-is-a-permgen-leak

Performance
Overview of memory and performance of J2EE app (PDF, Steven Haines)
http://info.appdynamics.com/rs/appdynamics/images/Top_10_Java_Performance_Problems_eBook.pdf

Upgrade to iOS 8.1.3: some tips and tricks

I decided to upgrade to iOS 7.1.3 to iOS 8 recently due to the t-mobile wifi calling feature which is only available on iOS 8. Previously I tried to delay the update due to the main reason: my iPhone 5s is almost full and I need to back up and delete photos. I know people talk about just update it by the way of iTunes desktop (using USB/lighting cable connecting iPhone and laptop).

After back up my photos to two iPhoto library, and made an iTunes backup to my PC (laptop), I need to decide one thing quickly: whether I want o upgrade to iOS 8.2 which can be done from iPhone itself (Settings => General …), or do I go through the trouble to find 8.1.3 from web and restore it myself. I decided to go the latter, because I don’t want the Apple watch app on 8.2, also my company does not support 8.2 (I need to use it for company email). So here is the site I found the ipsw file. After the download, run checksum using this md5 command, then verify the SHA-1 digest using openssl command. Restore it using iTunes 11.1.5 on my mac (hold alt and click “restore” to open ipsw file).

The usual new iPhone setup afterwards (I used restore from iCloud, an old backup). After that I got all the email, plus the apps on the phone a while back (May 2014 to be precise). One interesting thing is I noticed even the phone call/messages are also back dated to May 2014. But iCloud had my recent contacts and it appears all intact on the phone.

Some odds and ends
The day after the update (yesterday morning), I found I could not install apps. Turns out there is a massive iTunes store outage (I found it out as I tried iTunes music and tried my iPad mini).

My Sienna audio does not recognize the “updated” phone. The trick is delete “car multi-media” from phone and pair it again. I have another post dedicate to this topic. Interestingly, my Camry 2012 audio recognize the the “new” iPhone without problem. Also, all the existing WiFi connections are gone after upgrade, so I need to ask around the Wifi passwords again :-)

Google+ plus photo missed some recent pictures. I don’t think it’s a big deal, as I already have iPhoto and iTunes backup. But this combined with the iTunes outage, made me think twice about the cloud storage.

I will share more as I remember more things.

(Update 3-15-15) I learned form my friend, and also verified on my t-mobile account, the Wifi calling is not free. It’s merely use Wifi to enhance the signal. Because I am a prepaid user (legacy plan), so this does not really saving me money :-(

Update vs. restore
The main difference between update and restore is like this: think Windows update vs. re-install the OS. In other words, all the stuff (files and software; pictures and apps) will be intact after update, but restore will get rid of all that. It will start fresh. The reason I did restore instead of update is I don’t have enough space on my 16Gb iPhone 5s.

Is pair programming worth the hassle?

I have been doing programming for over 14+ years, and from time to time I did pair programming (mostly debugging problem). Since early last year I have been doing more pairing, mostly due to the hints from above. I have mixed feeling on this, mostly along the lines of advantages of disadvantages talked about here by Mark Weldon.

Quote Mark here, “When you pair program, you’re effectively joined at the hip with your pair. You can’t pair if only one of you is there. This means that you both come into work at the same time, you both take lunch at the same time, you both take breaks at the same time, and you both leave at the same time. The work is so concentrated that you work 8-hour days (which is good). But you can’t take time off without affecting your pair. Working from home is possible, but it’s obviously not as pairy as sitting beside one another…” I have very similar feeling here. Mark is probably an experienced developer. I also did a google search, saw other people with different background shared their thoughts on pair programming (manningdigital, a CS student). Mark also talked about the personality clash, essentially one developer with strong will (or more stubborn gets her/his way), which is also echoed by other 2.

Curious about this topic, and don’t want to limit by the google, I came to quora, and searched this topic. I came up with Is pair programming worth the trade off in engineering resources and Does Pair Programming really work.

And I saw this from the legendary coder Kent Beck, he suggested it works when

“The problem space is big and unknown. You don’t know what exact problem you are solving and the probability is high that any solution you find will change your understanding of the problem.
The solution space is big and unknown. You are working with technology you don’t fully understand and the chances are that any solution you find will change your understanding of what constitutes an effective solution.
You need to build relationships on a team…”

I think that’s where I agree with on pair programming at this time.