Category: Software development

Categories
Software development

Silly mistakes in web service implementation

Reading Time: < 1 minute

Came across a strange problem this week using a web service from 3rd party. It’s a soap based web service. When we pass the XML with the required data elements in certain order, it works. If we did not follow that order, it complains invalid parameter or something. I guess in the code they have something like this:

if parameter_1 is invalid
return bad_parameter

if parameter_2 is invalid
return bad_parameter
…

In my opinion, we can do a simple fix as shown below:

if (parameter_1 is invalid) || (parameter_2 is invalid) || …

To be fair, I have made this kind of mistake myself, probably a while ago 🙂 I think it’s important for experienced developers have empathy towards people who make those mistakes, and understand how they come to the imperfect solution, and how to avoid those problems in the future.

(Update 05-09-2015) It looks like a common problem in web service, as explained in this thread. Quote:…Some vendors may grab SOAP parameter values by element name (the proper way) and others may grab the values by node position (what you’re probably running into)…

Another post on stackoverflow about similar problem on WCF SOAP web service.

Categories
Software development

Sustainable software development : I

Reading Time: 2 minutes

I am starting a series of “Sustainable software development” blog posts. Like many series I did in the past, it could be a series of one and only, or it could be a true fall classic (WS series).

Back to the topic, I have been doing software development for almost 15 years, been through many organizations, sometimes move between different organizations inside one company. And I think among all these agile, water fall, off shore, near shore, on shore etc., the most important of all is: consistently deliver values to customer. The key of what I said is “consistently” and “value”. Note I am not trying to add another one or two buzz words here. By that I mean everybody should be happy: employee or contractor don’t get overworked, feel they got the short end of the stick, managers don’t feel they got squeezed between sales people and the end customer, the customer does not feel been lied to (by the “used car” salesman). How do we achieve that? I think it’s both a management and engineering question or discipline. Note I said “sustainable” in the topic, if any of the one conditions (symptoms) is true in real world, what will happen? Employee or contractor will leave, manager will feel depressed (they may not leave as fast because it’s usually harder to find a manager position), customer will get angry and make threatening calls…

Note I will mentioned “software”, not “application” or something. Software is a bit older word than application, or “apps” the new buzz word popularized by Apple (Steve Jobs?). Good old fashion software. A lot of times when I saw “applications” people developed, it’s merely a configuration or customization of some prepackaged software. I used to work for company does that sort of thing too. Don’t get me wrong, there is value in customization or configuration. But that is NOT “software development”, that is “software customization” or “configuration”. Also, there is a drawback to this approach, the “upgrade nightmare”, because it’s hard to upgrade to the original software due to poor customization. Customization, if properly done, I think the changes can be localized, and the application can be maintained. On the other hand, if it’s not properly done, just like “hard code” things in the code, it becomes harder and harder to catch up with the vendor as time goes. It’s very likely those kinds of system will generate a lot of outages and support calls, and we know most developers don’t like handle production support problem all day (and night). They like development better.

This reminds me a related question I came across on Quora How do you make programmers work 60-80 hours per week? The short answer is don’t do that. In today’s world, we still think about a problem when in shower (just like Warren Buffett got the idea to buy BoA stock in shower :-), as Ryan Matte said in Quora: a programmer never really turns it off. So from management point of view, they should really encourage the hard working developers to go off to take a break. I recall my middle school teacher had very similar philosophy.

So this is my 2 cents to get started on this topic. I will hopefully get back to this soon…maybe talk about how to use agile and team size to make software development more sustainable:-)

So long…

Categories
Software development

Java app memory leak, performance

Reading Time: < 1 minute

A collection, I will add my comments later. Symptom: PermGen run out of space

Memory leak
Overview (Mark Thomas)
http://people.apache.org/~markt/presentations/2010-11-04-Memory-Leaks-60mins.pdf

Memory leak happened in multiple deployment (more likely happens in development)
http://stackoverflow.com/questions/7788280/memory-leak-when-redeploying-application-in-tomcat

Tomcat 7 (Mark Thomas)
http://java.dzone.com/articles/memory-leak-protection-tomcat

quote…
Application or library code that can trigger this situation include:
JDBC driver registration
Some logging frameworks
Storing objects in ThreadLocals and not removing them
Starting threads and not stopping them

There are also a number of places that using the standard Java API can trigger a similar issue. These include:
Using the javax.imageio API (the Google Web Toolkit can trigger this)
Using java.beans.Introspector.flushCaches() (Tomcat does this to prevent memory leaks caused by this caching)
Using XML parsing (the root cause is unknown due to a bug in the JRE)
Using RMI (somewhat ironically, causes a leak related to the garbage collector)
Reading resources from JAR files

Overview and use VisualVM to debug memory leak (Colm Divilly)
https://cdivilly.wordpress.com/2012/04/23/permgen-memory-leak/

Nikita Salnikov-Tarnovski
https://plumbr.eu/blog/what-is-a-permgen-leak

Performance
Overview of memory and performance of J2EE app (PDF, Steven Haines)
http://info.appdynamics.com/rs/appdynamics/images/Top_10_Java_Performance_Problems_eBook.pdf

Categories
Software development

Is pair programming worth the hassle?

Reading Time: 2 minutes

I have been doing programming for over 14+ years, and from time to time I did pair programming (mostly debugging problem). Since early last year I have been doing more pairing, mostly due to the hints from above. I have mixed feeling on this, mostly along the lines of advantages of disadvantages talked about here by Mark Weldon.

Quote Mark here, “When you pair program, you’re effectively joined at the hip with your pair. You can’t pair if only one of you is there. This means that you both come into work at the same time, you both take lunch at the same time, you both take breaks at the same time, and you both leave at the same time. The work is so concentrated that you work 8-hour days (which is good). But you can’t take time off without affecting your pair. Working from home is possible, but it’s obviously not as pairy as sitting beside one another…” I have very similar feeling here. Mark is probably an experienced developer. I also did a google search, saw other people with different background shared their thoughts on pair programming (manningdigital, a CS student). Mark also talked about the personality clash, essentially one developer with strong will (or more stubborn gets her/his way), which is also echoed by other 2.

Curious about this topic, and don’t want to limit by the google, I came to quora, and searched this topic. I came up with Is pair programming worth the trade off in engineering resources and Does Pair Programming really work.

And I saw this from the legendary coder Kent Beck, he suggested it works when

“The problem space is big and unknown. You don’t know what exact problem you are solving and the probability is high that any solution you find will change your understanding of the problem.
The solution space is big and unknown. You are working with technology you don’t fully understand and the chances are that any solution you find will change your understanding of what constitutes an effective solution.
You need to build relationships on a team…”

I think that’s where I agree with on pair programming at this time.

Categories
Site Info Software development Web

Moving wordpress from shared host to aws ec2

Reading Time: 2 minutes

I did an experiment recently trying to move my wordpress website from a shared hosting site to Amazon aws ec2 micro instance (t1). The migration was mostly successful, I did not make the move eventually due to some technical and non-technical reasons. Nonetheless, I would like to share some of the lesson I learned from this process. I followed this blog post form smashmagazine as a blueprint for “moving wordpress”. Note I already have ubuntu installed on ec2, and has setup keys for my laptop.

1) Setup wordpress on new host. This is the link I followed to setup wordpress on ubuntu on my aws ec2. Another useful article is here.

2) Upload files: I used FileZilla (SFTP) for uploading the files. Here is the link regarding setup keys. Note the file permission is also very important to make wordpress work, which is especially important when moving the sites. More on this later.

3) File permission, one thing I noticed quickly is I don’t have any writing privilege after moving, e.g., update plug-ins etc. Did google quite a bit, and tries to change the file permissions using FileZilla. No luck. Eventually I read this from wordpress official doc. Quote:…(a) file ownership: all of your WordPress files must be owned by the user under which your web server executes. In other words, the owner of your WordPress files must match the user under which your web server executes. The web server user (e.g., “apache”, “web”, “www”, “nobody”, etc.) is not necessarily the owner of your WordPress files…

I did “ps -ef|grep httpd” at my ubuntu server, and find the user for apache web server (not “root”). And changed the file owner to this user (via “chown” command).

4) MySQL server crash: I found this happens quite often (as often as once every 3 days). Turns out to be a memory problem for the t1 micro instance and apache web server and mysql fights for the memory, eventually mysql lost. The solution I found is add virtual memory to the system, and reduce the memory requirement for mysql in config file. Eventually I was able to have mysql run as long as 13 days without shutting down. Not ideal, but still meaningful improvement (13 days uptime vs 3). Another idea, people talked about is use a lightweight http server such as this one.

5) MySQL database migration. Found out for large database, php export will not complete, and will end up with some junk html in the database file (.sql), it will fail during import. The workaround is do the export and import in the mysql command line. But I still have one problem with database, it appears the Chinese character got lost during this process. This problem along with the mysql server problem mentioned above, and the relative higher cost of AWS compared to shared host, made me decide to not switch at this time.

Other links:
Setup php/mysql, and phpMyAdmin.

Craig Emerson has an up-to-date article on this topic WordPress CloudFront CDN Setup Using W3 Total Cache.

Categories
iPhone app Software development

Update sqlite database during iOS app update

Reading Time: 2 minutes

We all know updating sqlite database on an existing iOS app could be tricky, esp. with any schema (table) changes. In the good old days when I was developing in-house app, I found out the app would crash when I added a column to the sqlite database, after merely updating the app. The problem was, the old database won’t get updated if we don’t explicitly do it during app start-up (after app update). So what did I tell my user to do? Delete the app, and re-install the app. Not the most user-friendly way, and what if the user has some data on it that he/she wants to keep? What if this app have many users?

I had this updating issue in my mind for a while, because one of my apps needs to refresh its data. There are a few ways to do it: 1) Use a web service to get the data; 2) Just update the sqlite database. The former has some limitations: we could get data via web service, but note we still need to manipulate the database if we need to add a column or table (this is same for both approaches). The latter is faster in terms of development. So I decided to do it. I did some research on stackoverflow and found this post as my reference. Here is my implementation, if you are interested. The following is done in viewDidLoad.

Continue reading “Update sqlite database during iOS app update”
Categories
Software development

Rockstar developer

Reading Time: 2 minutes

I have thought about this topic for a long time. I remember seeing at one place that an exceptional developer can do work usually done by 10 developers. Or in other words, he/she is a rockstar developer.

Besides productivity, I think rockstar developer has the following traits:

1) Willing to share the knowledge with fellow developers, keep in mind we all learn from each other, rockstar can learn from (dare I say) ordinary developer too;

2) Attention to the details and code quality, and other good development practice such as TDD (test driven development), again sharing knowledge is applicable here;

Continue reading “Rockstar developer”
Categories
Software development

Singleton pattern backfired on me

Reading Time: 2 minutes

This is a continuation of my earlier post of Tree, recursive function and my dumb mistake. As I said in my last post, my solution was to create hash maps (singleton) to store those for the session. The main motivation was to help out the performance. But I found out it backfired on me, a few days ago and today. In both cases, it has to do with the hash map I created, it was something like (object , a list of objects). In both cases the list of objects was actually dynamic, and I made an assumption it was static. In other words, I thought I only need to put the mapping to the map once, then I can use it happily ever after. I think there might be ways to get around it, i.e., I should update the map when that list does change. I will investigate and work on it more. (Update 05-30-14) So I found a solution to one scenario, I was able to update the map as needed. And I use put method (java hashmap) to do that update.

Related articles

    Categories
    iPhone app Software development

    What types of developer are you?

    Reading Time: 3 minutes

    Or am I? 🙂 I think I’m a pragmatic programmer. Note this is also a book title I read, by pragmatic (note not agile), I think it’s about balance between software quality, effort and delivery date. It’s also about releasing software with known risks (including bugs) 🙁

    Pragmatic_Programmer

    I thought about this as I started practicing some new (new to me) programming techniques, e.g., pair programming, test driven development (TDD), and also seen developers of different personalities/experience adjusting to this, at the same time trying to deliver the project on schedule. I felt and considered myself to be lucky in this aspect, as I thought my career at UGS/Siemens PLM Software laid the foundation for me as developer, in other words, I was very lucky to work with some excellent developers (definitely top 20% according to Jeff Atwood, co-founder of stack overflow), and received their guidance and mentoring in my programming formative years. Note I was not doing Agile, TDD, pair programming at that time, at least not formally. But at the latter part of my stay there, I did quite a bit automated unit/regression tests there. Also, although there is no pair programming, whenever I feel stuck, I could go to a senior member of the team, and talk. Not to mention the long debug session. I wrote about them in my blog long time ago (in Chinese, such as this one and this one).

    Continue reading “What types of developer are you?”
    Categories
    Software development

    Dumb code mistake continues

    Reading Time: < 1 minute

    So I was able to fix that performance problem I mentioned in my previous post. But I inadvertently introduced a new problem in my previous fix. And here is the gist of how I did:


    while (some condition) {
    if(some other condition) {
    ...
    return;
    }
    }

    Guess what, I was able to put it into infinite loop…I can explain more if needed 🙂

    Continue reading “Dumb code mistake continues”