Category Archives: Web

Production, production, production

I first learned the “production environment” in 2010, when I worked as contractor for a major railway company. Before that I was mostly in CAD software development and consulting environment the word “production” did not come often. To be precise at Siemens PLM/UGS as developers, we did have access to various production releases and did validation for bug and bug fixes from time to time. Our code goes to release per year or per quarter. But production is not as significant as the maintenance releases, so this is the world of shrink wrap (engineering) software world.

Came to the world of business applications, or web. The first thing I learned is it’s not a good idea for newbies to touch production data. Or for that matter, not good idea for devs to touch that either. Very few people has production access, besides admins (database, web), the few people have access are usually product owner, business analysts, or product support people. And fast forward 5, 6 years, I became one of the latter. This is a privilege. Something I learned over past year:

1) Start from baby steps: e. g., if we want to update 1000 records: start from one or two records, do the update, validate and if everything looks good, do the mass update. This goes th way of divide/conquer too: so for example, if I need to delete 3 or 4 million records in one script (one run), I know it will be a long operation, and I don’t want the operation hang or fail in the middle. So what do I do? I divide the deleting operation into a few, each operation deletes half a million, much more manageable, and I will get the it complete much faster or get feedback much faster.

Continue reading

Moving wordpress from shared host to aws ec2

I did an experiment recently trying to move my wordpress website from a shared hosting site to Amazon aws ec2 micro instance (t1). The migration was mostly successful, I did not make the move eventually due to some technical and non-technical reasons. Nonetheless, I would like to share some of the lesson I learned from this process. I followed this blog post form smashmagazine as a blueprint for “moving wordpress”. Note I already have ubuntu installed on ec2, and has setup keys for my laptop.

1) Setup wordpress on new host. This is the link I followed to setup wordpress on ubuntu on my aws ec2. Another useful article is here.

2) Upload files: I used FileZilla (SFTP) for uploading the files. Here is the link regarding setup keys. Note the file permission is also very important to make wordpress work, which is especially important when moving the sites. More on this later.

3) File permission, one thing I noticed quickly is I don’t have any writing privilege after moving, e.g., update plug-ins etc. Did google quite a bit, and tries to change the file permissions using FileZilla. No luck. Eventually I read this from wordpress official doc. Quote:…(a) file ownership: all of your WordPress files must be owned by the user under which your web server executes. In other words, the owner of your WordPress files must match the user under which your web server executes. The web server user (e.g., “apache”, “web”, “www”, “nobody”, etc.) is not necessarily the owner of your WordPress files…

I did “ps -ef|grep httpd” at my ubuntu server, and find the user for apache web server (not “root”). And changed the file owner to this user (via “chown” command).

4) MySQL server crash: I found this happens quite often (as often as once every 3 days). Turns out to be a memory problem for the t1 micro instance and apache web server and mysql fights for the memory, eventually mysql lost. The solution I found is add virtual memory to the system, and reduce the memory requirement for mysql in config file. Eventually I was able to have mysql run as long as 13 days without shutting down. Not ideal, but still meaningful improvement (13 days uptime vs 3). Another idea, people talked about is use a lightweight http server such as this one.

5) MySQL database migration. Found out for large database, php export will not complete, and will end up with some junk html in the database file (.sql), it will fail during import. The workaround is do the export and import in the mysql command line. But I still have one problem with database, it appears the Chinese character got lost during this process. This problem along with the mysql server problem mentioned above, and the relative higher cost of AWS compared to shared host, made me decide to not switch at this time.

Other links:
Setup php/mysql, and phpMyAdmin.

Craig Emerson has an up-to-date article on this topic WordPress CloudFront CDN Setup Using W3 Total Cache.

Set up Apple TV with AT&T u-verse

It seems the wifi router comes with AT&T u-verse does not work with Apple TV. Apple TV could not find the wifi network from the build in u-verse wifi router. I found one workaround, that is to turn off the build in Wifi, use an old wifi router (D-Link DI-524) for Wifi instead. But there is still problem with the mirroring from iPad to Apple TV. Since I still have my previous LinkSys wrt54g router, I set it up as second router behind the DI-524, and connecting both the Apple TV and iPad to wrt54g. That worked. Another side benefits is wrt54g is a newer/more powerful router, and could broadcast wider range. So in a summary the architecture from internet to apple TV is something like below.

u-verse modem (wifi off) => D-Link DI-524 router => LinkSys wrt54g router => Apple TV/iPad

You may wonder why I add this extra D-Link DI-524 old router. The problem was I could not easily connect LinkSys wrt54g (the newer router) to u-verse modem. So I used the old router as a bridge. If someone has idea to make it work without this extra router, please let me know. (Email: minjie DOT xu AT gmail DOT com; or twitter @stlplace)

There is another problem with setup AT&T u-verse which is not related to Apple TV. The problem is when creating new account at their web site, they prompt for secret question/answer, it appears they don’t take any space in the answer. It was a programming error (web page text field validation) in my opinion, since I have done some web programming lately. The workaround is not to type any space. Simple enough.

This is also after using cable internet for about 8 or 9 years, I am switching to AT&T. I was using their DSL before switching to cable. The main reason is their service is cheaper, another factor is the cable had some connection issues lately.

(Update 05-07-2015) Came across this post regarding how to make direct connection from linksys wrt54gl to uverse router. I tried it but was not successful. It does not like the new local ip address 10.0.0.1 for linksys. But the theory there (conflicting local ip address between uverse and wrt54gl) seems holds, as I saw people discuss similar problem on linksys support forum.

stlplace.com was hacked and restored, plus some thoughts

I found out this site was hacked yesterday evening, as I saw the loading of site on Safari was slower than usual. It also shows incorrect theme, more like a plain theme. With the admin link redirect to spammer site. I decided to tackle it right away. I recall about one year ago something similar happened to this site, and google webmaster tool told me about it. I was able to remove the offending files/directories, by following the recommendation set out by google and some other wordpress sites.

This time I made an almost fatal mistake, during deleting some of the files in wp-includes, I accidentally deleted all the useful php files there. Panicked, I used both the website restore tool, and the import feature of wordpress (mojo marketplace), to no avail. The symptom of the problem was I could not login, and it shows blank page when I login via the wp-admin or wp-login.php after taking my credential. And I can only see the pages at uudaddy.com (I could not log in there either).

I filed for help at the hosting company. But I still feel helpless as this site has about 9 years of my blogging and uudaddy.com has my last 4 years of blogs. Fortunately I was able to find this wordpress help page about updating wordpress, and fix internal server error by deactivating the plug-ins. The latter comes only after I gained some web dev experience lately, knowing more about error 500 :-)

So long story short, I was able to restore all the blogs (those two plus my wife’s happy mandarin.com) by the following:
1) Restore .htaccess file to avoid the redirect to spammer page;
2) Restoring the wp-includes and wp-admin page: upload zip file, extract; in the wp-content directory, I renamed plugin directory as plugin.SAV (this way it deactivates all the plugins);
3) Run the wordpress update as soon as I can log in.

I also backed up the MySQL databases for stlplace and uudaddy, and backed up those two blogs to wordpress.com (this one and this one). I understand blogging itself is a declining trend, and it’s probably not easy for small web hosting company to defend against hackers like Google/Amazon/Wordpress do. That’s why I am also evaluating whether to move to wordpress.com or Amazon EC2 (self hosting). It’s a bit emotional decision as I have hosted this site for 9 years, but I need to consider both my time, the cost of hosting, etc. I will make a decision on this shortly. Meanwhile check out the new blogging sites I mentioned above, in which I will blog both about software development and raising kids.

Spring MVC, Maven

I’m back to Java world again, after about 2 years stint on Objective-C and iOS development. I’m not new to Java land, as I have done Java work between 2010 and 2011, for 2 years.

My personal learning experience:
JSF => Spring MVC (jspx is still very similar): I have done JSF, which is a component based Java web framework, we know Spring MVC is different: it’s based on REST, GET, POST, and mapping etc. Luckily I have done RESTful web services in my past work, and used those services from iOS side. Another new thing at client side is javascript, and jQuery, I recall I used Richfaces (extension of JSF) for the AJAX, they are not javascript though.

IBM web sphere/Rational application developer RAD, CVS => Spring STS, SVN, note both RAD/STS are based on Eclipse: for the application server, and development IDE, the transition is easier such they are very similar.

Ant (build.xml) => Maven (pom.xml): this is a big change, actually. Maven has some learning curve, also with the Nexus repository. My only past experience using Maven: use Netbeans to open the project (pom.xml file). I found the following two tutorials helpful.

Tutorial on Maven:
http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html

http://maven.apache.org/guides/getting-started/

Spring MVC: using the Get Started Guide from the dashboard, or the link here.

Obviously, learn as much as possible from the work itself, from coworkers etc. The good thing is my current work place is one of the most collaborative places I’ve ever worked. This makes my life a bit easier :-)

Consuming a PHP SOAP web service from C# client

I need to create a SOAP client in C#, and in order to test it I need a SOAP web service. Since my hosting company uses MySQL/PHP technology stack, I am using this PHP Soap web service example.

For the C# soap client, I am following this example on stackoverflow. But there is one problem when I tried to run the example. I got the error (exception) like the following:
The content type text/xml; charset=ISO-8859-1 of the response message does not match the content type of the binding (text/xml; charset=utf-8). If using a custom encoder, be sure that the IsContentTypeSupported method is implemented properly.

Again I found this thread on stackoverflow which seems relevant. It talked about the custom message encoder developed by Microsoft, and Paul Morgado’s addition to the CustomTextMessageEncoder class (just add his method at the end of the class). There is one more thing, the configuration file for the app. In my case, it’s something like the following: (note the messageVersion and bindingElementExtensions definition, there was some error in the Microsoft documentation regarding the latter one).

Continue reading

Software updates

WordPress 3.1.1
Finally I got hands around my blog (this one stlplace) and did the WordPress upgrade. I put off the upgrade in the past primarily due to my laziness: there is no automatic update due to I was running on WP 2.0.2. The upgrade itself is not too bad, I followed those two articles on Backup database and Run manual update. I used phpMyAdmin for the WP database backup. For the WP upgrade the only tricky part for me is:

“…Upload the individual files from the new wp-content folder to your existing wp-content folder, overwriting existing files. Do NOT delete your existing wp-content folder. Do NOT delete any files or folders in your existing wp-content directory (except for the one being overwritten by new files)…”

I know “directory overwriting” generally works on Windows system, but we are talking about UNIX (Linux to be precise). The command I used goes like this (in the blog root directory):

cp -r -u new_wp-content_dir .
(here -r means recursive for directories; -u means update)
I believe it did the trick, as I can run upgrade database afterwards. After that I enabled the necessary plugins. The only plugin stopped working is the ultimate warrior tags. The rest are fine.

Btw, I also added WPtouch plugin so that this blog works better on iPhone.

Xcode 3.2.6
I found my old Xcode 3.2.2 can not handle block as it’s necessary to run Ray’s RSS reader example. Download the software from Apple, install, and found this “Base SDK missing” in the new Xcode. Did some google, the trick is to “use latest SDK” in the “project => build” setting. I noticed we can also build “older” targets by changing the target in “deployment target”. Obviously there is a limit as to how far we can go back (I have not checked that yet).

Btw, I found the iPhone 4 simulator looks cooler than iPhone 3 simulator :-)

Spammer stole my Yahoo email and sent out spam mails

Friends, in case you received an email selling you Viagra online, which appears sent from my Yahoo email. I apologize. Obviously I don’t sell those. Some hackers used my Yahoo email and did this. Please ignore that mail. More information (prevention tips) about this tactic can be seen at:

I had my email account hacked into and emails sent out.?

I realized this problem as my Gmail acct is also in my Yahoo address book. Also, one friend asked me “huh”? I think this is not really brand new tactic, and I believe most people already know it. This note is just in case.

Continue reading

Facebook, LinkedIn & Twitter

Blogging is declining. My friend Wang Jianshuo reminds me about this trend. Obviously Jianshuo has much more authority on this because he started blogging on Sept 11 2002 and has a large followers at his blog.

On the other hand, I have joined Facebook, LinkedIn, Twitter this year (not necessarily in this order). And it seems to me all three websites are going strong. The main reason? I think it’s the networking effect. Facebook has been hot for a while, esp. among college students. LinkedIn got popular this year particularly since the recession made job search more difficult, and networking is much more important to land an interview or a job (compared to easy times). Twitter is more interesting, I noticed it changed slogan from “what’re you doing” to “what’s happening”. This precisely reflects the change of emphasis of Twitter, and the power of twitter, from Iran post-election to StockTwits (and many sites based on twitter, see 20 Twitter Apps you need to know).

Interestingly, I noticed one co-founder of Twitter, Evan Williams, is also the author of blogger. Maybe Evan already knows about this trend when he co-founded Twitter, which started a a mini-blog platform?

How do you know about visitors to your website ?

How much Google knows about you? The google dashboard. I found out the google search I did last Saturday on “enable root on Mac” (by going to google => “My Account” => “web history”). I believe there is a setting a google user can change to disable google to collect the data. But for me sometimes it could be useful because I want to repeat an old search I did a while back. A related question for business is:

How do you know about visitors to your website?
My title may be a bit misleading, I think a better question is how do you measure the performance of your web site? In other words, which page is highly viewed, which page gets no attention etc. Here comes web analytics (Wiki). Web analytics came of age lately as Omniture being bought by Adobe recently.

Google analytics
Google analytics is a free web analytics service offered by Google (Wiki entry here). Setup tutorial (video, link here)

Continue reading