GitHub-copilot – stlplace

Reading Time: 2 minutes

We have using GitHub Copilot probably for 2 years now. First used as a tool for code view, and then expanded as plug-in inside the IDE (both IntelliJ and VS Code). We can ask all kinds of questions in the conversation dialog. Personally I used it to generate unit test code a few months ago.

While the tool is helpful both in code review and in software (or application) development in general, recently I also noticed two issues, both are annoying. And it seems the issues are somewhat related to what Martin Fowler said in his recent blog post “I don’t like LLMs“.

Issue one: it gave totally irrelevant code review comments. I don’t know where the comments originated from, but the tone it used again reminds me of the Mr. Fowler’s post. For me the only solution is add my comments in the Copilot code review comments and move on. It’s not the only time Copilot code review provided this kind of comments. The overall tone from Copilot is

Issue two: this one honestly stumbled upon me, and took me a while to understand the rationale of Copilot’s mistake. I asked Copilot to fix an issue of “unnecessary mockito stubbing” during unit testing. Note the code was generated by Copilot, and what it does was: to remove 2 lines of code which is not supposed to remove, while the correct solution is to remove the same 2 lines of code in a similar function. The main difference between the 2 functions here is one fucntion is with togglz off, while the other is with togglz on. It kept doing that for a few times, until I realized I can manually do it by removing those two lines in the correct function. I think in this case Copilot still is helpful to me – but this small incident reinforced my thinking of “trust but verify” for AI generated code.

At the same time, I read a bit of “When AI writes almost all code, what happens to software engineering?” (Substack link, I came across it from LinkedIn) by Pragmatic Programmer aka Gergely Orosz. It sounds like he is more bullish on AI or AI generating code. Note the Substack article was written in last Nov or Dec.

Another related observation is, I notice sometimes colleagues would say: let AI do the algorithm or something like that (the implied meaning seems to me is we are not as smart as AI).

One counter argument is: the big tech companies are the ones leading the charge using AI to generating code, but at the same time their code quality sometimes are horrendous, such as the Uber Driver App – e.g., this personal experience I had when I was delivering for Uber Eats.

Last but not least let me share a short talk I came across from my friend and a professor in management science at my alma mater Huazhong University of Science and Technology, as shown below. In Chinese. I assume you can ask ChatGPT or Google Gemini for translations if needed.

I learned a bit more GitHub Copilot recently

Reading Time: 2 minutes

I worked on a small project (in the Agile or JIRA world, it’s a story), and I experimented Copilot more this time, in fact, I think I tried to use Copilot as much or as practically as possible, for 2 out of the 3 main coding tasks. This is in addition to the Copilot code review feature which I have been using probably for a year now. Back to the coding tasks: refactoring and adding unit tests are the two main tasks that I used Copilot extensively. I also think Copilot did a decent job on bath. The 3rd task, which is to create a controller (an end point), I forgot to try the Copilot – hope I can try something like that down the road.

I used the IntelliJ Ultimate edition for my Java project IDE, and I was familiar with some IDE basic features on refactoring, such as extract methods, or change method signature. This mini project involves moving code from one layer down (from UI to the service layer). Copilot did it in one shot when I asked it to perform the task, from and to, mainly using the correct prompt.

For Unit Testing, I did two things, I changed the mocking framework from EasyMock to Mockito (which is more modern), also I added and cleaned all the unit tests, as much as I can. I admit I am still learning on the unit testing front. Sometimes I felt it’s somewhat like “teach an old dog a new trick”, I mean the “test driven development” or writing a complete unit test suite. But I am getting there: not just on Java, for JavaScript, I had some fun writing Jest test as well. For Java it’s JUnit.

Code review

This is similar as before I worked on this mini project, but I am getting more out of the Copilot code review comments, most of which I think is valid.

On a side note, I did a coding test for this LinkedIn AI trainer gig – not sure that I passed it, as I was not very confident on my Python language. But I still think it’s good exercise for me – I mean the coding test via HackerRank.