Tag: copilot

Categories
Software development

GitHub Copilot recent experience

Reading Time: 2 minutes

We have using GitHub Copilot probably for 2 years now. First used as a tool for code view, and then expanded as plug-in inside the IDE (both IntelliJ and VS Code). We can ask all kinds of questions in the conversation dialog. Personally I used it to generate unit test code a few months ago.

While the tool is helpful both in code review and in software (or application) development in general, recently I also noticed two issues, both are annoying. And it seems the issues are somewhat related to what Martin Fowler said in his recent blog post “I don’t like LLMs“.

Issue one: it gave totally irrelevant code review comments. I don’t know where the comments originated from, but the tone it used again reminds me of the Mr. Fowler’s post. For me the only solution is add my comments in the Copilot code review comments and move on. It’s not the only time Copilot code review provided this kind of comments. The overall tone from Copilot is

Issue two: this one honestly stumbled upon me, and took me a while to understand the rationale of Copilot’s mistake. I asked Copilot to fix an issue of “unnecessary mockito stubbing” during unit testing. Note the code was generated by Copilot, and what it does was: to remove 2 lines of code which is not supposed to remove, while the correct solution is to remove the same 2 lines of code in a similar function. The main difference between the 2 functions here is one fucntion is with togglz off, while the other is with togglz on. It kept doing that for a few times, until I realized I can manually do it by removing those two lines in the correct function. I think in this case Copilot still is helpful to me – but this small incident reinforced my thinking of “trust but verify” for AI generated code.

At the same time, I read a bit of “When AI writes almost all code, what happens to software engineering?” (Substack link, I came across it from LinkedIn) by Pragmatic Programmer aka Gergely Orosz. It sounds like he is more bullish on AI or AI generating code. Note the Substack article was written in last Nov or Dec.

Another related observation is, I notice sometimes colleagues would say: let AI do the algorithm or something like that (the implied meaning seems to me is we are not as smart as AI).

One counter argument is: the big tech companies are the ones leading the charge using AI to generating code, but at the same time their code quality sometimes are horrendous, such as the Uber Driver App – e.g., this personal experience I had when I was delivering for Uber Eats.

Last but not least let me share a short talk I came across from my friend and a professor in management science at my alma mater Huazhong University of Science and Technology, as shown below. In Chinese. I assume you can ask ChatGPT or Google Gemini for translations if needed.

Categories
Software development

I learned a bit more GitHub Copilot recently

Reading Time: 2 minutes

I worked on a small project (in the Agile or JIRA world, it’s a story), and I experimented Copilot more this time, in fact, I think I tried to use Copilot as much or as practically as possible, for 2 out of the 3 main coding tasks. This is in addition to the Copilot code review feature which I have been using probably for a year now. Back to the coding tasks: refactoring and adding unit tests are the two main tasks that I used Copilot extensively. I also think Copilot did a decent job on bath. The 3rd task, which is to create a controller (an end point), I forgot to try the Copilot – hope I can try something like that down the road.

I used the IntelliJ Ultimate edition for my Java project IDE, and I was familiar with some IDE basic features on refactoring, such as extract methods, or change method signature. This mini project involves moving code from one layer down (from UI to the service layer). Copilot did it in one shot when I asked it to perform the task, from and to, mainly using the correct prompt.

For Unit Testing, I did two things, I changed the mocking framework from EasyMock to Mockito (which is more modern), also I added and cleaned all the unit tests, as much as I can. I admit I am still learning on the unit testing front. Sometimes I felt it’s somewhat like “teach an old dog a new trick”, I mean the “test driven development” or writing a complete unit test suite. But I am getting there: not just on Java, for JavaScript, I had some fun writing Jest test as well. For Java it’s JUnit.

Code review

This is similar as before I worked on this mini project, but I am getting more out of the Copilot code review comments, most of which I think is valid.

On a side note, I did a coding test for this LinkedIn AI trainer gig – not sure that I passed it, as I was not very confident on my Python language. But I still think it’s good exercise for me – I mean the coding test via HackerRank.

Categories
Software development

My impressions on Copilot

Reading Time: 2 minutes

Code Review

At my current work place recently we switched from Atlassian’s Bitbucket (stash) to Microsoft’s GitHub for our source code version control system. Along with that, we get the GitHub Action (which is somewhat like Jenkins, the CI/CD tool) as well as Copilot Code Review and other Copilot tools.

Let me talk about Copilot Code Review first. In general I say it’s pretty good, with its quick human readable overview or summary for the code change. I think it’s probably better than I would write myself: if I ever have to write such a summary.

Now, the code review comments. There are various levels of comments in terms of severity. If you have done any code review, you will probably know quite a bit of suggestions or feedback are subjective, because coding is still not exact science, or like some of the other disciplines such as Mechanical Engineering. I will categorize the comments below:

  1. Nitpick: Copilot actually says that word. We can ignore those.
  2. Format or spacing, we can accept the suggestions, and commit directly. That’s probably the only place we can directly commit.
  3. Other suggestions: I think most are valid. And we should try to incorporate them in our coding. For example: things around better error handling, more robust code, efficiency, break out functions from common code (copy and paste code), and so on. This including feedback on the unit testing too. One thing to keep in mind, always run unit test or debug the code changes before we commit, don’t just commit the code change in the web browser. Because I learned the lesson hard way: I commit code directly, then I realized that I broke the CI (continuous integration) build.

Coding Assistant

My own exp: I have done a bit experiment on this, both via the Copilot inside IntelliJ and VS Code. I will continue use Copilot. Somewhat like code review suggestions: coding assistant (or Q&A) is not AI does the coding for us, we still need to think about the usual stuff: design, testing scenario, debugging, and so on. Again we can think copilot as a colleague or a coworker in pair programming, but we always need to verify in addition to the usual “trust”.

I noticed one approach that one coworker was using. Ask Copilot questions, and use the answers directly, and so on. It seems to me this is a bit like a junior engineer’s approach. And I can see one downside of this approach, see below picture, original source: this is a joke but it has some truth.

Vibe Coding (from XHS, or Red Notes)

I have done similar things in the past, except before all the AI tools, I was mostly googling, including using answers from stackoverflow.