ChatGPT couldn't solve this!
I wanted to see if AI would indeed be the thing that ends our careers. Turns out, the servant is really not greater than his master.
Context
So I was recently trying to debug an application. It was an extremely buggy legacy software that kept giving everyone headaches - from the users to the founders. Several developers had been on the project but each left behind huge amounts of unfinished work - as well as more and more bugs. Some of these bugs were so bad that the reports being generated were full of incorrect data.
One of the features of the application was to generate terminal reports of all the students and assign positions/rankings to each subject they were tested on, in order to determine which child was first place, second place, and so on.
For confidentiality, I will not post the original code.
However, while trying to rewrite the ranking mechanism, I ran into an interesting problem and thought, "Hey, let me leave this to ChatGPT to do".
ChatGPT has proved very helpful to me in my coding journey. I began with CoPilot last year but signed up for ChatGPT this year when it launched. If you've followed my blog for a while, you know that I like to explore AI and also explore deep questions about the technology. In fact, last year, I mused about the possibility of AI becoming "infinite" in its capabilities.
CoPilot was great, but it kept generating outdated and incorrect code for me. So I did not continue using it. ChatGPT was even better - with the ability to actually explain things in simple language. However, I never really used ChatGPT for coding until very recently. At first, I gave it simpler queries and it seemed to do very well. But today, I gave it a tougher challenge and it ended up wasting hours of my time. I also noticed that it kept lagging and freezing and I had to refresh the page each time that happened.
The problem
Let's jump right into the code.
The problem is this block of code:
<?php
$students = [
[
"STUDENT_ID" => "STD003420",
"TOTAL_SCORE" => "62.5"
],
[
"STUDENT_ID" => "STD003421",
"TOTAL_SCORE" => "60.0"
],
[
"STUDENT_ID" => "STD003066",
"TOTAL_SCORE" => "65.0"
],
[
"STUDENT_ID" => "STD003064",
"TOTAL_SCORE" => "70.5"
],
[
"STUDENT_ID" => "STD003041",
"TOTAL_SCORE" => "69.0"
],
[
"STUDENT_ID" => "STD003062",
"TOTAL_SCORE" => "74.0"
],
[
"STUDENT_ID" => "STD003425",
"TOTAL_SCORE" => "66.5"
],
[
"STUDENT_ID" => "STD003055",
"TOTAL_SCORE" => "72.5"
],
[
"STUDENT_ID" => "STD003069",
"TOTAL_SCORE" => "74.5"
],
[
"STUDENT_ID" => "STD003068",
"TOTAL_SCORE" => "63.0"
],
[
"STUDENT_ID" => "STD003045",
"TOTAL_SCORE" => "63.0"
],
[
"STUDENT_ID" => "STD003115",
"TOTAL_SCORE" => "57.0"
],
[
"STUDENT_ID" => "STD003427",
"TOTAL_SCORE" => "69.5"
],
[
"STUDENT_ID" => "STD003002",
"TOTAL_SCORE" => "72.5"
],
[
"STUDENT_ID" => "STD003536",
"TOTAL_SCORE" => "60.5"
],
[
"STUDENT_ID" => "STD003122",
"TOTAL_SCORE" => "87.0"
],
[
"STUDENT_ID" => "STD003534",
"TOTAL_SCORE" => "79.0"
],
[
"STUDENT_ID" => "STD003049",
"TOTAL_SCORE" => "77.0"
],
[
"STUDENT_ID" => "STD003121",
"TOTAL_SCORE" => "66.0"
],
[
"STUDENT_ID" => "STD003047",
"TOTAL_SCORE" => "85.0"
],
[
"STUDENT_ID" => "STD003043",
"TOTAL_SCORE" => "86.5"
],
[
"STUDENT_ID" => "STD003185",
"TOTAL_SCORE" => "70.5"
],
[
"STUDENT_ID" => "STD003001",
"TOTAL_SCORE" => "82.5"
],
[
"STUDENT_ID" => "STD003003",
"TOTAL_SCORE" => "86.5"
],
[
"STUDENT_ID" => "STD003538",
"TOTAL_SCORE" => "70.0"
],
[
"STUDENT_ID" => "STD003537",
"TOTAL_SCORE" => "70.0"
],
[
"STUDENT_ID" => "STD003535",
"TOTAL_SCORE" => "80.0"
],
[
"STUDENT_ID" => "STD003053",
"TOTAL_SCORE" => "73.5"
],
[
"STUDENT_ID" => "STD003120",
"TOTAL_SCORE" => "77.0"
],
[
"STUDENT_ID" => "STD003059",
"TOTAL_SCORE" => "70.0"
],
[
"STUDENT_ID" => "STD003426",
"TOTAL_SCORE" => "61.0"
],
[
"STUDENT_ID" => "STD003424",
"TOTAL_SCORE" => "60.0"
],
[
"STUDENT_ID" => "STD003422",
"TOTAL_SCORE" => "50.0"
],
[
"STUDENT_ID" => "STD003118",
"TOTAL_SCORE" => "68.0"
]
];
usort($students, function($a, $b) {
return $b['TOTAL SCORE'] - $a['TOTAL SCORE'];
});
var_dump($students);
?>
I wished to sort this class of 34 students based on their total scores, in descending order. The easiest way, of course, was to pass a callback to PHP's usort() method and sit back.
When I ran it, however, it seemed to have trouble getting the first value correct. I already know how usort() works under the hood, so that's not the issue. And you can see that my callback is correct. But instead of returning 87.0 as the correct answer, it returned 86.5.
This was puzzling, because 86.5 can never be greater than 87.0.
So I decided to ask our new best friend - ChatGPT.
Skipping to the response...
Alright, so I decided to copy the new code and use it.
As I expected, it did not work. So I went back to ChatGPT to complain about it, and even pasted the new code in there.
And just like that, it left me hanging:
But I still needed an answer and so I regenerated the response:
But interestingly, the answer it got was also not correct. Here's a snippet of it:
Looool... how on earth is 85.0 greater than 86.5?
Someone may say, "but why are you comparing strings?" Well, it turns out that if you compare these same strings "85.0" and "86.5" in PHP, you get the right answer.
So it's not about the usage of strings at all.
I then complained once more and this time, ChatGPT was just so overwhelmed that it gave up midway. (And even got the programming language wrong.)
To anyone who may be confused, the reason why the sorting didn't work is that the usort() method, after performing the subtraction, converts the result to an integer. And if the difference is something as small as -0.5, it assumes the result is 0, and so it sees both 87 and 86.5 as being equivalent. The solution, of course, is to use a different sorting method. I expected ChatGPT to notice this and warn me, but it failed to do so.
Conclusion
Is AI going to replace programmers? Not anytime soon.🥱