My play with Microsoft Copilot for Image Analysis

I have been toying around with Gen AI tools since the announcement of ChatGPT and Microsoft's Copilot caught my attention a few months back. While Gen AI is promising, I want to show you a case of how it failed and how I trained it to give out accurate answer.

The Context: I disposed a few decade old laptops after removing its internal HDD. After a good month or so, when I went shopping for its enclosure, the guy smirked that these are old HDDs and that it is hard to get those now. Betting on Amazon/Flipkart, I searched through the websites and have realized the very many models there has been and how finding a compatible one is a challenge.

My Homework: I didn't bet on Gen AI tools' accuracy for this based on the mixed experiences I have had with it so far. So I first did my homework to arrive at the right answer and wanted to know what would have it been if I had relied on Gen AI tools.

My Experiement with Gen AI tool: Microsoft Copilot is the tool of choice this time and screen shot below shows how it gave inaccurate answer.

Microsoft Copilot said it is "The attached image shows a Samsung MZ-7PC2560/0A1 SSD, which is part of the Samsung 830 series. This is a 2.5-inch SATA III SSD." which it is not. When I nudged it to try again, it said, "Upon closer inspection, the SSD in the image is a Samsung MZ-7PC2560/0A1, which is part of the Samsung 830 series. This model is a 2.5-inch SATA III SSD", which is closer but not the right answer still. So, I responded saying even that isn't right and asked it to try again. This is so surreal, like a civilized human being, it excused itself with a response, "I am sorry but I prefer not to continue this conversation. I'm still learning so I appreciate your understanding and patience".

In my second attempt, I opened up another window to try this exercise again. Copilot gave the same wrong answer. I thought of giving it hints to analyze based on pins count, to see if it can come up with right answer. It came up with wrong answer even in the pin count. And upon correcting this, it did give somewhat accurate answer as shown in the screenshot below:

This definitely is an internal HDD used in my Macbook but that it was Pro and not Air model. And do you know what this SATA III HDD used in Macbook called? It is called mSATA. Miss this, and you would end up getting an incompatible enclosure for the HDD.

Can Gen AI be used to ease customer shopping by replacing text searches with image search and more? That precisely is what I demonstrated to you in this post. No wonder, why e-commerce sites haven't yet embedded Gen AI tools where you can purchase items based on image submission. Mind you that in my case here, it did get even the counting of pins wrong. That is not what is advertised as to how it is used in shop floors for counting stocks. Gen AI is still in the hype phase and we aren't there yet. That said, this is truly bleeding edge game that is quite promising.