It’s easy to get overwhelmed with the explosion of AI tools and applications available nowadays. Pick a word, add “.ai”, and you’ll probably find a new startup in the space. But too many people are using these tools in simple search-like ways. They can do a lot more. Most of the time, at least…
A colleague told me about how she had been at Target, facing a wall full of board and card games, and utilized the AI on her smartphone to figure out which game had the best ratings. It was easy enough: In Perplexity, she took a photo and then asked “which game has the highest ratings and is suitable for teenagers?” Within a few seconds she had winnowed down a shelf with maybe 75 games to 2-3 possibilities, with ratings and descriptions. Smart.
This got me thinking about how we can use all of the capabilities of popular AI tools to our advantage when we’re out and about. Ideally, no typing involved, just taking photos or having them analyze photos from our archives. Just as importantly, which would get the analysis right and which would fall down or “hallucinate” a wrong, but earnestly shared answer?
Shortcuts: Book Identification | Ramen Spices | Cat Breed | Conclusions
For this test, I utilized two very popular AI programs: ChatGPT from OpenAI and Copilot from Microsoft. On the third test I snuck in some of Apple’s behind-the-scenes photo analysis (that pre-dates Apple Intelligence, but we’ll get to that). How’d they fare? Let’s find out!
TEST ONE: TELL ME ABOUT THIS BOOK
I was perusing books at the local public library and saw one featured on the shelf. The title and cover art intrigued me so I decided to ask AI to help me learn more about the book. I tapped to launch the ChatGPT app on my iPhone. The screen looked like this:
If you’ve used ChatGPT on your phone, you probably have immediately tapped in the “Message” field and typed in – or using the microphone icon, spoken – your query. But that “+” link on the left opens up all sorts of possibilities. A tap on it and…
Since I already took a photo of the book in question, I’ll tap on “Attach Photos” to select it from my camera gallery.
It shows up as a thumbnail image and I am then able to enter my related query, as shown:
The key question here is will ChatGPT be able to figure out what book it is from my less than ideal photograph? That’s easily ascertained with a tap on the up arrow “submit” button. The result isn’t so great…
ChatGPT has fallen flat on its digital face in this instance, identifying the wrong book entirely. It didn’t even get the author correct. On the other hand it did glean from the query that it’s about a book, so there’s that.
CAN COPILOT DO A BETTER JOB?
Next up, Microsoft Copilot (via the mobile app), given the same photo and the same basic query:
The results are much more useful and even for the correct book:
The latter portion of the response was intriguing. The app recognized that a fast answer was better than one that took a bit of grinding through online data, so it offered up the paragraph summary and synopsis. I waited a bit for the Goodreads rating to magically be added but it wasn’t. No worries, I just asked for more info:
Not too bad at all, and 4.22/5.0 is a good rating on Goodreads too. One to add to my to-read list, I’d say.
More importantly, how incredibly useful is this for evaluating items in front of you? I could have asked a librarian, but what if they hadn’t read the book? I could look on Amazon (and the Amazon mobile app also has a good photo analysis feature) but then it would have been potentially fake reviews and focused on selling me the book, not sharing the ratings.
Score: ChatGPT: 0. Copilot: 1
TEST TWO: SPICES AT A RAMEN SHOP
This second test came about because I was at a local ramen shop with a steaming bowl of miso ramen in front of me and two spice jars on the table. Which one would be better to add a bit of spice to the bowl? Not knowing either spice, I decided to let AI help me out. A perfect use for the image analysis capabilities, actually. Starting with Copilot…
Notice here I’m being fairly specific about whether they’re good for ramen, rather than just seeking a description of what they are and their intended purposes. Copilot delivers:
I added some of each spice to my ramen. So good!
CHATGPT SPICE ANALYSIS
But what about ChatGPT, how does it fare with this same challenge?
This time it’s mostly a tie because ChatGPT (using the 4o model) does a good job too:
This is a more comprehensive answer, with the Japanese names embedded in the response. There’s also a “Usage in Ramen” section offering more information about how these are traditionally used to enhance the popular Japanese soup. Delicious!
Copilot did well, but ChatGPT did better, as reflected in the result:
Score: ChatGPT: 2. Copilot: 1.
TEST THREE: WHAT KIND OF CAT IS THIS DOG?
With those tests done and my ramen enjoyed, it was time to try something a bit more challenging: Identifying the breed of a cat from a photo. I was also interested in whether I could mislead the software with a bad prompt. Starting with Copilot, here’s what I specified as my query along with the photo:
No snark, no correcting my “What breed of dog is this?” prompt. Very polite, just an answer, but it’s not actually even attempting to identify breed, it’s just describing the image. In fact, a response that’s more suited to a “describe what’s in this image” prompt than something that attempts to answer my query.
How about ChatGPT? Same image, same prompt:
A far superior answer that demonstrates consideration of the query, not just rudimentary analysis of the attached photograph. Odds are good that Cosmo, our cat, is a Russian Blue, though he also has characteristics of a Chartreux, notably eye color. Still, a clear winner here.
Can My iPhone Identify Cat Breed?
Most people don’t know that Apple quietly slipped photo analysis into its Photos app a few years ago and that it can do nifty things like identify the breed of an animal in a photo. I’ll start with the image itself, as shown in Photos:
See the “i” in the circle with the stars icon along the bottom? That doesn’t show up for all photos, but when it does, that means the phone’s identified certain aspects or other information related to the image. A tap on “i” and…
Apple Photos recognized the difference in eye color and how it’s an important factor in breed identification, which is most impressive: Russian Blue cats tend to have green eyes, while Chartreux have yellow, copper, or gold eyes. Cosmo is most likely a Chartreux!
Not only that, but a tap on “Look up Chartreux” and a bit more information is displayed:
More photos to confirm identification, and it offers up both Chartreux and Russian Blue as possibilities, with links to learn more. Better than many cat experts!
Score: ChatGPT: 1. Copilot: 0. Apple Photos: 1.
CONCLUSION
The point of this test was to demonstrate how, just like so-called experts, sometimes a second opinion can yield a better and more informative response. Microsoft Copilot blew OpenAI ChatGPT out of the water with the book analysis, but then ChatGPT roared back with its superior Japanese spice identification. Copilot then fell flat on cat ID, while unexpected contender Apple Photos actually did the best job (though it didn’t offer any explanation of why it thought Chartreux was the most probable breed).
By numeric score, ChatGPT was the winner (I didn’t score Apple Photos for the first two queries because it couldn’t glean any information from the images so had nothing to contribute, nor does it “answer” queries anyway). But ChatGPT failed with the book. The logical conclusion is that if one tool doesn’t offer the results you seek, try another. Add Perplexity to the mix, and maybe Genesis too.
More importantly, try to stump the apps with different types of tasks. Ask it to analyze a file or PDF, extract useful info from a photo, identify a drawing style by artist, or pick out a great board game the next time you’re at the store. Why not? Good luck and let me know what innovative uses you’re finding for your AI apps!
Pro Tip: I’ve been writing tutorials for the Apple iPhone since they were first released. Please check out my extensive iPhone help area for oodles of useful content, then check out my new Generative AI Help for lots of fun AI tutorials too.