ABOUT THIS EPISODE
Who won Super Bowl 13?
I understand that American football—and by extension, the Super Bowl—is a popular subject and that this seems like a simple question, however, I refuse to believe this kind of question represents how most people use or expect to use Siri, aside from testing its limits for the sole purpose of journalism specific to critiquing Siri.
Who won the 2004 North Dakota high school boys’ state basketball championship?
No matter how specific any one of these questions may be, there are an infinite amount of them. There is no limit to the questions that one could ask virtual assistants about facts and statistics. I reject the premise that this is even a competition Apple needs to be a part of. I don’t care if other “AI” chatbots get these right more often, because this problem will never be 100% solved anyway. And we have to recognize that.
While it is reasonable to expect Siri could support specific historical statistics like Super Bowl winners—even if Apple hard-codes the answers—it is not reasonable to expect data accuracy for every question of the sort. Because everything is actually a lot.
Today, it’s who won Super Bowl XIII. But if it gets that right, then tomorrow, it’s who was MVP of Super Bowl XIII.
The first search result for “mvp super bowl” on DuckDuckGo has all the answers to exactly that question. No one would actually need to ask Siri, but if they do, Siri should not be making guesses.
When asked trivia that can’t be answered by Apple’s own service integrations or partnerships with database owners, Siri should simply not answer the question and perform the search for you instead. That’s what it used to do. That’s what it should do.
Or you could just perform the search yourself, instead of placing trust in a service that has been an unreliable source of this kind of information since 2011.
There are databases that contain these kinds of statistics that Apple could partner with, but unfortunately, there are not databases that comprehensively contain every statistic about everything a human could ever ask. So we should probably stop judging Siri, virtual assistants, and LLMs by this forever-goalpost-moving metric.
At what point do people expect it to answer who the winning quarterback was for the Super Bowl? The losing one? What about which team never won the Super Bowl? Which player has played in the most Super Bowls?
Instead of listing every Super Bowl question I can think of, it’s easier to admit that any attempt made to answer every theoretical question will be flawed.
Even with increased accuracy in some areas, it’s impossible to know how reliable it actually is about everything else. If Siri one day answers the Super Bowl question correctly every time for every game, that will imply it can accurately answer similar questions, too.
But it won’t. And we won’t know whether it’s right or wrong without double-checking the answer anyway. So instead of relying on Siri for this kind of data, just… don’t ask for it.
For what it’s worth, Siri has never been 100% reliable, and we all know that. But because there are several apps and features that Apple makes that are effectively 100% reliable, I understand how the expectation exists.
Siri will never be 100% reliable for this kind of request.
I don’t pretend to represent how most people use Siri, but—for contrast—here’s what I usually ask of my HomePods:
Play music I like.
Where’s my phone?
What’s the weather today?
These requests are based on the faith I have in specific service integration. I know my HomePods are connected to the Internet, Apple Music, Find My, and Apple Weather. I can somewhat confidently make these kinds of requests based on that understanding.
But no one really knows all of Siri’s integrations, right? Even if someone thinks they do, there still are limitations for what we can request from those services.
I can ask where my phone is. I can ask where my friend is. Both work because I understand there’s Find My integration. But before heading out this morning, I used that faith to ask my HomePod a question I was less sure it could answer.
What’s the battery percentage of my MacBook?
Siri could not answer this question.
I know that isn’t an Apple Intelligence request, and that’s what today’s criticism is focused on, but I think being unable to answer that question is a much more damning critique of Siri than who won a Super Bowl.
My faith in that question is rooted in specific knowledge that battery level data is visible in the Find My app. I assumed—wrongly—that since battery percentage was visible in the app, it would be available through a Siri request.
No one understands the bounds of reasonable requests to make of Siri. That’s what makes the Super Bowl question seem reasonable to some people.
Forget Siri for a moment. Can you get this historical Super Bowl data in Apple’s own Sports app? Not a chance, right? No critique about that. Why do we expect more of Siri?
Instead of expecting Apple to provide answers for those types of questions with Siri, I would rather Apple spend the effort to provide exact parity between the visual interface (apps) and voice interface (Siri).
If you can’t do it in an Apple app, Siri shouldn’t either. But if you can do it in an app, then Siri should absolutely support it. That’s something we can expect. That’s something that can be 100% reliable.
If you like this, you can make a one-time donation, donate monthly, or buy something from my shop.
