Do voice assistants really understand us? We tested six of them
From idioms to regional accents, from complex commands to casual chat, we put six voice assistants through a language test. The results are in our video.
Table of contents
Until a few years ago, voice assistants struggled badly with anything beyond simple, clearly spoken commands. Advances in AI models changed that picture in no time. We tested six assistants with fifty commands we use often in daily life, including idioms, nested requests and speakers from different regions. Some assistants were surprisingly good, while others stumbled over a simple request to set an alarm.
Our test scenarios
We built our test scenarios in five categories: single-step commands, nested requests, Q&A, casual conversation and context tracking. We prepared fifty commands in total and repeated each one to the assistants in two different rooms at two different volumes. We read some of the commands in a studio setting and some in a noisy living room with the TV on.
We also put together a team for speaker diversity. Four people, women and men, young and older, with Southern, Northern and Scottish accents read the same commands. We scored each attempt as correct, partly correct or wrong. That let us judge whether the assistant understood not just the words but the intent. Two editors scored separately, and any differing results were settled by listening together.

When we told the assistant to 'spill the beans,' it suggested a bean recipe.
Mark Ellison
Idioms and regional accents
Idioms and regional accents were the most fun but toughest part of the test. With idioms like “spill the beans,” most assistants took the words literally rather than for their meaning. Only two assistants understood that “I'm at the end of my rope” was a complaint. The success rate dropped noticeably for commands spoken with a Scottish accent, while the gap was smaller for Southern accents.
With nested requests, the picture was better. Four of the six assistants handled two-step commands like “set an alarm for seven tomorrow morning and tell me what the weather will be like” without a hitch. The best assistant also led on context tracking, remembering the previous sentence and correctly answering “what about the day after?” The results show clear progress compared with last year's data.
Mind the privacy settings
Privacy settings are often overlooked when using voice assistants. Assistants keep the microphone listening constantly for the “wake word,” and some store recordings in the cloud. In settings, you can view your recording history, shorten the auto-delete period and turn off voice purchasing. On most devices, this takes just a few minutes. Remembering that the assistant may also record guests' voices, you can also make a habit of using the mute button when people come over.
| Topic | AI |
|---|---|
| Highlights | artificial intelligence, smart home, review |
| Reading time | 6 minutes |
| Written by | Mark Ellison |
Key takeaways
- Six assistants were tested with fifty commands and speakers with different accents.
- Assistants struggled with idioms but did well on two-step commands.
- Deleting recording history and turning off voice purchasing strengthen privacy.
A separate word of caution for families with children: the assistant's answers may not always be age-appropriate. Turning on a child profile and content filter is a good start. You may also prefer to keep the smart speaker in a central room, away from private spaces like bedrooms. However advanced AI becomes, you're still the one in control of the microphone.
What do you think about this post?




































































Leave a comment