Table of contents
AI coding assistants have become an integral part of developers' daily workflow. In this test, we built a small web app from scratch with each assistant, following the same steps. All of them produced fast and accurate results for simple functions, but clear differences emerged in complex business logic and debugging. Noticing that some suggestions contained insecure code showed that human oversight is still essential. Software developer Ryan Douglas says the healthiest approach is to treat the assistants like an intern and review every line.
The test project and the rules
Our test project was a small web app with user login, a task list and a simple reporting screen. We gave every assistant the same written task description and started each one in the same empty folder. We followed a twenty-two-step plan in order, running the code and recording the result at the end of each step. A reference version we wrote by hand served as the yardstick for comparison.
Our rules were simple: we could fix the assistant's code ourselves only if there was a build error; for logic errors, we had to ask the assistant. We measured the time taken, the number of fixes requested and the number of working features at the end. Software developer Ryan Douglas acted as referee, reviewing the code and scoring it for security and readability. That way, the results didn't hinge on a single person's opinion, and the criteria were open to everyone from the start.

The assistant writes fast, but the responsibility still lies with the developer.
Sophie Laurent
Speed and code quality
For simple functions such as data validation, date formatting or sorting a list, all of the assistants wrote correct code in seconds. The user interface skeleton was similarly quick. The differences emerged as the business logic got more complex: on queries joining multiple tables, two assistants produced the wrong result on the first try. The gap became more pronounced as the project grew.
During debugging, the assistants that took in the full error message together with the relevant code snippet performed best. On code quality, we saw differences in variable naming and comments; one assistant produced needlessly long, repetitive code, while another wrote short but hard-to-follow solutions. On average, the assistants let us finish the project 40 percent faster than the reference version we wrote by hand.
Watch out for security flaws
Our most striking finding came on the security side. Two assistants suggested code that inserted user input into a query without validating it, opening the door to a classic database vulnerability. Another suggestion stored passwords in plain text. These are bugs that are hard to spot at first glance because the code appears to work. A careful eye can catch them, but someone in a hurry can easily miss them.
| Topic | Software |
|---|---|
| Highlights | AI, comparison, review |
| Reading time | 8 minutes |
| Written by | Sophie Laurent |
Key takeaways
- Different coding assistants were tested with the same task and the same steps.
- All are fast at simple tasks; differences are clear in complex logic.
- Some suggestions contained security flaws, so human review is essential.
Ryan Douglas's advice is clear: treat the assistant like a capable intern, read every line it produces, run security scans automatically and keep your tests independent of the code the assistant writes. It's also important not to paste secret keys or customer data into the assistant. In short, the tool saves time, but it doesn't take over responsibility; the final call and oversight still rest with the developer.
What do you think about this post?



































































Leave a comment