State of AI code quality: what we find in real generated codebases

An ongoing study built from paid audits of applications generated with AI builders. Method first, findings second, limitations stated — the opposite order to most vendor research.

Custom software SaaS products AI development Web applications AI chatbots Nearshore teams

Method

How the study works

Published before any finding, because a study without a method is an advertisement.

Rules we hold ourselves to

Sample. Every paid audit we run on an AI-generated codebase, not a selected subset. We state the count in each edition.

What counts as a defect. A deviation that would cause data exposure, data loss, incorrect billing or an unrecoverable environment. Style is not a defect.

Who assesses. The senior engineer who ran the audit, against a fixed checklist, with a second reviewer on anything critical.

Excluded. Codebases we only saw partially, and prototypes explicitly built as throwaway demos.

Cadence. Updated every six months. A stale study is worse than none.

Edition

First edition in preparation

Basis

Paid audits, full sample

Next update

Every six months

Findings

Defect classes we keep seeing

Reported as classes rather than vendor league tables: the point is what to check, not whose generator to blame.

Access control

Row-level security missing or advisory; authorisation checks implemented in the client while the endpoint stays open. The most common critical class.

Secrets and configuration

API keys shipped to the browser; credentials committed; a single environment edited in production.

Data model and migrations

Schema existing only in a hosted console, so no environment can be rebuilt and no change can be reviewed.

Money paths

Payments left in test mode, or live without webhook handling, so paying customers are not reliably provisioned.

Five checks a founder can run in an hour

1. Log in as user A, change an ID in the URL, and see if you can read user B's data.
2. Open the browser devtools network tab and look for API keys in responses or bundles.
3. Ask where the database schema lives and whether a fresh environment can be created from it.
4. Make a test purchase and confirm entitlement is granted by a webhook, not by the browser.
5. Upload a file with a double extension and see what the server does with it.

If any of these fails, the fix is cheaper now than after your first real users arrive.

Limitations

What this study cannot tell you

Stated plainly, because a study that hides its limits is marketing.

Our sample is self-selected: people commission an audit when they already suspect a problem, so defect rates here are almost certainly higher than in the general population of generated apps.

We do not rank generators. Tooling changes monthly, our per-tool counts are small, and a league table would be the most quotable and least honest thing on this page.

Nothing here says AI builders are bad. They compress the first 70% of a product remarkably well. The findings are about the last 30% — which is exactly what our rescue service exists for.

Industries

Industries we know

We build with the domain in mind — from fintech and retail to manufacturing and healthcare.

18 sectors

IT companies and digital agencies
Insurers and brokers
Lenders and microfinance
E-commerce
Wholesale and distribution
Manufacturing
Freight and forwarding
Clinics and medical centres
Dental
Car dealers and service centres
EdTech
Legal and consulting firms
Equipment rental
Real estate agencies
Property developers
Travel and events
Beauty and wellness
Construction and renovation

FAQ

Questions about the study

Including the one every founder asks first.

Is it safe to build a product with an AI builder?

For a prototype, yes, and often the fastest path to learning. For anything handling payments, personal data or real users, get the security and data model reviewed before launch.

What should we fix first?

Access control, then secrets, then migrations. In that order, because the first one is the class that exposes other people's data.

How much does it cost to put it right?

Audit $1,790 for five days; fix-up from $5,900 over two to three weeks. Both fixed price.

Do we need to rewrite from scratch?

Usually not. The exception is a data model that is wrong at the root — then rebuilding on the learnings is cheaper than the archaeology, and the audit will say so.