Why AI-Generated Code Still Needs a Human Security Review

In September 1947, operators on Harvard's Mark II pulled a moth out of Relay #70 and taped it into the logbook beside a note calling it the first actual case of a bug being found. The story lasts partly because the failure was visible. You could see the insect, remove it, and watch the machine work again.

Modern failures rarely offer that courtesy, and AI coding assistants produce some of the least visible ones. A model can write a login handler, a database query, and a file upload endpoint in seconds. Each can run cleanly, pass your tests, and still leave a door open. Working code feels finished, and finished code stops getting read.

That is why AI-generated code needs a human security review. Review has not become optional. It is the one place where whole classes of mistakes still get caught. This guide shows what to check and how to run a focused review in about half an hour.

Is AI-Generated Code Secure Enough to Ship Without Review?

No, not by default. Veracode's 2026 GenAI Code Security Report tested more than 100 models and found an average security pass rate of about 56 percent, even though the code almost always compiled. Roughly 44 percent of tasks produced code with a detectable vulnerability, and cross-site scripting fared worst, with a pass rate near 15 percent. Veracode sells security tools and tested a selected set of tasks, so read these numbers as a warning about the trend, not as a defect rate for your own codebase.

The practical lesson holds either way: treat generated code like a pull request from a fast contributor you have never met. Six areas deserve a human look every time: authentication, input handling, database queries, file uploads, secrets and dependencies.

Why Models Write Plausible Code, Not Verified Code

A language model writes code by predicting what a competent developer would most likely type next. That explains why the output reads so fluently. It also explains the failure mode. Plausibility comes from training data, not from facts about your system, and that data is full of tutorial code written for clarity instead of for hostile users.

The bigger gap is context. The model does not know which endpoints face the open internet, which table holds regulated data, or what your threat model is. It writes to the shape of the request, and the request rarely carries the facts that decide whether a design is safe. Teams that want to vibe code safely tend to land on the same habit: treat model output as a draft from a fast, confident contributor who has no context, and review it that way.

Authentication Code That Looks Complete but Is Not

Auth code has a recognizable shape and an unforgiving substance. Ask for a protected route, and you get a middleware function, a token check, and a redirect on failure. Read it closely, and the gaps appear. The handler may decode a token without verifying its signature, accept a token issued for a different service, or conflate authentication with authorization, allowing any valid user to access any record.

The IETF's Best Current Practice for OAuth 2.0 Security spends most of its length on details like these. It requires exact string matching when servers compare redirect URIs, it forbids open redirectors that forward users to arbitrary addresses, and it requires public clients to use PKCE to block authorization code injection. None of that shows up in a passing test suite. All of it shows up to a reviewer who knows where to look.

Ask these questions on every auth path:

• Does the code verify the token signature, expiry, and audience?

• Does every route check authorization, not just login?

• Does any endpoint trust a user ID sent by the client?

Input That Nobody Ever Distrusted

Input validation fails differently, because generated code usually does something, and that something is close enough to correct to survive a skim. A comment form stores what it receives and shows it to every later visitor. MDN's guide to cross-site scripting states the rule: what is safe in one document context is unsafe in another. Its example makes the point well. Even when a templating engine encodes angle brackets and quotes, an unquoted attribute placeholder can still be exploited.

A model reaches for innerHTML when the surrounding code implies the developer wants raw markup, and it does not stop to ask whether an attacker can control that string. A reviewer traces the value backward until they find its source. That trace is work the generator never performed.

• Search the diff for innerHTML, dangerouslySetInnerHTML, and template filters that switch off encoding.

• Use output encoding by default, a reputable sanitizer when you need raw HTML, and a Content Security Policy as a backup.

Queries Assembled From Strings

Ask a model to filter orders by customer name, or to sort a table by whichever column the user clicks, and it will often paste request values straight into the SQL string, because a great deal of older example code did exactly that. The query returns the right rows for every honest input and hands the database to anyone who types a quote mark.

OWASP's SQL Injection Prevention Cheat Sheet puts prepared statements with parameterized queries first: the database treats values as data, not as syntax. It also names the edge that generators tend to miss. Bind variables cannot stand in for table names, column names, or sort indicators such as ASC and DESC. So a model asked for a sortable list may parameterize the WHERE clause and then append the sort column directly. A reviewer checks that the requested column maps to a fixed allow-list in code:

const SORTABLE = { name: "customer_name", date: "created_at" };
const column = SORTABLE[req.query.sort] ?? "created_at";

File Uploads That Trust the Filename

Generated upload handlers tend to trust everything the browser sends. A typical one checks the Content-Type header, preserves the original filename, and writes the file to a public folder so it can be served later. Each choice is convenient, and each one hands a security decision to the person uploading.

The OWASP File Upload Cheat Sheet is direct about it. The Content-Type header can be spoofed, so do not rely on it. Allow only the extensions the business needs, replace user filenames with names your application generates, and store files outside the webroot where possible. A model rarely volunteers these steps, because the request said "let users upload a profile picture" and an upload that accepts a picture is all the request described. A reviewer asks what happens when a script with a convincing name and a forged header arrives.

Secrets and Dependencies: The Quiet Two

Secrets are the most embarrassing category and the easiest to reproduce. Configuration examples across the web are stuffed with inline API keys, so generated setup code can arrive with the habit baked in. A hardcoded credential in a committed file is not something you patch quietly later. Once it lands in git history, treat it as disclosed and rotate it.

Dependencies are quieter still. A model suggests a package because it appears often in similar code, not because anybody checked its maintenance status or its dependency tree. Models can also suggest package names that do not exist, and attackers can register those names. Verify that every new package is real and maintained before you install it. Then run npm audit on every generated project. It asks the registry for known vulnerabilities, so it reports only what is already published.

What to Check: A Quick Reference

A 30-Minute Human Security Review Checklist

1. Read the auth path (5 minutes). Follow one request from the browser to the data and note every check along the way.

2. Trace untrusted input (7 minutes). Find each place user data enters, and follow it to where it is stored or displayed.

3. Inspect every query (5 minutes). Look for string concatenation, then check sort and filter columns against allow-lists.

4. Check file handling (4 minutes). Confirm extension limits, generated filenames, size limits and storage outside the webroot.

5. Search the diff for secrets (4 minutes). Look for keys, tokens, passwords and committed environment files, then run a secrets scanner.

6. Review new dependencies (5 minutes). Confirm each package exists, is maintained, and passes the audit.

What a Human Reviewer Adds That a Model Cannot

None of this argues against generating code. The speed is real, teams ship with it, and going back to hand-typed routes would be a strange response to a tool this useful. The argument is narrower: generation and verification are different jobs, and one system cannot reliably do both, because the context that would catch the mistake is exactly what the generator lacks.

Review is where a person supplies that context. They know which surfaces face the internet, that this table holds payment data and that one holds cached avatars, and that an endpoint has no business trusting a client-supplied user ID. Picture a team that asks an assistant for a "delete invoice" endpoint. It works in every test. A reviewer notices that it never checks whether the invoice belongs to the person calling it. No test failed, and the hole was still there.

Common Mistakes When Reviewing AI-Generated Code

• Treating passing tests as proof of safety. Tests check what you thought of, not what an attacker will try.

• Asking the model to review its own code and stopping there. A model shares its own blind spots. Use its review as a head start, not a verdict.

• Reviewing only the new files. Check the sibling routes, too, because the same check is often missing from a near twin.

• Deleting a leaked key instead of rotating it. Git history keeps the old value.

Final Thoughts

The Mark II moth was taped into a logbook because someone went looking for the cause of a fault that had already announced itself. Vulnerabilities in generated code do not announce themselves. They sit inside features that work, in routes that return the right data to the right person during every test anyone thought to write, and they wait for someone who is testing something else entirely.

So read the auth path. Trace the untrusted input. Search the diff for keys. Check what the package manager pulled in. It is an unglamorous half hour, and it decides whether the code you shipped this morning was fast work or expensive work.

Frequently Asked Questions

Q: Is AI-generated code secure?

Not by default. Models produce working code far more reliably than secure code, so you should review and test generated code before it reaches production data or real users.

Q: Can an AI tool review its own code?

It can catch some problems, and it makes a useful first pass. It still shares the blind spots of the model that wrote the code, so pair it with scanners and a human reviewer.

Q: How long does a security review of generated code take?

For a typical feature, about 30 minutes of focused reading catches the common gaps. Features that handle payments, health data, or admin access need a deeper review.