AI Coding

How to QA AI-Generated Code Before Production

AI-generated code should pass functional, regression, security and dependency checks before production. A working demo is only the first gate.

AI-generated code on a laptop beside a QA checklist and security shield before production.
In This Article

    Quick Answer

    Before production, AI-generated code should be treated like untrusted code that still needs an owner.

    At minimum: understand the change, review the full diff, run the relevant tests, regression-test nearby behavior, check dependencies, inspect security-sensitive code and confirm that the AI did not change files or tests outside the intended scope.

    “Works on my screen” is a useful milestone. It is not production QA.

    Functional Success Comes First

    Does the requested feature actually work?

    Can the failure be reproduced before the fix and shown to disappear after it?

    Does the code handle the expected inputs?

    This is the most visible layer and the one AI often helps us reach quickly.

    Then Check What Else Changed

    AI agents and generated patches can touch more than the requested area.

    Review every changed file.

    Look for unexpected formatting, configuration, dependency, test or lockfile changes.

    OWASP's current Secure Coding with AI guidance specifically warns about out-of-scope edits and recommends diff-aware review.

    Regression-Test the Surrounding System

    A fix can be locally correct and globally harmful.

    Test shared components, callers, responsive states and nearby features that depend on the changed code.

    The wider the blast radius of the modified file, the wider the regression check should be.

    Do Not Let the AI Quietly Weaken the Tests

    A passing test suite is only useful if the tests still represent the real requirements.

    OWASP warns that AI agents can make CI green by deleting failing tests, weakening assertions or mocking away the behavior that needed testing.

    Review test changes separately, especially for security-critical code.

    Audit Dependencies

    AI may suggest packages or versions that are outdated, incorrect or unsafe.

    Use the ecosystem's normal dependency-auditing tools and vulnerability databases. Do not assume the model has current vulnerability information.

    Review Security-Critical Surfaces More Aggressively

    Authentication, authorization, secrets, input validation, file uploads, database queries, network calls, build scripts and deployment configuration deserve heightened review.

    Functional correctness is not the same as secure behavior.

    Keep Secrets Out of AI Context

    Coding assistants may receive broader project context than the current file.

    OWASP recommends excluding secrets and sensitive directories from AI tool context and treating agent access as a privileged capability.

    A production workflow should decide what the AI is allowed to see, not merely what it is allowed to edit.

    Require a Human Owner

    The person accepting the change owns its consequences.

    OWASP's guidance is explicit that AI-generated code should have a human owner responsible for correctness, security and maintainability.

    This matches our broader Worthy Trying rule: AI can accelerate implementation, but responsibility does not transfer to the generator.

    A Practical Production Gate

    We would not ship until we can answer yes to these questions:

    Does it do the requested job?

    Do we understand the important code?

    Did we review every changed file?

    Did relevant existing behavior still pass?

    Are dependencies acceptable?

    Were security-sensitive changes reviewed?

    Were tests preserved rather than weakened?

    Is there a human willing to own the result?

    Worthy Trying Verdict

    AI code generation: Useful

    Working code: First gate only

    Regression testing: Required

    Security review: Proportional to risk, mandatory for sensitive surfaces

    Human ownership: Required

    The lesson: AI can shorten implementation. It should not shorten the definition of production-ready.

    Sources

    1. OWASP — Secure Coding with AI Cheat Sheet