Known and Unknowns: Testing AI Generated Code

**Abstract**

Let's face it AI-assisted coding is rapidly becoming the primary way applications are being written. The power of being able to write out a complicated application or feature with a just a few commands is something too alluring to pass up.

But between the imprecise nature of language and LL

Ms ability to "hallucinate", this creates many opportunities for bugs and unintended behavior to introduced. Just telling an AI coding agent to "write tests" isn't going to be enough to build confidence in the correctness of an application either.

In this presentation we are going to review three distinct phases to testing AI written code. The known knowns; using spec driven development to define the business meaningful behavior of your application. Known unknowns; using property-based testing to handle edge cases and avoid over fitting and unintended biases. And finally unknown unknowns, using static analysis and mutation testing to avoid introducing bugs you didn't think about.

AI-a