AI-Powered Testing — Let AI Write Your Tests

Nobody likes writing tests. But everybody likes having them. AI coding tools have gotten surprisingly good at generating tests — and surprisingly bad at certain parts of it.

The teams winning with AI in 2026 are not generating the most code. They are building processes to ship reliable code despite the fact that AI introduces roughly 1.7 times more bugs than humans, according to recent studies. Good test generation is a critical part of that process.

If you need background on AI test generation basics, start with AI Test Generation. This article focuses on the workflow — how to generate tests that actually catch bugs, how to review them, and how to avoid the traps.

The AI Testing Workflow

AI test generation is not “press a button, get tests.” It is a four-step process. Skip any step and you end up with tests that pass but test nothing.

Step 1: Generate — Ask AI to create tests for your code.

Step 2: Review — Read every test. Check that it tests behavior, not implementation.

Step 3: Run — Execute the tests. Fix any that fail due to setup issues.

Step 4: Iterate — Ask AI to cover the edge cases it missed. Add cases for error paths.

This loop usually takes two or three rounds before you have solid test coverage.

Generating Unit Tests That Actually Work

The quality of AI-generated tests depends almost entirely on your prompt. Here is the difference between a bad and good test generation prompt.

Bad prompt:

Write tests for userService.ts

Good prompt:

Write unit tests for the createUser function in
src/services/userService.ts.

Context:
- We use Vitest as our test framework
- We use Prisma for database access
- The function validates input, hashes the password,
  and saves to the database
- Mock the Prisma client, do not use a real database

Test these scenarios:
1. Successful user creation with valid input
2. Missing required fields (email, password, name)
3. Invalid email format
4. Password too short (minimum 8 characters)
5. Duplicate email (Prisma unique constraint error)
6. Database connection failure

For each test, assert the return value AND verify
which Prisma methods were called.

The second prompt tells AI exactly which scenarios to cover, which framework to use, and what to assert. The first prompt lets AI guess — and it will miss the error cases.

Real output from a structured prompt:

import { describe, it, expect, vi, beforeEach } from 'vitest';
import { createUser } from '../services/userService';
import { prisma } from '../lib/prisma';
import bcrypt from 'bcrypt';

vi.mock('../lib/prisma');
vi.mock('bcrypt');

describe('createUser', () => {
  beforeEach(() => {
    vi.clearAllMocks();
  });

  it('creates a user with hashed password', async () => {
    const mockUser = {
      id: '1',
      email: 'alex@example.com',
      name: 'Alex',
      createdAt: new Date(),
    };

    vi.mocked(bcrypt.hash).mockResolvedValue('hashed_password');
    vi.mocked(prisma.user.create).mockResolvedValue(mockUser);

    const result = await createUser({
      email: 'alex@example.com',
      password: 'securepass123',
      name: 'Alex',
    });

    expect(result).toEqual(mockUser);
    expect(bcrypt.hash).toHaveBeenCalledWith('securepass123', 10);
    expect(prisma.user.create).toHaveBeenCalledWith({
      data: {
        email: 'alex@example.com',
        passwordHash: 'hashed_password',
        name: 'Alex',
      },
    });
  });

  it('throws on duplicate email', async () => {
    vi.mocked(bcrypt.hash).mockResolvedValue('hashed_password');
    vi.mocked(prisma.user.create).mockRejectedValue({
      code: 'P2002',
      meta: { target: ['email'] },
    });

    await expect(
      createUser({
        email: 'existing@example.com',
        password: 'securepass123',
        name: 'Alex',
      })
    ).rejects.toThrow('Email already exists');
  });

  // ... more test cases
});

This is solid output. The AI correctly mocked the dependencies, tested both success and error paths, and asserted the function arguments — not just the return value.

The Weak Assertion Problem

The most dangerous trap in AI-generated tests is weak assertions. The test passes, the coverage number goes up, but the test does not actually verify anything meaningful.

Weak assertion (bad):

it('creates a user', async () => {
  const result = await createUser(validInput);
  expect(result).toBeDefined();
});

This test passes even if createUser returns the wrong data. It only checks that something came back.

Strong assertion (good):

it('creates a user with correct data', async () => {
  const result = await createUser(validInput);
  expect(result.email).toBe('alex@example.com');
  expect(result.name).toBe('Alex');
  expect(result.passwordHash).toBeUndefined(); // should not expose hash
  expect(prisma.user.create).toHaveBeenCalledTimes(1);
});

When reviewing AI-generated tests, search for these weak patterns:

expect(result).toBeDefined() — proves nothing
expect(result).toBeTruthy() — almost always too weak
expect(result).not.toBeNull() — usually insufficient
No assertion on mock function calls — the test does not verify the function did the right work

Prompt to fix weak assertions:

Review these tests and strengthen the assertions.
Replace toBeDefined() and toBeTruthy() with specific
value checks. Add assertions for mock function call
arguments, not just call counts.

Integration Tests — Harder but Worth It

AI generates decent unit tests. Integration tests are trickier because they involve real dependencies, database state, and service boundaries.

Common AI mistakes with integration tests:

Mocking things that should not be mocked (the database in an integration test defeats the purpose)
Assuming services or containers are running
Not cleaning up test data between tests
Creating dependencies between test cases (test B depends on data from test A)

Good prompt for integration tests:

Write integration tests for the POST /api/users endpoint.

Context:
- We have a test database configured in .env.test
- Tests should use real database calls, not mocks
- Each test should clean up its data in afterEach
- The test server setup is in tests/setup.ts

Test these scenarios:
1. Create a user with valid data — check the database
2. Reject invalid email — check no record was created
3. Reject duplicate email — create one user first, then
   try to create another with the same email
4. Reject missing fields — test each required field

The key instruction is “use real database calls, not mocks.” Without this, AI defaults to mocking everything, turning your integration test into another unit test.

Test-Driven Vibe Coding

One of the most effective patterns is writing tests first and then letting AI implement the code to pass them.

Step 1: Write the tests (with AI help):

Write test cases for a password strength validator.
The function takes a password string and returns
{ strong: boolean, issues: string[] }.

Requirements:
- Minimum 8 characters
- At least one uppercase letter
- At least one lowercase letter
- At least one number
- At least one special character (!@#$%^&*)
- Cannot contain the username
- Cannot be in the common passwords list

Write tests for all passing and failing scenarios.
Do NOT implement the function yet.

Step 2: Implement the code (with AI):

Now implement the validatePassword function in
src/utils/password.ts that passes all these tests.
Run the tests after implementing to verify.

This approach works well because:

The tests define the exact requirements
The AI has clear success criteria (tests pass)
You review the tests first, which are easier to understand
The implementation is verifiable immediately

Claude Code is especially good at this because it can run the tests after implementing the code and iterate if any fail.

Edge Cases AI Misses

AI is good at testing the happy path. It consistently misses certain categories of edge cases. After generating tests, ask specifically for these:

Boundary values:

Add boundary value tests:
- Empty string input
- Single character input
- Maximum length input (what is our max?)
- Unicode characters (emoji, CJK, RTL text)

Concurrency:

Add tests for concurrent access:
- Two users registering with the same email at the same time
- Parallel requests to the same resource
- Race conditions in the update flow

Error recovery:

Add tests for failure scenarios:
- Database connection drops mid-transaction
- External API times out
- Disk full when writing files
- Out of memory conditions

State transitions:

Add tests for state machine transitions:
- Order: created → paid → shipped → delivered
- What happens if you try to ship an unpaid order?
- What happens if you try to pay a cancelled order?
- Can you go backwards? delivered → shipped?

AI rarely generates these tests unprompted. You need to ask for them explicitly.

Setting Up Automated Test Generation in CI

You can integrate AI test generation into your CI/CD pipeline. Here is how to set it up with Claude Code in GitHub Actions.

name: AI Test Coverage
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  test-coverage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run existing tests with coverage
        run: npm test -- --coverage

      - name: Check for uncovered files
        id: coverage
        run: |
          # Parse coverage report for files under 80%
          node scripts/check-coverage.js > uncovered.txt

      - name: Generate tests for uncovered code
        if: steps.coverage.outputs.has_uncovered == 'true'
        uses: anthropics/claude-code-action@v1
        with:
          prompt: |
            These files have less than 80% test coverage:
            $(cat uncovered.txt)

            Generate tests for the uncovered lines.
            Follow our test patterns in tests/.
            Use Vitest. Mock external dependencies.
          allowed_tools: "read,write,bash"

This workflow runs on every PR. It checks coverage, identifies uncovered files, and asks Claude Code to generate tests for them. The generated tests go into a suggestion comment on the PR — a human still reviews and merges them.

Important: Do not auto-merge AI-generated tests. They need human review. Use advisory mode, not blocking mode.

Evaluating AI-Generated Test Quality

Not all AI-generated tests are good. Here is a checklist for reviewing them.

Does the test have a clear name? “should create user” is not great. “should create user with hashed password and return user without hash” is better.

Does the test verify behavior or implementation? Testing that bcrypt.hash was called with the right arguments verifies behavior. Testing that the function calls bcrypt.hash on line 15 tests implementation. The first survives refactoring. The second breaks when you move code around.

Does the test cover the failure path? If the function can throw an error, there should be a test for it. Check that error messages and status codes are asserted, not just that an error was thrown.

Are the mocks realistic? AI sometimes creates mocks that return data in a format different from the real dependency. Check that mock return values match the actual API responses.

Is the test independent? Tests should not depend on each other. If test B fails only when test A runs first, there is shared state that needs cleanup.

The Coverage Trap

Chasing coverage numbers with AI leads to bad tests. AI can easily generate tests that hit every line of code without testing anything meaningful.

A function with 100% line coverage can still have bugs if:

The tests only check that the function runs without errors
The assertions are too weak (toBeDefined, toBeTruthy)
No edge cases are covered
The mocks do not match real-world behavior

Focus on meaningful coverage instead:

Every error path is tested with specific error messages
Every branch condition is tested in both directions
Edge cases specific to your domain are covered
Integration points are tested with realistic data

A file with 70% meaningful coverage is better than a file with 100% coverage from weak tests.

When AI Testing Goes Wrong

AI-generated tests can cause real problems if you are not careful.

Tests that verify bugs. AI looks at your current code and writes tests that confirm the current behavior. If the current behavior is wrong, the test locks in the bug. Always check: “Is this the behavior I want, or the behavior that exists?”

Flaky tests. AI sometimes writes tests that depend on timing, random data, or system state. Watch for tests that pass locally but fail in CI, or pass 9 out of 10 times.

Tests that are harder to maintain than the code. If the test is 100 lines long for a 10-line function, something is wrong. Ask AI to simplify or use test utilities to reduce boilerplate.

Snapshot tests that snap the wrong thing. AI loves generating snapshot tests. They are easy to write and provide high coverage numbers. But they break on every change, including intentional ones. Use snapshots sparingly and only for stable outputs.

Key Takeaways

Prompt specifically. List every scenario you want tested. AI does not guess well about edge cases.
Review every test. Check for weak assertions, missing error paths, and mock accuracy.
Test the failures. AI tests the happy path well. You need to specifically request error, boundary, and concurrency tests.
Try test-driven vibe coding. Write tests first, then let AI implement the code to pass them. This produces better results than code-first.
Do not chase coverage numbers. Meaningful tests at 70% coverage beat meaningless tests at 100%.

What’s Next?

In AI Code Review — Catch Bugs Before They Ship, you will learn how to use AI for automated code review. Combined with AI testing, you get a complete quality pipeline that catches problems before they reach production.

Part 11 of the Vibe Coding series.

The AI Testing Workflow#

Generating Unit Tests That Actually Work#

The Weak Assertion Problem#

Integration Tests — Harder but Worth It#

Test-Driven Vibe Coding#

Edge Cases AI Misses#

Setting Up Automated Test Generation in CI#

Evaluating AI-Generated Test Quality#

The Coverage Trap#

When AI Testing Goes Wrong#

Key Takeaways#

What’s Next?#