The Playwright BDD Reporter
with AI Failure Analysis

1. Install

npm install -D playwright-spec-doc-reporter

2. Create the reporter shim

Playwright loads reporters in a separate worker context. Create a thin shim in your project root:

// reporter.mjs
export { default } from 'playwright-spec-doc-reporter/reporter';

3. Configure playwright.config.ts

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['./reporter.mjs', {
      outputFolder:    './test-results',
      theme:           'dark-glossy',   // 'dark-glossy' | 'dark' | 'light'
      aiAnalysis:      false,           // true to enable AI failure analysis
      aiProvider:      'anthropic',     // 'anthropic' | 'openai' | 'azure' | custom
      jiraIntegration: false,
      prComment:       false,
    }]
  ],
});

4. Run your tests

npx playwright test
# → open test-results/index.html

import { test } from '@playwright/test';
import { feature, scenario, behaviour } from 'playwright-spec-doc-reporter/annotations';

test('user can complete checkout', async ({ page }) => {
  feature('Checkout Flow');
  scenario('Complete purchase with credit card');
  behaviour('Given the user has items in their cart');
  behaviour('When they proceed to checkout and submit payment');
  behaviour('Then an order confirmation is displayed');

  await page.goto('/cart');
  // ... your test steps
});

"The checkout button wasn't clickable because a promotional modal appeared and blocked the element. This is likely a race condition — the modal loads asynchronously after page navigation. Consider waiting for the modal to dismiss or handling it in beforeEach setup."

Context sent to the model includes the error message, test code, BDD annotations, attached screenshots, and any inline API request/response logs.

Enable AI analysis

// playwright.config.ts
['./reporter.mjs', {
  aiAnalysis:  true,
  aiProvider:  'anthropic',           // or 'openai' | 'azure' | custom URL
  aiModel:     'claude-3-5-haiku-20241022',
  aiApiKey:    process.env.ANTHROPIC_API_KEY,
}]