Belajar QA Automation Engineer - Test Reporting & Dashboards

#Pendahuluan

Setelah di episode 16 kita mengoptimalkan execution speed, pada episode ini kita membahas bagaimana melihat dan memahami hasil test — reporting yang baik memungkinkan informed decisions.

#Playwright Reporters

#Built-in Reporters

typescript

// playwright.config.ts
export default defineConfig({
  reporter: [
    // HTML report (interactive)
    ['html', { outputFolder: 'playwright-report', open: 'never' }],
 
    // JSON (untuk processing)
    ['json', { outputFile: 'test-results.json' }],
 
    // JUnit XML (untuk CI)
    ['junit', { outputFile: 'test-results.xml' }],
  ],
});

#GitHub Actions Integration

yaml

# Upload report
- uses: actions/upload-artifact@v4
  if: always()
  with:
    name: playwright-report
    path: playwright-report/
    retention-days: 14
 
# Publish test results
- uses: EnricoMi/publish-unit-test-result-action@v2
  if: always()
  with:
    files: test-results.xml

#Allure Report

#Setup

bash

# Install Allure
npm install -D @playwright/test allure-js-commons
 
# Jalankan dengan Allure reporter
npx playwright test --reporter=allure-playwright
 
# Generate report
npx allure generate allure-results --clean
npx allure open

#Custom Metadata

typescript

import { allure } from 'allure-js-commons';
 
test('login test', async ({ page }) => {
  allure.epic('Authentication');
  allure.feature('Login');
  allure.story('Valid Login');
  allure.severity('critical');
  allure.tag('smoke');
 
  // Test code...
});

#Metrics Collection

#Key Metrics

typescript

// Collect custom metrics
test('track metrics', async ({ page }) => {
  const startTime = Date.now();
 
  await page.goto('/');
  await page.waitForLoadState('networkidle');
 
  const loadTime = Date.now() - startTime;
 
  // Log metric
  console.log(`Load time: ${loadTime}ms`);
 
  // Store for reporting
  const metrics = {
    test: 'homepage load',
    duration: loadTime,
    timestamp: new Date().toISOString(),
  };
 
  // Write to file
  fs.appendFileSync('metrics.json', JSON.stringify(metrics) + '\n');
});

#Dashboard Data

json

{
  "totalTests": 150,
  "passed": 145,
  "failed": 3,
  "skipped": 2,
  "duration": 180000,
  "flakyRate": 0.02,
  "coverage": {
    "critical": 100,
    "high": 85,
    "medium": 70
  }
}

#Trend Analysis

text

Dashboard Metrics:
├── Pass rate: 97% (↑ from 95% last week)
├── Average duration: 12ms (↓ from 15ms)
├── Flaky rate: 2% (↓ from 5%)
├── Coverage: 85% (↑ from 80%)
├── New tests: 10 (this sprint)
└── Fixed tests: 5 (flaky fixes)

Note

Reporting harus automated — jangan manual compile. Setiap CI run harus generate report dan update dashboard.

#Praktik: Complete Reporting

typescript

// playwright.config.ts
export default defineConfig({
  reporter: [
    ['html', { open: 'never' }],
    ['json', { outputFile: 'results.json' }],
    ['junit', { outputFile: 'results.xml' }],
  ],
});

#Penutup

Playwright reporters: HTML (interactive), JSON (processing), JUnit (CI).
Allure: rich reports dengan custom metadata.
Metrics: collect pass rate, duration, flaky rate, coverage.
Dashboard: automated trend analysis dan reporting.

Di episode 18 selanjutnya kita akan membahas secure test automation — secrets handling, test accounts, dan environment isolation. Sampai jumpa di episode 18!