Testing Agile

Introduction

In software testing, managing test cases efficiently is crucial, especially when dealing with smoke tests, regression tests, and bug-related tests. Playwright allows testers to categorize tests using tags, helping teams run targeted test executions, filter failures, and optimize test suites.

In this article, we’ll cover the different types of tags in Playwright, how to handle them, and how to leverage them effectively in automated testing.

1️⃣ Types of Test Tags in Playwright

Before diving into implementation, let’s classify the main types of tags commonly used in testing:

📌 Smoke Test Tags (@smoke)

• Covers critical functionalities of the application.
• Ensures that core features work before proceeding with deeper testing.
• Typically runs on every build/deployment.

📌 Regression Test Tags (@regression)

• Covers broader test cases to catch defects introduced by new changes.
• Runs periodically (e.g., nightly runs, pre-release tests).

📌 Bug Test Tags (@bug-XXXXX)

• Identifies tests linked to specific known bugs.
• Useful for verifying bug fixes and retesting issues.

📌 Feature-Based Tags (@feature-XYZ)

• Helps organize tests based on functional modules.
• Used for executing tests related to specific product features.

2️⃣ How to Handle Test Tags in Playwright

Playwright does not have built-in tagging, but we can manage it through metadata annotations, grep filters, and configuration settings.

📍 Using .meta() to Add Tags

import { test, expect } from '@playwright/test';

test('Login test', async ({ page }) => {
  await page.goto('https://example.com/login');
  await page.fill('#username', 'user');
  await page.fill('#password', 'password');
  await page.click('#login');
  expect(await page.url()).toContain('/dashboard');
}).meta({ tag: 'smoke' });

Running Tests Based on Tags

Run only smoke tests

npx playwright test --grep smoke

Run only regression tests

npx playwright test --grep regression

Run tests except for bug-tagged ones

npx playwright test --grepInvert BUG-

Handling Smoke Tests in Playwright

Why Smoke Testing?

Smoke tests validate that critical features work before deeper testing. A failure in smoke tests usually means stopping further execution.

Setting Up Smoke Tests

test.describe('@smoke', () => {
  test('User should be able to login', async ({ page }) => {
    await page.goto('https://example.com');
    await page.fill('#username', 'admin');
    await page.fill('#password', 'admin123');
    await page.click('#login');
    expect(await page.url()).toContain('/dashboard');
  });
});

Running Smoke Tests

npx playwright test --grep @smoke

Handling Regression Tests in Playwright

Why Regression Testing?

Regression tests ensure new changes do not break existing functionality. These tests typically include edge cases and complex user flows.

Setting Up Regression Tests

test.describe('@regression', () => {
  test('Verify checkout process', async ({ page }) => {
    await page.goto('https://example.com/checkout');
    await page.click('#purchase');
    expect(await page.textContent('#confirmation')).toContain('Order placed');
  });
});

Running Regression Tests

npx playwright test --grep @regression

Handling Bug Tags in Playwright

Bug tags allow teams to track failing tests due to known defects.

To tag a bug in Playwright, you can follow this method:

Using .meta() to Tag a Test with a Bug ID

You can use .meta() to attach a bug ID to a test. This makes it easier to track and filter tests related to known defects.

Example: Tagging a Test with a Bug ID

import { test, expect } from '@playwright/test';

test('Checkout process should complete', async ({ page }) => {
  await page.goto('https://example.com/checkout');
  await page.click('#purchase');
  expect(await page.textContent('#confirmation')).toContain('Order placed');
}).meta({ bug: 'BUG-5678' });
 

Run only tests related to Bug 5678:

npx playwright test --grep BUG-5678

Skipping Tests for Known Bugs

test.skip('Bug #1234 - Checkout button does not work', async ({ page }) => {
  await page.goto('https://example.com');
  await page.click('#checkout');
  expect(await page.isVisible('#confirmation')).toBeTruthy();
});

Running All Tests Except Bugged Ones

npx playwright test --grepInvert BUG-

Marking Expected Failures Due to Bugs

test.fail('Bug #7890 - Incorrect total price in checkout', async ({ page }) => {
  await page.goto('https://example.com/cart');
  expect(await page.textContent('#total-price')).toBe('$100'); // Expected wrong value
});

Configuring Playwright to Manage Tags Automatically

To exclude bug-tagged tests by default, update playwright.config.ts:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  grepInvert: /BUG-/, // Exclude all tests tagged with "BUG-"
});

Run tests while ignoring bugged ones

npx playwright test

Conclusion

Handling tags effectively in Playwright helps in test categorization, selective execution, and debugging known issues.

• Use @smoke for critical functionality tests.
• Use @regression for broad test coverage.
• Use @bug-XXXXX to track known defects.
• Use grep filters to run or exclude specific tests.

By integrating these tagging strategies, you can streamline your test execution and ensure a structured, efficient test suite.