Playwright is a free open-source framework for automation tests. It is new to the market but is building up popularity fast. 1st release happened on 2020. Microsoft maintains it. It receives regular updates and improvements. If we look at the number of downloads for similar frameworks which have been on the market for a while, you can see Playwright has burst onto the scene.
Key Features:
Support for cross-browser platforms built on Chromium, WebKit, and Firefox – which includes Chrome, Edge, Firefox, Opera, and Safari.
Support cross-platform execution on Windows, Linux, and macOS.
Languages supported are JavaScript & TypeScript, Python, .NET, C#, and Java.
Auto-wait built-in, smart assertions that retry until the element is found, and test data tracing – keep track of logs, videos and screenshots easily.
Built with modern architecture and no restrictions – interact with multi-page, multi-tab websites like a real user, and tackle frames and browser events with ease.
Run test parallelly and it runs faster than another automation tool.
Integrate POMs as extensible fixtures.
Disadvantages:
It is new so APIs are evolving.
It has no support for IE browser and Native Mobile Apps.
Community support is not so good, but yes they are improving.
Overview
Setup
It is simple to set up Playwright in your project or create a new test project.
The above command installs the Playwright version of Chromium, Firefox, and Webkit browser.
Writing Tests
Automatic test generation
Playwright also ships with a code generator that can generate the tests for you. Run codegen and interact with the browser. The playwright will generate the code for the user interactions. codegen will attempt to build text-based selections that are durable.
You can copy generated code by clicking the button near the record button.
// example.spec.ts
import { test, expect } from '@playwright/test';
import { PlaywrightDevPage } from './playwright-dev-page';
test('getting started should contain table of contents, async ({ page }) => {
const playwrightDev = new PlaywrightDevPage(page);
await playwrightDev.goto();
await playwrightDev.getStarted();
await expect(playwrightDev.tocList).toHaveText([
'Installation',
'First test',
'Configuration file',
'Writing assertions',
'Using test fixtures',
'Using test hooks',
'VS Code extension',
'Command line',
'Configure NPM scripts',
'Release notes'
]);
});
test('should show Page Object Model article', async ({ page }) => {
const playwrightDev = new PlaywrightDevPage(page);
await playwrightDev.goto();
await playwrightDev.pageObjectModel();
await expect(page.locator('article')).toContainText('Page Object Model is a common pattern');
});
Run Tests against different environment
In many projects, we maintain different environments like dev, QA and prod, etc. To make environment variables easier to manage, consider something like .env files. Here is an example that uses the dotenv package to read environment variables directly in the configuration file.
Global Setting for pick env file to run a test against: In the playwright.config.ts file add the below code.
// playwright.config.ts
import type {
PlaywrightTestConfig
} from '@playwright/test';
import {
devices
} from '@playwright/test';
//ADD THIS CODE
import dotenv from 'dotenv';
//ADD THIS CODE
if (process.env.test_env) {
// Alternatively, read from env file.
dotenv.config({
path: `.env.${process.env.test_env}`,
override: true
})
} else {
dotenv.config({
path: `.env.development`,
override: true
})
}
/**
* See https://playwright.dev/docs/test-configuration.
*/
const config: PlaywrightTestConfig = {
testDir: './tests',
timeout: 30 * 1000,
globalSetup: 'tests/global-setup.ts',
use: {
// READ REDIRECT_URI FROM THE ENVIRONMENT FILE
baseURL: process.env.REDIRECT_URI,
storageState: 'storageState.json'
actionTimeout: 0,
trace: 'on-first-retry',
},
expect: {
timeout: 5000
},
fullyParallel: true,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 2 : undefined,
reporter: 'html',
/* Configure projects for major browsers */
projects: [{
name: 'chromium',
use: {
...devices['Desktop Chrome'],
baseURL: process.env.REDIRECT_URI,
// Tell all tests to load the signed-in state from 'storageState.json'.
storageState: 'storageState.json'
},
}, ],
};
export default config;
Pass environment in test command:
In package.json add the below commands to the script section. Install the cross-env package which Runs scripts that set and use environment variables across platforms.
We have environment files like the below .env.prod file. Same REDIRECT_URI key added in .env.development and .env.qafiles but with different values.
//.env.prod
REDIRECT_URI=https://app.redpen.ai/
Run test against the environment :
Now run a test against the different environments by running commands as per env.
// FOR DEV
npm run test-dev
// FOR QA
npm run test-qa
// FOR PROD
npm run test-prod
// YOU CAN PASS OTHER OPTIONS BY ADDING --
npm run test-prod -- --headed
Parameterization of tests
Like Selenium Playwright also offers parametrized tests. You can either parametrize tests on a test level or on a project level.
Test Level: You can also do it with the test.describe() or with multiple tests as long the test name is unique.
// example.spec.ts
const people = ['Alpha', 'Beta'];
for (const name of people) {
test(`testing with ${name}`, async () => {
// ...
});
}
Project Level:
Playwright Test supports running multiple test projects at the same time. In the following example, we'll run two projects with different options.
We declare the option person and set the value in the config. The first project runs with the value Alpha and the second with the value Beta.
// my-test.ts
import { test as base } from '@playwright/test';
export type TestOptions = {
person: string;
};
export const test = base.extend >TestOptions >({
// Define an option and provide a default value.
// We can later override it in the config.
person: ['John', { option: true }],
});
Now in the test file, we can use it.
// example.spec.ts
import { test } from './my-test';
test('test 1', async ({ page, person }) => {
await page.goto(`/index.html`);
await expect(page.locator('#node')).toContainText(person);
// ...
});
Now, we can run tests in multiple configurations by using projects. We can also use the option in a fixture.
When testing complex, integrated applications, test teams frequently turn to "modular testing" as a way to break down application functionality into small pieces. The modular pattern provides an easier-to-follow road map and then rearranges the chunks of functionality into software testing scenarios that represent different customer workflows. Creating a modular regression suite takes time, but in the end, the business has a complete list of functional areas, along with integration points.
A detailed Blog on the Modular test is here Modular Testing.
Running automation tests from CI/CD
GitHub Actions
It is very easy to run the Playwright automation tests using the GitHub action. Like all the other GitHub actions, you need to create a YAML file for the execution.
An Example (Worth a thousand words)
Here is a sample YAML file that can help you to get started. Please go through each step carefully.
# Provide a name of the action that will appear in the actions
name: Playwright Tests
on:
# The pipeline will be executed when a changeset is pushed to one of the branches defined here
push:
branches: [ 'dev' ]
# The pipeline will be executed when a changeset is pushed to a pull request created against one of the branches defined here
pull_request:
branches: [ 'dev' ]
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
# Install the required node version by the Playwright and your project
- uses: actions/setup-node@v2
with:
node-version: '14.x'
# Install the node dependencies for your project
- name: Install dependencies
run: npm ci
# This will install the browsers on which you want to run the tests
- name: Install Playwright Browsers
run: npx playwright install chromium --with-deps
# This will run the tests along with the configurations defined in this command and your configuration file
# The result of the tests will be exported according to the configurations.
# Here, we have set the configuration in a way that the test result will be a JUnit test result.
# The test result will be exported in a file named results.xml.
- name: Run Playwright tests
run: npx playwright test --project=chromium
Azure Pipeline
Create an Azure Pipeline for Playwright Tests
Integration with Test Case Management Tools
The GitHub action or the Azure Pipeline can also publish the test result directly to your test management tool.
Let's see how it can be done for the two most used test management tools in the industry.
Xray Integration
Xray is a test management tool widely used in Jira. It integrates very well in Jira. Test cases, suits, plans, and executions are also created as Jira issues by Xray. Let's see how our CI/CD pipeline can be integrated with the Xray.
Prerequisites
A Jira site having Xray installed.
Xray client id and client secret.
Xray test plan.
YAML file changes
# Here, we are going to use the APIs of Xray to report the test result.
# The first API call is made to get an Xray token.
# You will require an Xray client id and client secret to make this API call. Your Jira site admin can generate them.
# Once you have them, store them in the GitHub secrets.
You will require a client id and client secret for the same.
- name: Get XRay Token
id: tokenRequest
uses: fjogeleit/http-request-action@v1
with:
url: 'https://xray.cloud.getxray.app/api/v1/authenticate'
method: 'POST'
customHeaders: '{"Content-Type": "application/json"}'
data: '{"client_id":"${{ secrets.XRAY_CLIENT_ID }}","client_secret":"${{ secrets.XRAY_CLIENT_SECRET }}"}'
timeout: 30000
# This API call is made to publish a test execution to the Xray test plan.
# Please replace your_project_key and your_test_plan_id with your actual values in the URL below
# If the API call is successful, you will be able to see a new test execution in the test plan mentioned in the URL
- name: Post the test results to XRay
uses: fjogeleit/http-request-action@v1
with:
url: 'https://xray.cloud.getxray.app/api/v1/import/execution/junit?projectKey=your_project_key&testPlanKey=your_test_plan_id'
method: 'POST'
customHeaders: '{"Content-Type": "application/xml"}'
bearerToken: ${{ fromJSON(steps.tokenRequest.outputs.response) }}
timeout: 30000
file: 'results.xml'