This guide outlines the process for setting up and running Playwright tests for the Orion project using Azure DevOps. The tests are designed to automate the testing of the Orion web application.
Note: The Playwright tests and pipeline configurations are stored in this repository, separate from the main web application codebase.
npm ci in the terminal.This file specifies which files and directories should be ignored by Git:
node_modules/
test-results/
tests-examples/
playwright-report/
playwright/.cache/
results.xml
tests/example.spec.js
This file configures Playwright for your project:
// @ts-check
const { defineConfig, devices } = require('@playwright/test');
// Fetch the environment variables set in Azure DevOps
Object.assign(global, {
BASE_URL: process.env.BASE_URL,
USERNAME: process.env.USERNAME,
PASSWORD: process.env.PASSWORD,
});
/**
* @see https://playwright.dev/docs/test-configuration
*/
module.exports = defineConfig({
testDir: './tests',
/* Run tests in files in parallel */
fullyParallel: true,
/* Fail the build on CI if you accidentally left test.only in the source code. */
forbidOnly: !!process.env.CI,
/* Retry on CI only */
retries: process.env.CI ? 2 : 0,
/* Opt out of parallel tests on CI. */
workers: process.env.CI ? 1 : undefined,
/* Reporter to use. See https://playwright.dev/docs/test-reporters */
reporter: [
['html', { open: 'never' }],
['junit', {outputFile: 'results.xml'}] //required for Azure DevOps Pipeline
],
use: {
trace: 'on',
screenshot: 'only-on-failure',
video: {
mode: 'on'
},
headless: true,
viewport: { width: 1900, height: 940 },
launchOptions: {
slowMo: 500,
},
},
/* Configure projects for major browsers */
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
},
{
name: 'webkit',
use: { ...devices['Desktop Safari'] },
},
],
});
This file defines the job template for running Playwright tests in Azure DevOps:
parameters:
- name: BASE_URL
type: string
jobs:
- job: test
displayName: Run Playwright Tests
steps:
- download: none
- checkout: self
- task: NodeTool@0
displayName: 'Use Node version 16'
inputs:
versionSpec: 16.x
- script: |
npm ci
displayName: "NPM Install"
- script: |
npx playwright install --with-deps
displayName: "Playwright Install"
- script: |
set BASE_URL=${{ parameters.BASE_URL }}
set CI=true
set PLAYWRIGHT_JUNIT_OUTPUT_NAME=results.xml
npx playwright test
displayName: "Run Playwright Tests"
env:
USERNAME: orionUsername
PASSWORD: orionPassword
continueOnError: true
- task: ArchiveFiles@2
displayName: 'Add playwright-report to Archive'
inputs:
rootFolderOrFile: '$(Pipeline.Workspace)/s/playwright-report/'
archiveFile: '$(Agent.TempDirectory)/$(Build.BuildId)_$(System.JobAttempt)$(System.StageAttempt).zip'
- task: ArchiveFiles@2
displayName: 'Add test-results to Archive'
inputs:
rootFolderOrFile: '$(Pipeline.Workspace)/s/test-results/'
archiveFile: '$(Agent.TempDirectory)/$(Build.BuildId)_$(System.JobAttempt)$(System.StageAttempt).zip'
replaceExistingArchive: false
- task: PublishPipelineArtifact@1
displayName: 'Publish Pipeline Artifacts'
inputs:
targetPath: '$(Agent.TempDirectory)/$(Build.BuildId)_$(System.JobAttempt)$(System.StageAttempt).zip'
artifact: pipeline-artifacts
- task: PublishTestResults@2
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '$(Pipeline.Workspace)/s/results.xml'
testRunTitle: 'Playwright ADO Demo - $(System.StageName)'
displayName: 'Publish Test Results'
This file defines the main pipeline structure:
pool:
vmImage: 'windows-latest'
variables:
orionUsername: $(username)
orionPassword: $(password)
trigger:
branches:
include:
- 'main'
name: $(Build.BuildId)
stages:
- stage: qa
displayName: 'Run Automation Test - QA'
dependsOn: []
jobs:
- template: playwright_template.yml
parameters:
BASE_URL: 'https://orionfi.net/'
This file contains example test scenarios for the Orion web application. Here's a snippet:
import { test, expect } from '@playwright/test';
test('Web Portal Fleet', async ({ page }) => {
await page.goto(global.BASE_URL);
});
test('Login', async ({ page }) => {
test.slow();
await page.goto(global.BASE_URL);
await page.getByPlaceholder('User Name').click();
await page.getByPlaceholder('User Name').fill(global.USERNAME);
await page.getByPlaceholder('Password').click();
await page.getByPlaceholder('Password').fill(global.PASSWORD);
await page.getByRole('button', { name: 'Log in to Orion' }).click();
});
// ... more test scenarios ...
To run the Playwright tests on your local machine:
npx playwright testThe pipeline is configured to run Playwright tests automatically. Key components include:
playwright-automation.yml: Defines the main pipeline structure.playwright_template.yml: Contains the job template for running Playwright tests.The pipeline uses the following environment variables:
BASE_URL: The base URL of the application being tested (e.g., 'https://orionfi.net/')USERNAME: Orion username for authenticationPASSWORD: Orion password for authenticationThese variables are securely stored in Azure DevOps and referenced in the pipeline configuration.
Tests are written in TypeScript and located in the tests directory. The test-1.spec.ts file contains various test scenarios covering different aspects of the Orion web application, including:
Test results are published in two formats:
The pipeline archives and publishes the following artifacts:
These can be accessed after the pipeline run for detailed analysis of test outcomes.