Skip to content

Commit

Permalink
Address review comments
Browse files Browse the repository at this point in the history
- Introduce TheiaMonacoEditor object as helper member to share functions for TheiaTextEditor and TheiaOutputViewChannel
- Stabilize TheiaOutputView, ensure activation before accessing channels
- Align Theia main menu test to latest localization updates
- Update GETTINGS_STARTED.md and DEVELOPING.md
- Update build scripts to use theia-ext-scripts
  - Ensure all necessary playwright dependencies are installed on initial build
  - Ensure linting is executed on building
- Add repeatEach config to repeat tests automatically

Signed-off-by: Nina Doschek <ndoschek@eclipsesource.com>
  • Loading branch information
ndoschek committed Apr 6, 2023
1 parent 9ffd2fa commit 299b7e8
Show file tree
Hide file tree
Showing 10 changed files with 161 additions and 106 deletions.
4 changes: 3 additions & 1 deletion examples/playwright/.eslintrc.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
/** @type {import('eslint').Linter.Config} */
module.exports = {
extends: ['../../configs/build.eslintrc.json'],
extends: [
'../../configs/build.eslintrc.json'
],
ignorePatterns: ['playwright.config.ts'],
parserOptions: {
tsconfigRootDir: __dirname,
Expand Down
38 changes: 28 additions & 10 deletions examples/playwright/docs/DEVELOPING.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,33 +3,51 @@
## Building

Run `yarn` in the root directory of the repository.
In order to build Playwright and install dependencies (ex: chromium) run `yarn --cwd examples/playwright` at the root of the repository.

In order to build Playwright and install dependencies (ex: chromium) run `yarn --cwd examples/playwright build` at the root of the repository.

## Executing the tests

### Prerequisites

To work with the tests the Theia Application under test needs to be running.
Before running your tests, the Theia application under test needs to be running.
This repository already provides an example Theia application, however, you might want to test your custom Theia-based application instead.

Run `yarn browser start` to start the browser-app located in this repository.
The Playwright configuration however is aware of that and starts the backend (`yarn theia:start`) on port 3000 if not already running.
This is valid for executing tests with the VS Code Playwright extension or from your command line.

You may also use the `Launch Browser Backend` launch configuration in VS Code.

### Running the tests headless
### Running the tests in VS Code via the Playwright extension

For quick and easy execution of tests in VS Code, we recommend to use the [VS Code Playwright extension (`ms-playwright.playwright`)](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright).

Once you have installed the VS Code Playwright test extension, open the *Test* view and click the `Run Tests` button on the top toolbar or the `Run Test` button for a particular test.
It uses the default configuration with chromium as test profile by default.

To run the tests headful, simply enable the checkbox `Show browser` in the Playwright section of the *Test* view.

To start the tests run `yarn ui-tests` in the root of this package. This will start the tests located in `src/tests` in a headless mode.
### Running the tests headless via CLI

To start the tests run `yarn ui-tests` in the folder `playwright`.
This will start the tests in a headless state.

To only run a single test file, the path of a test file can be set with `yarn ui-tests <path-to-file>` or `yarn ui-tests -g "<partial test file name>"`.
See the [Playwright Test command line documentation](https://playwright.dev/docs/running-tests#command-line).
See the [Playwright Test command line documentation](https://playwright.dev/docs/intro#command-line).

### Running the tests headful
### Running the tests headful via CLI

To start the tests in a headful mode run `yarn ui-tests-headful` in the root of this package. This will start the tests located in `src/tests` in a headful mode.
If you want to observe the execution of the tests in a browser, use `yarn ui-tests-headful` for all tests or `yarn ui-tests-headful <path-to-file>` to only run a specific test.

### Watch the tests

To watch, run `yarn watch` in the root of this package. This ensures, that the testing code is up-to-date also for executing via the Playwright VSCode Extension.
Run `yarn watch` in the root of this package to rebuild the test code after each change.
This ensures, that the executed tests are up to date also when running them with the [Playwright VS Code Extension](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright).

### Debugging the tests

Please refer to the section [debugging tests](./GETTING_STARTED.md#debugging-the-tests).
Please refer to the section [Debugging the tests via the VS Code Playwright extension](./GETTING_STARTED.md#debugging-the-tests-via-the-vs-code-playwright-extension).

### UI Mode - Watch and Trace Mode

Please refer to the section [UI Mode - Watch and Trace Mode](./GETTING_STARTED.md#ui-mode---watch-and-trace-mode).
19 changes: 9 additions & 10 deletions examples/playwright/docs/GETTING_STARTED.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ test("should undo and redo text changes and correctly update the dirty state", a
});
```

Below you can see this example test in action by stepping through the code with the VSCode debug tools.
Below you can see this example test in action by stepping through the code with the VS Code debug tools.

<div style='margin:0 auto;width:100%;'>

Expand Down Expand Up @@ -124,16 +124,16 @@ Before running your tests, the Theia application under test needs to be running.
This repository already provides an example Theia application, however, you might want to test your custom Theia-based application instead.

The Playwright configuration however is aware of that and starts the backend (`yarn theia:start`) on port 3000 if not already running.
This is valid for executing tests via the VSCode extension and via CLI.
This is valid for executing tests with the VS Code Playwright extension or from your command line.

### Running the tests in VSCode via the Playwright extension
### Running the tests in VS Code via the Playwright extension

For quick and easy execution of tests in VSCode, please install the extension [`ms-playwright.playwright`](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright).
For quick and easy execution of tests in VS Code, we recommend to use the [VS Code Playwright extension (`ms-playwright.playwright`)](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright).

To quickly start the playwright tests, open the playwright testing extension and click the `Run Tests` button on the top toolbar or the `Run Test` for a particular test.
Once you have installed the VS Code Playwright test extension, open the *Test* view and click the `Run Tests` button on the top toolbar or the `Run Test` button for a particular test.
It uses the default configuration with chromium as test profile by default.

To run the tests headful, simply enable the checkbox `Show browser` in the Playwright testing extension.
To run the tests headful, simply enable the checkbox `Show browser` in the Playwright section of the *Test* view.

### Running the tests headless via CLI

Expand All @@ -147,17 +147,16 @@ See the [Playwright Test command line documentation](https://playwright.dev/docs

If you want to observe the execution of the tests in a browser, use `yarn ui-tests-headful` for all tests or `yarn ui-tests-headful <path-to-file>` to only run a specific test.

### Debugging the tests via the Playwright extension
### Debugging the tests via the VS Code Playwright extension

To debug playwright tests, open the playwright testing extension and click the `Debug Tests` button on the top toolbar or the `Debug Test` for a particular test.
To debug Playwright tests, open the *Test* view in VS Code and click the `Debug Tests` button on the top toolbar or the `Debug Test` for a particular test.
It uses the default configuration with chromium as test profile by default.

For more information on debugging, please refer to the [Playwright documentation](https://playwright.dev/docs/debug).

### UI Mode - Watch and Trace Mode

For an advanced test development experience, Playwright introduced the UI Mode. To enable this, simply add the flag `--ui` to the CLI command.
I.e. run the following command:
For an advanced test development experience, Playwright provides the so-called *UI Mode*. To enable this, simply add the flag `--ui` to the CLI command.

```bash
yarn ui-tests --ui
Expand Down
17 changes: 9 additions & 8 deletions examples/playwright/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,14 @@
},
"homepage": "https://github.com/eclipse-theia/theia",
"scripts": {
"clean": "rimraf lib *.tsbuildinfo",
"build": "tsc --incremental && yarn playwright:install",
"watch": "tsc -w --incremental",
"clean": "theiaext clean",
"build": "yarn && yarn clean && theiaext build && yarn playwright:install",
"watch": "theiaext watch",
"theia:start": "rimraf .tmp.cfg && THEIA_CONFIG_DIR=$PWD/.tmp.cfg yarn --cwd ../browser start",
"lint": "eslint -c ./.eslintrc.js --ext .ts ./src",
"lint:fix": "eslint -c ./.eslintrc.js --ext .ts ./src --fix",
"lint": "theiaext lint",
"playwright:install": "playwright install chromium",
"ui-tests": "yarn && playwright test",
"ui-tests-headful": "yarn && playwright test --headed",
"ui-tests": "yarn build && playwright test",
"ui-tests-headful": "yarn build && playwright test --headed",
"ui-tests-report-generate": "allure generate ./allure-results --clean -o allure-results/allure-report",
"ui-tests-report": "yarn ui-tests-report-generate && allure open allure-results/allure-report"
},
Expand All @@ -33,9 +32,11 @@
"fs-extra": "^9.0.8"
},
"devDependencies": {
"@theia/ext-scripts": "1.36.0",
"@types/fs-extra": "^9.0.8",
"allure-commandline": "^2.21.0",
"allure-playwright": "^2.1.0"
"allure-playwright": "^2.1.0",
"rimraf": "^2.6.1"
},
"publishConfig": {
"access": "public"
Expand Down
2 changes: 2 additions & 0 deletions examples/playwright/playwright.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ export default defineConfig({
testMatch: ['**/*.test.js'],
workers: process.env.CI ? 1 : 2,
retries: process.env.CI ? 1 : 0,
// The number of times to repeat each test, useful for debugging flaky tests
repeatEach: 1,
// Timeout for each test in milliseconds.
timeout: 30 * 1000,
use: {
Expand Down
4 changes: 2 additions & 2 deletions examples/playwright/src/tests/theia-main-menu.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -74,10 +74,10 @@ test.describe('Theia Main Menu', () => {

test('should be able to show menu item in submenu by path', async () => {
const mainMenu = await menuBar.openMenu('File');
const openPreferencesItem = await mainMenu.menuItemByNamePath('Preferences', 'Settings');
const openPreferencesItem = await mainMenu.menuItemByNamePath('Preferences', 'Open Settings (UI)');

const label = await openPreferencesItem?.label();
expect(label).toBe('Settings');
expect(label).toBe('Open Settings (UI)');
});

test('should close main menu', async () => {
Expand Down
83 changes: 83 additions & 0 deletions examples/playwright/src/theia-monaco-editor.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
// *****************************************************************************
// Copyright (C) 2023 EclipseSource and others.
//
// This program and the accompanying materials are made available under the
// terms of the Eclipse Public License v. 2.0 which is available at
// http://www.eclipse.org/legal/epl-2.0.
//
// This Source Code may also be made available under the following Secondary
// Licenses when the conditions for such availability set forth in the Eclipse
// Public License v. 2.0 are satisfied: GNU General Public License, version 2
// with the GNU Classpath Exception which is available at
// https://www.gnu.org/software/classpath/license.html.
//
// SPDX-License-Identifier: EPL-2.0 OR GPL-2.0 WITH Classpath-exception-2.0
// *****************************************************************************

import { ElementHandle } from '@playwright/test';
import { TheiaPageObject } from './theia-page-object';
import { TheiaApp } from './theia-app';

export class TheiaMonacoEditor extends TheiaPageObject {
constructor(protected readonly selector: string, app: TheiaApp) {
super(app);
}

async waitForVisible(): Promise<void> {
await this.page.waitForSelector(this.selector, { state: 'visible' });
}

protected viewElement(): Promise<ElementHandle<SVGElement | HTMLElement> | null> {
return this.page.$(this.selector);
}

async numberOfLines(): Promise<number | undefined> {
await this.waitForVisible();
const viewElement = await this.viewElement();
const lineElements = await viewElement?.$$('.view-lines .view-line');
return lineElements?.length;
}

async textContentOfLineByLineNumber(lineNumber: number): Promise<string | undefined> {
await this.waitForVisible();
const lineElement = await this.lineByLineNumber(lineNumber);
const content = await lineElement?.textContent();
return content ? this.replaceEditorSymbolsWithSpace(content) : undefined;
}

async lineByLineNumber(lineNumber: number): Promise<ElementHandle<SVGElement | HTMLElement> | undefined> {
await this.waitForVisible();
const viewElement = await this.viewElement();
const lines = await viewElement?.$$('.view-lines > .view-line');
if (!lines) {
throw new Error('Couldn\'t retrieve lines of monaco editor');
}

const linesWithXCoordinates = [];
for (const lineElement of lines) {
const box = await lineElement.boundingBox();
linesWithXCoordinates.push({ x: box ? box.x : Number.MAX_VALUE, lineElement });
}
linesWithXCoordinates.sort((a, b) => a.x.toString().localeCompare(b.x.toString()));
return linesWithXCoordinates[lineNumber - 1].lineElement;
}

async textContentOfLineContainingText(text: string): Promise<string | undefined> {
await this.waitForVisible();
const lineElement = await this.lineContainingText(text);
const content = await lineElement?.textContent();
return content ? this.replaceEditorSymbolsWithSpace(content) : undefined;
}

async lineContainingText(text: string): Promise<ElementHandle<SVGElement | HTMLElement> | undefined> {
const viewElement = await this.viewElement();
return viewElement?.waitForSelector(`.view-lines .view-line:has-text("${text}")`);
}

protected replaceEditorSymbolsWithSpace(content: string): string | Promise<string | undefined> {
// [ ] &nbsp; => \u00a0 -- NO-BREAK SPACE
// [·] &middot; => \u00b7 -- MIDDLE DOT
// [] &zwnj; => \u200c -- ZERO WIDTH NON-JOINER
return content.replace(/[\u00a0\u00b7]/g, ' ').replace(/[\u200c]/g, '');
}
}
40 changes: 9 additions & 31 deletions examples/playwright/src/theia-output-channel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import { ElementHandle } from '@playwright/test';
import { TheiaOutputView } from './theia-output-view';
import { TheiaPageObject } from './theia-page-object';
import { isElementVisible } from './util';
import { TheiaMonacoEditor } from './theia-monaco-editor';

export interface TheiaOutputViewChannelData {
viewSelector: string;
Expand All @@ -26,8 +27,12 @@ export interface TheiaOutputViewChannelData {
}

export class TheiaOutputViewChannel extends TheiaPageObject {

protected monacoEditor: TheiaMonacoEditor;

constructor(protected readonly data: TheiaOutputViewChannelData, protected readonly outputView: TheiaOutputView) {
super(outputView.app);
this.monacoEditor = new TheiaMonacoEditor(this.viewSelector, outputView.app);
}

protected get viewSelector(): string {
Expand Down Expand Up @@ -56,13 +61,12 @@ export class TheiaOutputViewChannel extends TheiaPageObject {

async numberOfLines(): Promise<number | undefined> {
await this.waitForVisible();
const viewElement = await this.viewElement();
const lineElements = await viewElement?.$$('.view-lines .view-line');
return lineElements?.length;
return this.monacoEditor.numberOfLines();
}

async maxSeverityOfLineByLineNumber(lineNumber: number): Promise<'error' | 'warning' | 'info'> {
const lineElement = await this.lineByLineNumber(lineNumber);
await this.waitForVisible();
const lineElement = await this.monacoEditor.lineByLineNumber(lineNumber);
const contents = await lineElement?.$$('span > span.mtk1');
if (!contents || contents.length < 1) {
throw new Error(`Could not find contents of line number ${lineNumber}!`);
Expand All @@ -79,32 +83,6 @@ export class TheiaOutputViewChannel extends TheiaPageObject {
}

async textContentOfLineByLineNumber(lineNumber: number): Promise<string | undefined> {
const lineElement = await this.lineByLineNumber(lineNumber);
const content = await lineElement?.textContent();
return content ? this.replaceEditorSymbolsWithSpace(content) : undefined;
}

protected async lineByLineNumber(lineNumber: number): Promise<ElementHandle<SVGElement | HTMLElement> | undefined> {
await this.waitForVisible();
const viewElement = await this.viewElement();
const lines = await viewElement?.$$('.view-lines > .view-line');
if (!lines) {
throw new Error(`Couldn't retrieve lines of output view channel ${this.channelName}`);
}

const linesWithXCoordinates = [];
for (const lineElement of lines) {
const box = await lineElement.boundingBox();
linesWithXCoordinates.push({ x: box ? box.x : Number.MAX_VALUE, lineElement });
}
linesWithXCoordinates.sort((a, b) => a.x.toString().localeCompare(b.x.toString()));
return linesWithXCoordinates[lineNumber - 1].lineElement;
}

protected replaceEditorSymbolsWithSpace(content: string): string | Promise<string | undefined> {
// [ ] &nbsp; => \u00a0 -- NO-BREAK SPACE
// [·] &middot; => \u00b7 -- MIDDLE DOT
// [] &zwnj; => \u200c -- ZERO WIDTH NON-JOINER
return content.replace(/[\u00a0\u00b7]/g, ' ').replace(/[\u200c]/g, '');
return this.monacoEditor.textContentOfLineByLineNumber(lineNumber);
}
}
3 changes: 3 additions & 0 deletions examples/playwright/src/theia-output-view.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ export class TheiaOutputView extends TheiaView {
}

async isOutputChannelSelected(outputChannelName: string): Promise<boolean> {
await this.activate();
const contentPanel = await this.page.$('#theia-bottom-content-panel');
if (contentPanel && (await contentPanel.isVisible())) {
const channelList = await contentPanel.$('#outputChannelList');
Expand All @@ -43,6 +44,7 @@ export class TheiaOutputView extends TheiaView {
}

async getOutputChannel(outputChannelName: string): Promise<TheiaOutputViewChannel | undefined> {
await this.activate();
const channel = new TheiaOutputViewChannel(
{
viewSelector: 'div.p-Widget.theia-editor.p-DockPanel-widget > div.monaco-editor',
Expand All @@ -59,6 +61,7 @@ export class TheiaOutputView extends TheiaView {
}

async selectOutputChannel(outputChannelName: string): Promise<boolean> {
await this.activate();
const contentPanel = await this.page.$('#theia-bottom-content-panel');
if (contentPanel && (await contentPanel.isVisible())) {
const channelSelectComponent = await contentPanel.$('#outputChannelList');
Expand Down
Loading

0 comments on commit 299b7e8

Please # to comment.