In case you are not familiar with this plugin - you should definitely check it out if you think seriously about your mobile application. Nativescript-dev-appium is an open-source plugin which wraps/provides access to Appium and it is a suitable way to test NativeScript applications.
The main priorities of the nativescript-dev-appium plugin are:
Having these in mind, we provided a few new functionalities that will make your experience with Appium in NativeScript even more awesome.
Let’s started from the point when you try to run your tests on your local machine and how painful that can be. Appium requires a bunch of dependencies that needs to be installed and not only that but we need to provide capabilities and probably some additional setup for the JavaScript testing framework that we use. So let's be honest - resolving dependencies is not that comfortable, but at least most of the time it is a one-time job. Unfortunately, there is not much to be done from our side as well so we focused on how to relieve the execution of the tests, especially during the development process.
Starting from there the first thing we needed was to remove the necessity to provide Appium capabilities. So let's imagine that you want to run your tests on an emulator/simulator on which your application has been live-synced. Previously, you were forced to provide mandatory Appium capabilities but now you simply need to specify only the platform:
before npm run e2e -- --runType some-device-capabilities
now npm run e2e android/ios
And that's it! What would happen if you have multiple simulators/emulators running? 'nativescript-dev-appium' cannot execute simultaneous tests for now but you can provide additional parameters like --device.name="android-emulator-28" and the test will be executed on the specified device only.
Well, this seems to be nice but then we have to deal with the fact that each time this command has been executed, Appium needs to start appium-server and then appium-driver and this always takes some time. So, we came up with an idea to start the Appium process separately and reuse an already existing session. And thanks to this, it is no longer needed to wait for Appium initialization. To do this you need to execute one more command before test execution.
First, run:
./node_modules/.bin/ns-appium [android or ios] --startSession
in a separate terminal and wait until the sessionId is displayed. Don’t kill that terminal unless it is needed.
Then, you can simply execute tests like
npm run e2e -- --attachToDebug
or for short
npm run e2e -- -a
or simply create a command like
npm run e2e-debug
Of course, in case there are multiple sessions started we need to provide --sessionId as well. The other option is to start the application with Appium desktop application and attach it to the provided session from Appium. Have in mind, this is very convenient in case you want to test some actions/gestures or find the best strategy to localize elements.
To better illustrate this idea, take a look at how I debug while building some new test for the Material Components:
The next very important thing related to testing is to observe the test results. From now on, you can also use Mochawsome and it will display the results of the image comparison. Have in mind that if you are installing nativescript-dev-appium for the first time, all of these functionalities will be set up automatically and you don’t need to do this. Initialize report context in setup.ts/js :
const addContext = require('mochawesome/addContext');
const testReporterContext = <ITestReporter>{};
testReporterContext.name = "mochawesome";
testReporterContext.reportDir = "mochawesome-report";
testReporterContext.log = addContext;
testReporterContext.logImageTypes = [LogImageType.screenshots];
nsCapabilities.testReporter = testReporterContext;
Each describe should contain before hook where the report context should be provided:
describe("sample scenario", async function(){
let driver: AppiumDriver;
before("start server", async function () {
nsCapabilities.testReporter.context = this;
await startServer();
});
…
});
Setup mocha opt, file to use Mochawesome reporter [TODO: Which File?]:
--timeout 999999
--recursive e2e
--reporter mochawesome
--reporter-options quiet=true,html=true,inline=true,autoOpen=true
--exit
The viewPortRect option or in other words which part of the display should be included in image comparison. Fortunately, Appium provides viewPortRect functionality which helps us to understand which part of the display should be included in image comparison. This functionality is available since Appium >= 1.10.0 combined with UIAutomator2(Android) and XCUITests(iOS). In case you need to override this setting it is possible to provide viewPortRect in your appium capabilities file. But that’s not all. You can also provide for each test different rectangle that ns-dev-appium will consider in image verification.
driver.imageHelper.options.cropRectangle = {x:150, y:150, width: 150, height: 150}
We have also provided some additional methods which can help in comparing multiple times images and assert the result at the end of the test. Let’s imagine that you have more actions in your test and that it is needed after each action to compare an image.
beforeEach(async function () {
// this property will be used to set image name in case it is omitted in test
driver.imageHelper.testName = this.currentTest.title;
});
it(`sample-test`, async function () {
const button = await driver.waitForElement(`sampleBtn`);
await button.click();
// This method will create image (if not exists) with the name as the test name.
// In this case, the image name will be `sample-test.png`
await driver.imageHelper.compareScreen();
await button.click();
// This method will create an image (if not exists) with the same name as the test // and will add a cpunter as a postfix. In this case, the image name will be `sample-test-2.png`
await driver.imageHelper.compareScreen();
// assert the results of image comparisson.
assert.isTrue(driver.imageHelper.hasImageComparisonPassed());
});
Each comare method also accept IImageCompareOptions which can be used to override default properties that are used in image comparison.
export interface IImageCompareOptions {
imageName?: string;
timeOutSeconds?: number;
/**
* pixel
* percentage thresholds: 1 = 100%, 0.2 = 20%"
*/
tolerance?: number;
/**
* pixel
* percentage thresholds: 1 = 100%, 0.2 = 20%"
*/
toleranceType?: ImageOptions;
/**
* Wait milliseconds before capture creating image
* Default value is 2000
*/
waitBeforeCreatingInitialImageCapture?: number;
/**
* This property will preserve not to add be added _actual postfix on initial image capture
*/
donNotAppendActualSuffixOnIntialImageCapture?: boolean;
/**
* This property will ensure that the image name will not be manipulated with count postfix.
* This is very convenient in order to reuses image.
* Default value is false.
*/
keepOriginalImageName?: boolean;
/**
* Clip image before compare. Default value excludes status bar(both android and ios) and software buttons(android).
*/
cropRectangle?: IRectangle;
/**
* Default value is set to true which means that nativescript-dev-appium will save the image
* in original size and compare only the part which cropRectangle specifies.
* If false, the image size will be reduced and saved by the dimensions of cropRectangle.
*/
keepOriginalImageSize?: boolean;
/**
* Default value is set to false. nativescript-dev-appium will recalculate view port for iOS
* so that the top/y will start from the end of the status bar
* So far appium calculates it even more and some part of safe areas are missed
*/
keepAppiumViewportRect?: boolean;
/**
* Defines if an image is device-specific or only by the platform.
* Default value is true and the image will be saved in device-specific directory.
* If the value is set to false, the image will be saved under ios or android folder.
*/
isDeviceSpecific?: boolean;
/**
* Overwrite actual image if doesn't match. Default value is false.
*/
overwriteActualImage?: boolean;
}
waitForElement - this method tries to find element by automationText.
Let’s cover the next logical step “automate testing of the application”. This topic is very extensive but I will try to cover main points, which are especially important for CI. ns-dev-appium should be able to work on all platforms or services that Appium supports. Usually for local runs ns-dev-appium relies on mobile-devices-controller but this plugin is not intended to work with different vendors. That is why “--ignoreMobileController” should be applied within those runs. For SauceLab, there is an additional option --sauceLab that can provide some additional setup required by SauceLab and also will automatically ignore the mobile-devices-controller plugin. There are already some blog posts that explain how to execute tests on SauceLabs and others, so I will not go deeper
in this direction. I will try to focus on different options that might be useful to you.
--imagesPath - path to images relative to resources/images. Since the emulator name is the same for all API levels you need to differentiate it when storing images that are device-specific.
"isHeadless": true - use headless mode. This setting should be provided in appium capabilities:
"android28": {
"platformName": "Android",
"platformVersion": "28",
"deviceName": "Emulator-Api28-Google",
"avd": "Emulator-Api28-Google",
"lt": 60000,
"newCommandTimeout": 720,
"isHeadless": true,
"noReset": false,
"fullReset": false
},
One of our next goal is to enable execution of tests on multiple devices simultaneously. This will help you to scale your testing infrastructure and spend less time in waiting and more time in being productive.
But the roadmap of the Appium integration is mainly driven by you. If you have a feature request or just struggle with something that you don't know how to solve - open a new issue in the nativescript-dev-appium repository.