[plugin-mobile-app] Add steps to record performance data on iOS devices #1988

dtargons · 2021-09-21T19:11:12Z

codecov · 2021-09-21T21:10:09Z

Codecov Report

Merging #1988 (07def96) into master (a0e448d) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##             master    #1988   +/-   ##
=========================================
  Coverage     96.34%   96.34%           
- Complexity     5396     5400    +4     
=========================================
  Files           764      765    +1     
  Lines         15483    15499   +16     
  Branches       1032     1032           
=========================================
+ Hits          14917    14933   +16     
  Misses          442      442           
  Partials        124      124

Impacted Files	Coverage Δ
...vidus/bdd/mobileapp/steps/IOSPerformanceSteps.java	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a0e448d...07def96. Read the comment docs.

...dus-plugin-mobile-app/src/main/java/org/vividus/bdd/mobileapp/steps/IOSInstrumentsSteps.java

docs/modules/plugins/pages/plugin-mobile-app.adoc

...dus-plugin-mobile-app/src/main/java/org/vividus/bdd/mobileapp/steps/IOSInstrumentsSteps.java

uarlouski · 2021-09-22T08:17:42Z

...dus-plugin-mobile-app/src/main/java/org/vividus/bdd/mobileapp/steps/IOSInstrumentsSteps.java

+        args.put(PROFILE_NAME, instrument);
+
+        File traceZip = new File(path);
+        String b64Zip = (String) getWebDriver().executeScript("mobile: stopPerfRecord", args);


what if I forget to stop recording by the end of scenario? can it lead to resource leaks?

This data is recorder by appium. I haven't seen any information about possibility of memory leaks.

My question was about consequences if we don't stop performance recording, is the recording just aborted if we close the session?

Yeah, recording will be aborted on Appium session close.

uarlouski · 2021-09-22T08:30:57Z

...dus-plugin-mobile-app/src/main/java/org/vividus/bdd/mobileapp/steps/IOSInstrumentsSteps.java

+
+        File traceZip = new File(path);
+        String b64Zip = (String) getWebDriver().executeScript("mobile: stopPerfRecord", args);
+        byte[] bytesZip = Base64.getMimeDecoder().decode(b64Zip);


why do we need mime decoder here?

I don't know. I don't familiar with encode/decode functionality. I just reused official appium's approach to work with such data.

ikalinin1 · 2021-09-22T08:42:44Z

...dus-plugin-mobile-app/src/main/java/org/vividus/bdd/mobileapp/steps/IOSInstrumentsSteps.java

+     * @param path Path to the location for saving the screenshot
+     * @throws IOException If an input or output exception occurred
+     */
+    @When("I stop recording '$instrument' metrics and save data to '$path'")


it's unclear how these results could be helpful, should we import them into instruments manually?

Is possible to parse the results and push some HTML attachment into allure?

Result is *.trace file. It could be opened by Xcode Instrument application to see data in visual format. Also it is possible to parse this data into xml format using xcrun xctrace Xcode's utility

Am I correct that in your case you're parsing perf results with xctrace and validating XML? If that so. could you please submit a short user guide on that here:
https://github.com/vividus-framework/vividus/tree/master/docs/modules/guides

We don't actually measure performance of application now, but use this functional only for collecting custom development metrics which where added into code. I don't think it will be useful. I added link to documentation for xctrace utility with examples of usage

...dus-plugin-mobile-app/src/main/java/org/vividus/bdd/mobileapp/steps/IOSInstrumentsSteps.java

valfirst · 2021-09-22T08:56:46Z

...dus-plugin-mobile-app/src/main/java/org/vividus/bdd/mobileapp/steps/IOSInstrumentsSteps.java

+    {
+        if (!webDriverManager.isIOS())
+        {
+            throw new IllegalStateException("Step is only supported on IOS devices");


is it applicable for iOS real devices only? or simulators are in scope as well?

Simulators are in scope

...dus-plugin-mobile-app/src/main/java/org/vividus/bdd/mobileapp/steps/IOSInstrumentsSteps.java

Co-authored-by: Valery Yatsynovich <valfirst@yandex.ru>

valfirst · 2021-11-08T06:31:50Z

docs/modules/plugins/pages/plugin-mobile-app.adoc

+When I stop recording `$instrument` metrics and save results to file `$path`
+----
+
+. `$instrument` - The instrument of Xcode Instruments utility


I believe it was already mentioned in the previous comments, but not fixed yet: please enumerate the available instruments or provide the link how to find the list

valfirst · 2021-11-08T06:31:59Z

docs/modules/plugins/pages/plugin-mobile-app.adoc

+When I stop recording `$instrument` metrics and save results to file `$path`
+----
+
+. `$instrument` - The instrument of Xcode Instruments utility


Suggested change

. `$instrument` - The instrument of Xcode Instruments utility

. `$instrument` - The instrument of Xcode Instruments utility.

valfirst · 2021-11-08T06:32:32Z

docs/modules/plugins/pages/plugin-mobile-app.adoc

+----
+
+. `$instrument` - The instrument of Xcode Instruments utility
+. `$path` - Path to save archive with collected data


Suggested change

. `$path` - Path to save archive with collected data

. `$path` - The path to the file to save an archive with collected data.

valfirst · 2021-11-08T06:33:30Z

...dus-plugin-mobile-app/src/main/java/org/vividus/bdd/mobileapp/steps/IOSPerformanceSteps.java

+
+    private void checkIOS()
+    {
+        Validate.isTrue(webDriverManager.isIOS());


please put some message here so that users can understand the error

[plugin-mobile-app] Add steps to record performance data on iOS devices

902356a

valfirst requested review from valfirst, uarlouski and ikalinin1 September 21, 2021 21:06

uarlouski reviewed Sep 22, 2021

View reviewed changes

...dus-plugin-mobile-app/src/main/java/org/vividus/bdd/mobileapp/steps/IOSInstrumentsSteps.java Outdated Show resolved Hide resolved

uarlouski reviewed Sep 22, 2021

View reviewed changes

...dus-plugin-mobile-app/src/main/java/org/vividus/bdd/mobileapp/steps/IOSInstrumentsSteps.java Outdated Show resolved Hide resolved

uarlouski reviewed Sep 22, 2021

View reviewed changes

docs/modules/plugins/pages/plugin-mobile-app.adoc Outdated Show resolved Hide resolved

uarlouski reviewed Sep 22, 2021

View reviewed changes

...dus-plugin-mobile-app/src/main/java/org/vividus/bdd/mobileapp/steps/IOSInstrumentsSteps.java Outdated Show resolved Hide resolved

uarlouski reviewed Sep 22, 2021

View reviewed changes

ikalinin1 reviewed Sep 22, 2021

View reviewed changes

valfirst requested changes Sep 22, 2021

View reviewed changes

ikalinin1 mentioned this pull request Sep 23, 2021

Introduce step' availability on the platform checking mechanism #1991

Open

dtargons and others added 2 commits October 7, 2021 15:53

Apply suggestions from code review

bf91c4f

Co-authored-by: Valery Yatsynovich <valfirst@yandex.ru>

Review fixes

7ed5a92

valfirst requested changes Nov 8, 2021

View reviewed changes

valfirst requested review from ikalinin1, valfirst and uarlouski November 8, 2021 06:58

ikalinin1 approved these changes Nov 8, 2021

View reviewed changes

ikalinin1 self-requested a review November 8, 2021 08:37

ikalinin1 approved these changes Nov 8, 2021

View reviewed changes

Fixes for code review

57f4796

dtargons force-pushed the ios_perf branch from 11536cc to 57f4796 Compare January 11, 2022 14:17

Merge branch 'master' into ios_perf

07def96

uarlouski approved these changes Jan 12, 2022

View reviewed changes

valfirst approved these changes Jan 12, 2022

View reviewed changes

valfirst merged commit 7908a18 into vividus-framework:master Jan 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[plugin-mobile-app] Add steps to record performance data on iOS devices #1988

[plugin-mobile-app] Add steps to record performance data on iOS devices #1988

dtargons commented Sep 21, 2021 •

edited by valfirst

Loading

codecov bot commented Sep 21, 2021 •

edited

Loading

uarlouski Sep 22, 2021 •

edited

Loading

dtargons Oct 7, 2021

uarlouski Nov 8, 2021

dtargons Jan 11, 2022

uarlouski Sep 22, 2021

dtargons Oct 7, 2021

ikalinin1 Sep 22, 2021

ikalinin1 Sep 22, 2021

dtargons Oct 7, 2021

ikalinin1 Nov 8, 2021

dtargons Jan 11, 2022

valfirst Sep 22, 2021

dtargons Oct 7, 2021

valfirst Nov 8, 2021

valfirst Nov 8, 2021

valfirst Nov 8, 2021

valfirst Nov 8, 2021

	. `$instrument` - The instrument of Xcode Instruments utility
	. `$instrument` - The instrument of Xcode Instruments utility.

	. `$path` - Path to save archive with collected data
	. `$path` - The path to the file to save an archive with collected data.

[plugin-mobile-app] Add steps to record performance data on iOS devices #1988

[plugin-mobile-app] Add steps to record performance data on iOS devices #1988

Conversation

dtargons commented Sep 21, 2021 • edited by valfirst Loading

codecov bot commented Sep 21, 2021 • edited Loading

Codecov Report

uarlouski Sep 22, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dtargons commented Sep 21, 2021 •

edited by valfirst

Loading

codecov bot commented Sep 21, 2021 •

edited

Loading

uarlouski Sep 22, 2021 •

edited

Loading