Fix flakiness in timeOrigin test

[chromium-wpt-export-bot]

Fix flakiness in timeOrigin test

This CL uses now() calls instead of arbitrary numbers to prevent
flakiness in this test when ran on slower machines or when the test
is interrupted by debugging.

Bug: chromium:752574
Change-Id: I9b3610a44ca5199b8a906dbe9715d280bd3439c5
Reviewed-on: https://chromium-review.googlesource.com/602381
Cr-Commit-Position: refs/heads/master@{#492112}
WPT-Export-Revision: 944cf9134ec1b2fa192de14f43ecc4790e90295b

[wpt-pr-bot]

Already reviewed downstream.

[npm1]

@jgraham do you think this is a reasonable way to tackle the flakiness of the tests or should we just remove the potentially flaky asserts? I was told we should assume that tests could be interrupted for an arbitrary amount of time. While this would be nice to have, I don't think it's something we can afford at this point because it limits our testing abilities. In fact, there are already tests that work under the assumption that the test runs uninterrupted, for example https://github.com/w3c/web-platform-tests/blob/master/hr-time/window-worker-time-origin.html

[jgraham]

I mean all I can do is try it and see what happens. Testing that things are close in time is particularly difficult especially given shared VMs in which we can just lose the timeslice for a few hundred ms.

[foolip]

@bobholt, Firefox is failing, but no results shown for any browser. Are we mid-transition to the PR results web app?

[bobholt]

@foolip. Not sure what happened. I know occasionally w3c-test.org or just the bot go down, and I think Mike added a cron job to bring it back up, so it may have hit one of those gaps.

In the closing stages of landing #6719 to switch to the new bot, so hopefully this won't be an issue in the future.

[ghost]

Build PENDING

Started: None
Finished: None

Unstable Results

Browser: "Firefox Nightly"

View in: Dashboard | TravisCI

Test	Subtest	Results	Messages
/hr-time/timeOrigin.html		OK: 10
	Window and worker timeOrigins are close when created one after another.	PASS: 10
	Window and worker timeOrigins differ when worker is created after a delay.	PASS: 10
	Window timeOrigin is close to Date.now() when there is no system clock adjustment.	PASS: 3 FAIL: 7	assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292802521.9617 but got 1502292802522 assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292816317.6418 but got 1502292816318 assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292829870.694 but got 1502292829871 assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292812896.654 but got 1502292812897 assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292799066.6682 but got 1502292799067 assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292823147.6155 but got 1502292823148 assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292819728.7422 but got 1502292819729

[ghost]

Build PENDING

Started: None
Finished: None

Unstable Results

Browser: "Firefox Nightly"

View in: Dashboard | TravisCI

Test	Subtest	Results	Messages
/hr-time/timeOrigin.html		OK: 10
	Window timeOrigin is close to Date.now() when there is no system clock adjustment.	PASS: 3 FAIL: 7	assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292802521.9617 but got 1502292802522 assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292816317.6418 but got 1502292816318 assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292829870.694 but got 1502292829871 assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292812896.654 but got 1502292812897 assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292799066.6682 but got 1502292799067 assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292823147.6155 but got 1502292823148 assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292819728.7422 but got 1502292819729
	Window and worker timeOrigins are close when created one after another.	PASS: 10
	Window and worker timeOrigins differ when worker is created after a delay.	PASS: 10

[ghost]

View the complete job log.

Firefox (nightly)

Testing web-platform-tests at revision d8ec8c5
Using binary /home/travis/build/w3c/web-platform-tests/_venv/firefox/firefox
Starting tests

Unstable results

Test	Subtest	Results	Messages
/hr-time/timeOrigin.html	Window timeOrigin is close to Date.now() when there is no system clock adjustment.	FAIL: 7/10, PASS: 3/10	assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292802521.9617 but got 1502292802522;assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292816317.6418 but got 1502292816318;assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292829870.694 but got 1502292829871;assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292812896.654 but got 1502292812897;assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292799066.6682 but got 1502292799067;assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292823147.6155 but got 1502292823148;assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292819728.7422 but got 1502292819729

All results

1 test ran

/hr-time/timeOrigin.html

Subtest	Results	Messages
	OK
Window timeOrigin is close to Date.now() when there is no system clock adjustment.	FAIL: 7/10, PASS: 3/10	assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292802521.9617 but got 1502292802522;assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292816317.6418 but got 1502292816318;assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292829870.694 but got 1502292829871;assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292812896.654 but got 1502292812897;assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292799066.6682 but got 1502292799067;assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292823147.6155 but got 1502292823148;assert_less_than_equal: Date.now() should be close to window timeOrigin. expected a number less than or equal to 1502292819728.7422 but got 1502292819729
Window and worker timeOrigins are close when created one after another.	PASS
Window and worker timeOrigins differ when worker is created after a delay.	PASS

[ghost]

View the complete job log.

Sauce (safari)

Testing web-platform-tests at revision 931f1f3
Using binary None
Starting tests
All results were stable

All results

1 test ran

/hr-time/timeOrigin.html

Subtest	Results	Messages
	ERROR
Window timeOrigin is close to Date.now() when there is no system clock adjustment.	FAIL	assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877580567;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877735807;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877913501;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877622956;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877782814;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877847373;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877612978;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877700454;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877795833;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877874221
Window and worker timeOrigins are close when created one after another.	TIMEOUT	Test timed out
Window and worker timeOrigins differ when worker is created after a delay.	TIMEOUT	Test timed out

[ghost]

View the complete job log.

Chrome (unstable)

Testing web-platform-tests at revision d8ec8c5
Using binary None
Starting tests
All results were stable

All results

1 test ran

/hr-time/timeOrigin.html

Subtest	Results	Messages
	OK
Window timeOrigin is close to Date.now() when there is no system clock adjustment.	FAIL	assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1502292752080;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1502292759017;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1502292754273;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1502292753393;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1502292757573;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1502292761453;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1502292755141;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1502292756134;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1502292760487;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1502292750646
Window and worker timeOrigins are close when created one after another.	FAIL	assert_greater_than_equal: Worker timeOrigin should be greater than the window timeOrigin. expected a number but got a "undefined"
Window and worker timeOrigins differ when worker is created after a delay.	FAIL	assert_greater_than_equal: We waited 200ms to spawn the second worker, so its timeOrigin should be greater than that of the window. expected a number greater than or equal to 200 but got NaN

[ghost]

View the complete job log.

Sauce (MicrosoftEdge)

Testing web-platform-tests at revision 931f1f3
Using binary None
Starting tests
All results were stable

All results

1 test ran

/hr-time/timeOrigin.html

Subtest	Results	Messages
	OK
Window timeOrigin is close to Date.now() when there is no system clock adjustment.	FAIL	assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877872332;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877892516;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877994902;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877906968;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877972397;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877923366;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877938821;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877837048;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877854500;assert_greater_than_equal: Date.now() should be at least as large as the window timeOrigin. expected a number greater than or equal to undefined but got 1501877956491
Window and worker timeOrigins are close when created one after another.	FAIL	assert_greater_than_equal: Worker timeOrigin should be greater than the window timeOrigin. expected a number but got a "undefined"
Window and worker timeOrigins differ when worker is created after a delay.	FAIL	assert_greater_than_equal: We waited 200ms to spawn the second worker, so its timeOrigin should be greater than that of the window. expected a number greater than or equal to 200 but got NaN

[kereliuk]

@bobholt I ran this test locally and with the travis CI without this PR and found it was not flaky.
Can you investigate what makes it flaky now?

[foolip]

Leaving negative review so that this isn't somehow automatically merged before #6822. (The "do not merge yet" label would get automatically removed.)

[foolip]

#6822 is merged, closing this.