-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Add moz-webgpu-cts process-reports
subcmd. with multiple --preset
s
#14
Conversation
cd7902c
to
454a4db
Compare
I was struggling to get traction on resolving conflicts between expected outcomes in metadata and reported outcomes. I realized I was missing an answer to the question, “How do I structure new expectations' conditional values once reports have been analyzed?” An example answer: one could organize by platform: [test]
[subtest]
expected:
if os == "win": FAIL
if os == "linux": TIMEOUT …or one could organize by build profile: [test]
[subtest]
expected:
if debug: FAIL
if not debug: TIMEOUT …or even potentially mix them, which seems reasonable, but only muddies up the solution space for “what do we emit”: [test]
[subtest]
expected:
if os == "win": NOTRUN
if debug: FAIL In addition to the variables explored above, WPT upstream lets users define what I'll call “branching points” in expectation structure for any run info variable. WPT upstream's I determined to answer the question I've posed instead by (1) baking a branch point configuration into code for now, and (2) granting Footnotes
|
4d39480
to
eb6f649
Compare
20f8b91
to
9a92a59
Compare
cfa3114
to
d35ca95
Compare
moz-webgpu-cts adjust-exps
subcmd.moz-webgpu-cts process-reports
subcmd.
d35ca95
to
2cedac7
Compare
323f8a3
to
02ea3ee
Compare
09caea1
to
3705a3b
Compare
Co-authored-by: Jim Blandy <jimb@red-bean.com>
3705a3b
to
fe5ada9
Compare
moz-webgpu-cts process-reports
subcmd.moz-webgpu-cts process-reports
subcmd. with multiple --preset
s
Resolves #13, #9.
Depends on:
whippit
to be a set oftraits
instead ofstruct
s #40