BatchNormalization should be an optional operation

[pyu10055]

BatchNormalization usually are optimized away for inference (fused into conv2d op filter and bias), for example TFLite does not have this op, for compatibility purpose we shall move this op to optional section of the spec.

[wchao1115]

BatchNorm is actually fully supported as a native operation in DirectML. It is true that for performance benefit it is often fused with the preceding convolution and following relu activation. However, those are framework choices to make. The WebNN implementer can also choose to apply that same fusion at the backend through the graph compilation step. But again, it is not a requirement that this fusion must be made at all time. A backend implementation should be allowed to treat BatchNorm as a standalone operation, or fuse it with just the following relu, etc. I think it's still an implementation choice.

[pyu10055]

Thanks, @wchao1115. Given that TF Lite does not have this op, would it be ok to mark it as optional in the spec? It's fine if the DirectML implementation includes it by default. It would be nice if the spec focuses first on what's common to all the major ML frameworks.

[wchao1115]

For the implementation that lacks native support for this operation, it can be entirely implemented by compositing other operations as illustrated in the note section of the batchNormalization operation spec. I don't think we can define this operation as optional because it'll break the user's expectation.