-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Support coordinate transformation modes for Resample2d #270
Comments
Most of these resampling transformations were mistakes (yes, I added them to DML for completeness using a generic transform approach, but I don't recommend propagating them all into perpetuity). The correct approach from computer vision/imaging experts is to resample the center point of the pixels, not asymmetrically (which causes image shifting) nor as point samples (which causes misalignment). That approach (half pixel) is used by libraries like OpenCV, SciPy, Matlab. Recent versions of PyTorch and TF do the right thing too, and it's ONNX's default. Also see: https://medium.com/hackernoon/how-tensorflows-tf-image-resize-stole-60-days-of-my-life-aba5eb093f35 |
Now mind you, this is from an imaging perspective, which is the primary use for resampling anyway, but if someone was using resampling in some other domain like audio, or for some other clever use (such as quickly initializing a linearly spaced 1D series of numbers 0 to 100 by interpolating a 2-element tensor into a larger target tensor, like numpy.linspace) then we may want to support modes treating the endpoints as point samples rather than pixels. Though, we can always extend WebNN later when we hit the use case. ⌛... Content to close. |
I'd normally leave it to the issue opener to close an issue, but I see @Honry liked Joshua's comment along with Ningxin too. So closing. 📫➡️📪 |
Thanks @fdwr, @inexorabletash, +1 to close. 😊 |
Issue: DeepLabV3 contains
Resize
node in ONNX model andResizeBilinear
in TFLite model. It's corresponding op in WebNN isResample2d
.Resize
node containscoordinate_transformation_mode=align_corners
,ResizeBilinear
node containsalign_corners=true
andhalf_pixel_centers=false
. While WebNN doesn't support such coordinate transformation modes and its default behavior equals toalign_corners=false
andhalf_pixel_centers=true
.Compares all the behaviors in different backends and frameworks as follows:
Note: DML backend can support these modes by calculating its
InputPixelOffsets
andoutputPixelOffsets
members.You can refer to its implementation in ONNXRumtime DML backend.
Open: Can we support these coordinate transformation modes in WebNN? At least for
asymmetric
,half_pixel_centers
andalign_corners
. Or maybe we could refer to DML by definingInputPixelOffsets
andoutputPixelOffsets
options inResample2d
to implement various coordinate transformation modes.The text was updated successfully, but these errors were encountered: