feat(XR): Update to WebXR functionality in OpenGL RenderWindow

[tbirdso]

PR and Code Checklist

• semantic-release commit messages
• Run npm run reformat to have correctly formatted code

Context

Replaces WebVR with WebXR methods for running an immersive VR session from an OpenGL RenderWindow.

WebXR exposes methods for both VR and AR sessions across a diverse set of platforms, including standalone headsets, fixed displays, and mobile devices. These changes use WebXR to run a VR session in a dedicated headset or mobile viewer. Applications/SkyboxViewer and Geometry/VR examples are updated for API changes.

Planned future additions (outside the scope of this PR):

• WebXR polyfill
• XR gamepad API
• AR

Changes

• RenderWindow now manages VR sessions with startXR, enterXR, stopXR, and xrRender
• Applications/SkyboxViewer and Geometry/VR examples are updated to query availability/launch immersive sessions with VR
• Gamepad support for VR examples is deprecated and will be reintroduced with the WebXR Gamepads API in the near future.

Results

• Applications/SkyboxViewer and Geometry/VR examples query VR availability and launch/halt VR sessions correctly in WebXR-compatible browsers
• VR examples compatible with tethered headsets (such as HTC Vive Pro) or mobile headsets with dedicated processor (Quest)

Testing

• All tests complete without errors on the following environment:
  • vtk.js: v21.1.2
  • OS: Windows 10
  • Browser: Chrome 96.0.4664.45
• VR examples run properly in WebXR-supported environment
  • WebXR emulator
  • Oculus Browser
  • Chrome (via Oculus WebXR backend)

[tbirdso]

Current changes allow scene to be appropriately rendered onscreen before sending to XR and then in a headset once the button is pressed. However, stopXR method does not seem to properly reset the scene for viewing on a fixed display. After returning from XR, trying to interact with the renderwindow via click and drag gives two error messages repeatedly when traverseAllPasses is called:

WebGL: INVALID_FRAMEBUFFER_OPERATION: clear: Cannot render to a XRWebGLLayer framebuffer outside of an XRSession animation frame callback.
WebGL: INVALID_FRAMEBUFFER_OPERATION: drawArrays: Cannot render to a XRWebGLLayer framebuffer outside of an XRSession animation frame callback.

It seems that the XR animation frame is cancelled as expected, so the question is why the XRWebGLLayer is still used outside of expected XR methods.

This error is not observed with an XR emulator or when the page is rendered natively on a mobile headset (tested on Quest), in which cases the user can interact with the scene as expected after returning from VR. This seems to be an issue with trying to select which display to render to, perhaps as a result of making an existing GL layer XR compatible.

@floryst @martinken Any thoughts or suggestions here? I do not see this issue when running the WebXR VR sample, but that example also never tries to render to the main canvas in the browser.

EDIT: Resolved below.

[tbirdso]

Also pinging @jadh4v as an intro to WebXR investigation

[tbirdso]

Above issue is resolved, I was not unbinding the XR context framebuffer in stopXR, thanks @floryst for the find. With this fix the scene can be interacted with again via mouse events after exiting VR.

Some remaining issues to note:

• Attempting to interact with the canvas while the XR session is underway still yields INVALID_FRAMEBUFFER_OPERATION, I assume due to trying to draw to the canvas while it is not available. Previous WebVR code handles this by checking whether the display is external or not, but so far it doesn't look like WebXR exposes this value. Will continue to investigate.
• Entering and exiting VR repeatedly with SteamVR as the OpenXR backend can cause SteamVR to crash. Not seeing this issue with Oculus as the OpenXR backend.
• Cone appears at different heights/positions for SteamVR and Oculus backends.

EDIT: Pinging @martinken and @floryst for discussion of lingering issues, particularly the framebuffer error.

EDIT2: Framebuffer issue resolved in second fixup commit below.

[tbirdso]

Second fixup commit contains minor bug fixes and cleanups:

• Fixed bug where RenderWindowInteractor tried to 1) request animation frame on mouse click + drag for rotation and 2) force render on mouse release from rotation. Mouse events have no bearing on the scene while in VR.
• Better handling for potential error where the user takes off the headset and the XRSession ends before Return from VR is clicked. For now we assume that the XRSession should continue until the user manually indicates that it should end. We could try to handle an end event to do the same, but risk mishandling situations such as where two users pass a headset between each other and the session "ends" in between.
• Reset the camera once the XRSession is manually ended to regain a good view of the scene.

Will plan to squash fixup commits before merge.

[tbirdso]

@floryst @martinken this is ready for re-review when you have time

[martinken]

+1

[tbirdso]

Squashed fixup commits.

[floryst]

LGTM. Thanks!

[github-actions]

🎉 This PR is included in version 21.2.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀