Unable to stream camera on Jetson Orin NX 8GB, gstreamer warnings

[costasvav]

I am attempting to run the NanoOWL examples on my Seeed ReComputer J4011 (Orin NX 8GB with J401 carrier board). I can get the container to work with the Owl test, but not the Tree test. Same results for this repo running directly on the Jetson, outside the container. I do not appear to be able to stream the live video at all through the code with the following errors:

(test_owl.py:3347): Gdk-CRITICAL **: 17:22:29.833: gdk_cursor_new_for_display: assertion 'GDK_IS_DISPLAY (display)' failed
/home/user/.local/lib/python3.8/site-packages/torch/functional.py:505: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/native/TensorShape.cpp:3490.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
INFO:root:Opening camera.
[ WARN:0] global /tmp/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp (1760) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global /tmp/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /tmp/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
INFO:root:Loading predictor.
======== Running on http://0.0.0.0:7860 ========
(Press CTRL+C to quit)
[ WARN:1] global /tmp/build_opencv/opencv/modules/videoio/src/cap_v4l.cpp (1004) tryIoctl VIDEOIO(V4L2:/dev/video1): select() timeout.

This occurs on both camera ports. I am able to stream the two camera feeds with Gstreamer just fine through my own Python code.

Thoughts of where to diagnose? I can even built OpenCV with CUDA support for the device and has passed the basic tests.

[jaybdub]

Hi @costasvav ,

Thanks for reaching out!

You mention that this error only occurs with the Tree detector.

Does this example work for you (without using camera)?

https://github.com/NVIDIA-AI-IOT/nanoowl/blob/main/examples/tree_predict.py

John

[costasvav]

Thanks for the prompt response. I cannot run Tree demos, as CLIP is not installed. Let me know if you have a tutorial on how to install it, I've only found a Roboflow container, no normal install guide, and it does not come included in the nVidia container.

I tried to run OpenCV in Python3 outside of the other test code, but I get the same error above the moment I call camera = cv2.VideoCapture(0). Gstreamer by itself works, Jetson_Utils works too. I will purge OpenCV and reinstall to see if that fixes any issues. It didn't work with the stock OpenCV that comes with JetPack 5.1.2, hence the reason I tried to build OpenCV w/ CUDA support.

OpenCV 4.4 w/ CUDA

[jaybdub]

For CLIP, can you try

pip install git+https://github.com/openai/CLIP.git

Best,
John

[costasvav]

Thanks. I will look into it and get back to you. I now have another issue and will make a new ticket for it.

[costasvav]

Okay, CLIP working, I can run through the owl_predict and tree_predict on the owl image. But I cannot run the tree_demo example due to same errors as above.
This is native on the Jetson, not in a container.