Autoware.universe install fail on Jetson Xavier platform

[esteve]

Discussed in #120

^{Originally posted by jason914 March 25, 2022}
Dear all,

I use the Jetson Xavier platform. It is ARM64 platform. The OS is JetPack 4.4 (Ubuntu 18.04).
I refer https://autowarefoundation.github.io/autoware-documentation/main/installation/autoware/docker-installation/ commands.
But there are 2 questions when I install Autoware.universe.

Q1:
There is an error when use following command.
./setup-dev-env.sh docker
Following is error log.

`nvidia@xavier:~$ ./setup-dev-env.sh docker
Setting up the build environment take up to 1 hour.

Are you sure to run the setup? [y/N] y
[sudo] password for nvidia:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package 'ansible' is not installed, so not removed
The following packages were automatically installed and are no longer required:
apt-clone archdetect-deb bogl-bterm busybox-static cryptsetup-bin
dpkg-repack gir1.2-timezonemap-1.0 gir1.2-xkl-1.0 grub-common
kde-window-manager kpackagetool5 kwayland-data kwin-common kwin-data
kwin-x11 libdebian-installer4 libkdecorations2-5v5
libkdecorations2private5v5 libkf5declarative-data libkf5declarative5
libkf5globalaccelprivate5 libkf5idletime5 libkf5kcmutils-data
libkf5kcmutils5 libkf5newstuff-data libkf5newstuff5 libkf5newstuffcore5
libkf5package-data libkf5package5 libkf5plasma5 libkf5quickaddons5
libkf5waylandclient5 libkf5waylandserver5 libkscreenlocker5
libkwin4-effect-builtins1 libkwineffects11 libkwinglutils11
libkwinxrenderutils11 libllvm9 libopts25 libqgsttools-p1 libqt5multimedia5
libqt5multimedia5-plugins libqt5multimediaquick-p5 libqt5multimediawidgets5
libxcb-composite0 libxcb-cursor0 libxcb-damage0 os-prober python-websocket
python3-dbus.mainloop.pyqt5 python3-icu python3-pam python3-pyqt5
python3-pyqt5.qtsvg python3-pyqt5.qtwebkit
qml-module-org-kde-kquickcontrolsaddons qml-module-qtmultimedia
qml-module-qtquick2 rdate sntp tasksel tasksel-data
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 503 not upgraded.
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see pypa/pip#5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
WARNING: The directory '/home/nvidia/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
ERROR: Could not find a version that satisfies the requirement ansible==5.* (from versions: 1.0, 1.1, 1.2, 1.2.1, 1.2.2, 1.2.3, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.4, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5, 1.5, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.5.5, 1.6, 1.6.1, 1.6.2, 1.6.3, 1.6.4, 1.6.5, 1.6.6, 1.6.7, 1.6.8, 1.6.9, 1.6.10, 1.7, 1.7.1, 1.7.2, 1.8, 1.8.1, 1.8.2, 1.8.3, 1.8.4, 1.9.0.1, 1.9.1, 1.9.2, 1.9.3, 1.9.4, 1.9.5, 1.9.6, 2.0.0.0, 2.0.0.1, 2.0.0.2, 2.0.1.0, 2.0.2.0, 2.1.0.0, 2.1.1.0, 2.1.2.0, 2.1.3.0, 2.1.4.0, 2.1.5.0, 2.1.6.0, 2.2.0.0, 2.2.1.0, 2.2.2.0, 2.2.3.0, 2.3.0.0, 2.3.1.0, 2.3.2.0, 2.3.3.0, 2.4.0.0, 2.4.1.0, 2.4.2.0, 2.4.3.0, 2.4.4.0, 2.4.5.0, 2.4.6.0, 2.5.0a1, 2.5.0b1, 2.5.0b2, 2.5.0rc1, 2.5.0rc2, 2.5.0rc3, 2.5.0, 2.5.1, 2.5.2, 2.5.3, 2.5.4, 2.5.5, 2.5.6, 2.5.7, 2.5.8, 2.5.9, 2.5.10, 2.5.11, 2.5.12, 2.5.13, 2.5.14, 2.5.15, 2.6.0a1, 2.6.0a2, 2.6.0rc1, 2.6.0rc2, 2.6.0rc3, 2.6.0rc4, 2.6.0rc5, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.6.5, 2.6.6, 2.6.7, 2.6.8, 2.6.9, 2.6.10, 2.6.11, 2.6.12, 2.6.13, 2.6.14, 2.6.15, 2.6.16, 2.6.17, 2.6.18, 2.6.19, 2.6.20, 2.7.0.dev0, 2.7.0a1, 2.7.0b1, 2.7.0rc1, 2.7.0rc2, 2.7.0rc3, 2.7.0rc4, 2.7.0, 2.7.1, 2.7.2, 2.7.3, 2.7.4, 2.7.5, 2.7.6, 2.7.7, 2.7.8, 2.7.9, 2.7.10, 2.7.11, 2.7.12, 2.7.13, 2.7.14, 2.7.15, 2.7.16, 2.7.17, 2.7.18, 2.8.0a1, 2.8.0b1, 2.8.0rc1, 2.8.0rc2, 2.8.0rc3, 2.8.0, 2.8.1, 2.8.2, 2.8.3, 2.8.4, 2.8.5, 2.8.6, 2.8.7, 2.8.8, 2.8.9, 2.8.10, 2.8.11, 2.8.12, 2.8.13, 2.8.14, 2.8.15, 2.8.16rc1, 2.8.16, 2.8.17rc1, 2.8.17, 2.8.18rc1, 2.8.18, 2.8.19rc1, 2.8.19, 2.8.20rc1, 2.8.20, 2.9.0b1, 2.9.0rc1, 2.9.0rc2, 2.9.0rc3, 2.9.0rc4, 2.9.0rc5, 2.9.0, 2.9.1, 2.9.2, 2.9.3, 2.9.4, 2.9.5, 2.9.6, 2.9.7, 2.9.8, 2.9.9, 2.9.10, 2.9.11, 2.9.12, 2.9.13, 2.9.14rc1, 2.9.14, 2.9.15rc1, 2.9.15, 2.9.16rc1, 2.9.16, 2.9.17rc1, 2.9.17, 2.9.18rc1, 2.9.18, 2.9.19rc1, 2.9.19, 2.9.20rc1, 2.9.20, 2.9.21rc1, 2.9.21, 2.9.22rc1, 2.9.22, 2.9.23rc1, 2.9.23, 2.9.24rc1, 2.9.24, 2.9.25rc1, 2.9.25, 2.9.26rc1, 2.9.26, 2.9.27rc1, 2.9.27, 2.10.0a1, 2.10.0a2, 2.10.0a3, 2.10.0a4, 2.10.0a5, 2.10.0a6, 2.10.0a7, 2.10.0a8, 2.10.0a9, 2.10.0b1, 2.10.0b2, 2.10.0rc1, 2.10.0, 2.10.1, 2.10.2, 2.10.3, 2.10.4, 2.10.5, 2.10.6, 2.10.7, 3.0.0b1, 3.0.0rc1, 3.0.0, 3.1.0, 3.2.0, 3.3.0, 3.4.0, 4.0.0a1, 4.0.0a2, 4.0.0a3, 4.0.0a4, 4.0.0b1, 4.0.0b2, 4.0.0rc1, 4.0.0, 4.1.0, 4.2.0, 4.3.0, 4.4.0, 4.5.0, 4.6.0, 4.7.0, 4.8.0, 4.9.0, 4.10.0, 5.0.0a1, 5.0.0a2, 5.0.0a3, 5.0.0b1, 5.0.0b2, 5.0.0rc1)
ERROR: No matching distribution found for ansible==5.*`

Q2:
There is an error when use following command.
rocker --nvidia --x11 --user --volume $HOME/autoware -- ghcr.io/autowarefoundation/autoware-universe:latest-arm64
Following is error log.

`nvidia@xavier:~$ rocker --nvidia --x11 --user --volume $HOME/autoware -- ghcr.io/autowarefoundation/autoware-universe:latest-arm64
Extension volume doesn't support default arguments. Please extend it.
Active extensions ['nvidia', 'volume', 'x11', 'user']
Step 1/12 : FROM python:3-slim-stretch as detector
---> 839514fd3bcb
Step 2/12 : RUN mkdir -p /tmp/distrovenv
---> Using cache
---> 1568774de25e
Step 3/12 : RUN python3 -m venv /tmp/distrovenv
---> Using cache
---> d457dc3a1045
Step 4/12 : RUN apt-get update && apt-get install -qy patchelf binutils
---> Using cache
---> 8633694baf65
Step 5/12 : RUN . /tmp/distrovenv/bin/activate && pip install distro pyinstaller==4.0 staticx==0.12.3
---> Running in ea9bf28a2354
Collecting distro
Downloading https://files.pythonhosted.org/packages/e1/54/d08d1ad53788515392bec14d2d6e8c410bffdc127780a9a4aa8e6854d502/distro-1.7.0-py3-none-any.whl
Collecting pyinstaller==4.0
Downloading https://files.pythonhosted.org/packages/82/96/21ba3619647bac2b34b4996b2dbbea8e74a703767ce24192899d9153c058/pyinstaller-4.0.tar.gz (3.5MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Collecting staticx==0.12.3
Downloading https://files.pythonhosted.org/packages/92/ff/d9960ea1f9db48d6044a24ee0f3d78d07bcaddf96eb0c0e8806f941fb7d3/staticx-0.12.3.tar.gz (68kB)
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-m_nm8mya/staticx/setup.py", line 4, in
from wheel.bdist_wheel import bdist_wheel
ModuleNotFoundError: No module named 'wheel'

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-m_nm8mya/staticx/

You are using pip version 19.0.3, however version 22.0.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Removing intermediate container ea9bf28a2354
no more output and success not detected
Failed to build detector image
WARNING unable to detect os for base image 'ghcr.io/autowarefoundation/autoware-universe:latest-arm64', maybe the base image does not exist
nvidia@xavier:~$

`

Could you give me suggestion?

Best Regards

-Shu-Kang

[esteve]

According to https://forums.developer.nvidia.com/t/mounting-of-img-file/128257/8 it should be possible to mount the Jetson image as a loopback device, but so far I haven't succeeded. fdisk reports an offset of 1187840, but neither losetup or mount seem to like it:

sudo mount -o loop,offset=$((512 * 1187840)) sd-blob-b01.img /home/esteve/Downloads/jetson/mnt/
mount: /home/esteve/Downloads/jetson/mnt/: failed to setup loop device for sd-blob-b01.img.

I'll see if I can extract the image via dd and mount it without offset.

[esteve]

More progress, I managed to use kpartx to mount the first partition in the image and chrooted in to inspect the / filesystem. However, running Docker from inside the chroot is not as easy, but I'll continue trying.

[esteve]

According to https://github.com/orgs/autowarefoundation/discussions/255#discussioncomment-2664151 upgrading to Jetpack 5.0 fixes any CUDA-related issues. @jason914 could you try upgrading to Jetpack 5.0 and see if you can run Autoware? Thanks.

[Sharrrrk]

@esteve I just tested using Jetson AGX Xavier with JetPack 5.0 (Ubuntu 20.04) and latest arm64 docker image, after a minor fix 869, build success without errors.
Summary: 195 packages finished [1h 54min 26s]Summary: 195 packages finished [1h 54min 26s]

CUDA and OS version as below:

nv@xavier-5:~$ uname -a
Linux xavier-5 5.10.65-tegra #1 SMP PREEMPT Wed Apr 6 11:45:49 PDT 2022 aarch64 aarch64 aarch64 GNU/Linux
nv@xavier-5:~$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_11_23:44:05_PST_2021
Cuda compilation tools, release 11.4, V11.4.166
Build cuda_11.4.r11.4/compiler.30645359_0

[Sharrrrk]

There is an error when use following command.
rocker --nvidia --x11 --user --volume $HOME/autoware -- ghcr.io/autowarefoundation/autoware-universe:latest-arm64
Following is error log.

Having the same behavior for Q2

[mitsudome-r]

Since it is resolved in JetPack 5.0, I'm closing this issue