"Illegal Instruction" error when converting 7B model to ggml FP16 format (Raspberry Pi 4, 8GB, Raspberry Pi OS, 64-bit)

[lesp]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [ /] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• [ /] I carefully followed the README.md.
• [ /] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• [ /] I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

I expected the command to convert the 7B model to ggml FP16 format

Current Behavior

Illegal instruction error

les@raspberrypi:~/llama.cpp $ python3 convert-pth-to-ggml.py models/7B/ 1
Illegal instruction

Environment and Context

Raspberry Pi 4 8GB
Latest Raspberry Pi OS 64-bit, fully updated

• Physical (or virtual) hardware you are using, e.g. for Linux:
  $lscpu
  Architecture: aarch64
  CPU op-mode(s): 32-bit, 64-bit
  Byte Order: Little Endian
  CPU(s): 4
  On-line CPU(s) list: 0-3
  Thread(s) per core: 1
  Core(s) per socket: 4
  Socket(s): 1
  Vendor ID: ARM
  Model: 3
  Model name: Cortex-A72
  Stepping: r0p3
  CPU max MHz: 1800.0000
  CPU min MHz: 600.0000
  BogoMIPS: 108.00
  L1d cache: 128 KiB
  L1i cache: 192 KiB
  L2 cache: 1 MiB
  Vulnerability Itlb multihit: Not affected
  Vulnerability L1tf: Not affected
  Vulnerability Mds: Not affected
  Vulnerability Meltdown: Not affected
  Vulnerability Mmio stale data: Not affected
  Vulnerability Retbleed: Not affected
  Vulnerability Spec store bypass: Vulnerable
  Vulnerability Spectre v1: Mitigation; __user pointer sanitization
  Vulnerability Spectre v2: Vulnerable
  Vulnerability Srbds: Not affected
  Vulnerability Tsx async abort: Not affected
  Flags: fp asimd evtstrm crc32 cpuid

• Operating System, e.g. for Linux:

$ uname -a
Linux raspberrypi 5.15.84-v8+ #1613 SMP PREEMPT Thu Jan 5 12:03:08 GMT 2023 aarch64 GNU/Linux

• SDK version, e.g. for Linux:

$ python3 --version
Python 3.9.2
$ make --version
GNU Make 4.3
Built for aarch64-unknown-linux-gnu

$ g++ --version

g++ (Debian 10.2.1-6) 10.2.1 20210110

Failure Information (for bugs)

Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.

Steps to Reproduce

Please provide detailed steps for reproducing the issue. We are not sitting in front of your screen, so the more detail the better.

1. Cloned the repo.
2. Changed directory to llama.cpp.
3. Used make.
4. Downloaded the 7B model and copied it to /models/7B.
5. Installed the Python modules torch numpy sentencepiece.
6. Used convert-pth-to-ggml.py models/7B/ 1 to convert the model to ggml FP16 format.
7. Received the illegal instruction error after a few seconds.

[mqy]

Should due to lack of memory.
convert-pth-to-ggml.py loads the entire 13GB models/7B/consolidated.00.pth as pytorch model.

Another similar issue #402

[lesp]

Confirmed that it was a RAM issue. I managed to complete the project on my AMD Ryzen 5600X with 32GB RAM. Copied the files from /models over to the Pi and all worked well.
Thanks for the advice.