llama-bench : add model sizes

[slaren]

Renames llama_model_type API to llama_model_desc, adds llama_model_size and llama_model_n_params APIs to llama.cpp.

Currently, the sizes are always shown in the markdown output. I am ok with that, but if it adds too much clutter, I could make them optional.

Example output with markdown:

Device 0: NVIDIA GeForce RTX 3090 Ti, compute capability 8.6

model	model_size	model_n_params	backend	n_gpu_layers	test	t/s
LLaMA 7B mostly Q4_0	3.56 GiB	6.74 B	CUDA	99	pp 512	2235.89 ± 34.61
LLaMA 13B mostly Q4_0	6.86 GiB	13.02 B	CUDA	99	pp 512	1326.61 ± 100.20
LLaMA 30B mostly Q4_0	17.09 GiB	32.53 B	CUDA	99	pp 512	619.07 ± 2.03

build: d0f77b1 (1055)

[ggerganov]

Maybe change model_size to just size and model_n_params to params
Also GiB -> G to make the table a bit more compact

[SlyEcho]

Is it possible to add this kind of metadata to GGUF?

Never mind, this is all calculated.

[slaren]

How about something like this? Tried to make it a bit more compact, while still keeping the units.

model	size	params	backend	ngl	test	t/s
LLaMA 7B mostly Q4_0	3.56 GB	6.74 B	CUDA	99	pp 512	2239.86 ± 22.42
LLaMA 13B mostly Q4_0	6.86 GB	13.02 B	CUDA	99	pp 512	1379.74 ± 2.01
LLaMA 30B mostly Q4_0	17.09 GB	32.53 B	CUDA	99	pp 512	614.50 ± 2.52

[ggerganov]

The size unit is incorrect.

• 1 Gibibyte is 1073741824 bytes and the shorthand is G or GiB
• 1 Gigabyte is 1000000000 bytes and the shorthand is GB

We want to report Gibibytes so it's better to use the shorter shorthand G

[slaren]

I just have never seen G used to refer to GiB. I have been looking for references for the usage of G and I couldn't find anything. Anyway, it's just two characters, I have left it as GiB for now, it can be changed later if needed.

[ggerganov]

ls -lh reports with G - that's where I picked it up:

$ ls -lh
total 41G
-rw-rw-r-- 1 ggerganov ggerganov  13G Jul 19 15:10 ggml-model-f16.bin
-rw-rw-r-- 1 ggerganov ggerganov  13G Aug 25 14:07 ggml-model-f16.gguf
-rw-rw-r-- 1 ggerganov ggerganov 3.6G Jul 24 16:47 ggml-model-q4_0.bin
-rw-rw-r-- 1 ggerganov ggerganov 3.6G Aug 25 14:08 ggml-model-q4_0.gguf
-rw-rw-r-- 1 ggerganov ggerganov 4.0G Aug 16 15:19 ggml-model-q4_1.gguf
-rw-rw-r-- 1 ggerganov ggerganov 4.8G Aug 14 10:56 ggml-model-q5_1.gguf