CUDA: add bf16 and f32 support to cublas_mul_mat_batched

[am17an]

Add bf16 and f32 to support batched cuBLAS mul mat. Speed up when we do

--cache_type_v bf16 --cache_type_k bf16 when running llama-bench

Model	Test	t/s master	t/s add_bp16_fp32_to_cublas_batched	Speedup
llama 7B Q5_K_M	pp512	3681.87	4807.34	1.31
llama 7B Q5_K_M	tg128	129.40	129.28	1.00

[am17an]

mul_mat_batched with bf16 is failing for ubuntu-22-cmake-vulkan, should I remove the extra tests?

[am17an]

mul_mat_batched with bf16 is failing for ubuntu-22-cmake-vulkan, should I remove the extra tests?

@JohannesGaessler mul-mat tests in bf16 which fail for vulkan because of an assert /home/runner/work/llama.cpp/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:5134: GGML_ASSERT(ggml_vk_dim01_contiguous(src0) || src0->type == GGML_TYPE_F32 || src0->type == GGML_TYPE_F16) failed . Not familiar with the vulkan code, so I'm not sure what to do

[JohannesGaessler]

Sorry, I didn't see the Vulkan comment. The problem from what I can tell is that the logic in ggml_backend_vk_device_supports_op and the assert are inconsistent. What I assume happened is that one of the 2 was changed without updating the other one, so presumably the problem can be fixed by updating the outdated one (and until now this simply wasn't noticed). @0cc4m @jeffbolznv can either of you weigh in on which version is correct?

[jeffbolznv]

I think this was supposed to work, but just changing the assert I see the test fail. I'll debug it.

[jeffbolznv]

#14378 should fix the new tests.