Multi-thread ggml_cpy()

[ggerganov]

This is a task suitable for new contributors

See how we multi-threaded the ggml_rope() operator.
Do the same for the ggml_cpy() operator and see if there is any benefit.

Use the ggml profiler (GGML_PERF) to measure the benefit of multi-threaded vs non-multi-threaded ggml_cpy()

[Fs77X]

Hi, this is my first contribution to a large project so forgive me for being a newbie! I attempted to multithread cpy following the code from the rope commit but I started getting garbage output given an initial prompt. Would appreciate any guidance on what I'm doing wrong! Fs77X@3c8a304

[ggerganov]

So I have updated ggml_cpy() in the latest commit (c3ac702) since I backported some changes from whisper.cpp - we can significantly improve the performance when dst is contiguous. The code is currently quite messy, but I think it can be easily simplified in the future.

While at it, I tried to multi-thread it and didn't observed any measurable improvements, so I guess there is no point in multi-threading it.

I will close this issue now

[slaren]

@ggerganov Do you still have your multi-threaded implementation of ggml_cpy? I would like to build on that to parallelize quantization in ggml_cpy for LoRA.

[ggerganov]

No, but it was pretty much the same as: https://github.com/ggerganov/llama.cpp/pull/824/files
Just use some shorter variable names like:

    const int ith = params->ith;
    const int nth = params->nth;

    int ir = 0;

[shkim-emily]

@ggerganov Hello, Does [ggml profiler (GGML_PERF)] not support now?